well known to a lot of us, and he's a long year ahead
of the computing center of the Max Planck Society in Guyhing,
was until recently also head of technical director
of the German Climate Computing Center in Hamburg.
And as such, can make a good comparison of both worlds.
So that's why he has also focused his talk
on a comparison of these general purpose aspects
and the focus computing center like DKRZ.
So our next speaker, Stefan Heintzer, please.
Yes, good afternoon, everybody.
Yes, I try to compare both centers, a general HPC center
and a dedicated HPC center as climate centers.
As you know, currently, in general, HPC applications
are confronted in severe improvements, especially
by the new architectures that we see on the horizons.
So at first, I try to show you and to summarize
the challenges that applications has
to handle concerning by technology and by, especially
by hardware and system software.
Later, I try to elaborate on the dedicated needs of the HPC
centers for science models.
I would like to thank my colleagues at DKRZ
and Max Planck Institute of Metology
and ACMWF for the input of this talk.
To understand the challenges of upcoming applications
in our sciences and in general for other sciences,
we have to look to the Moore's law.
And as we know, Moore's law is holding,
but we have to reinterpret the Moore's law.
In the former time, you have a clock rate,
and the clock rate growth is over.
And nowadays, the performance improvements
are now coming from the increase of the number of cores.
And although we know that we have in the near future,
in one node, maybe 64 or 512 threads per node.
And this implies a dramatic change
of the applications in the future.
Amr's law exists, and we see currently
that the million core systems are on the horizons.
And even today, if you see the systems in Lawrence Livermore
in Argonne, in Oak Ridge, or here in Yurish,
the current systems have already 10,000 of cores.
And the upcoming systems are already designed and announced.
And you see here the blue water system
of the IBM Power 7 architecture.
You see the Sequoia system in Lawrence Livermore
and the reading project.
And all will go in the direction of nearly a million cores.
So the improvement of the existing codes
will become complex and partly impossible.
As you know, a flat MPI doesn't work anymore.
So you have to implement a shared memory in the node.
And this implementation of these new programming methods
in existing large applications has not always
a promising perspective.
So we are expecting that you have to design or to develop
new, brand-new applications in different areas.
First, I would like to stress the impact
of the different new architecture on the applications.
We will see hundreds of nodes, as I already mentioned,
this may be transactional memory for atomic updates.
You see new microarchitectures for latency hiding.
You will see prefetch hints.
Maybe some of these new architectures
in the new technologies will be hidden
by compilers or libraries.
But there is no question that most of the applications
have to implement the new SMP programming
model in the different applications.
There's another issue concerning the interconnect.
I would expect that applications that
have to scale higher than 10,000 core
need additional networks for barrier and global operations.
They need latency near physical limits.
They need bandwidth over petabytes.
And to do this, this is not possible
if you map the MPI task directly to possible threads.
This is nearly impossible since you
have enormous memory problems and interconnect problems.
So new communication patterns have
to be adapted in the application as well.
Besides these implications from the processor nodes,
there are other impacts from file systems.
The climate application, for instance,
use normally heavy IO.
And for these applications, you have
to adapt the new IO methods in the applications.
There are other implications from the operating
system and the middleware.
Hopefully, most of them are hidden for the applications.
As special synchronization methods as OS jitter impacts.
But I see that at least some application has
to adapt the page sizes dynamically.
And I would expect that application
has to write their own checkpoint research files
since operating systems will not be able to do it efficiently.
What are the challenges for the HPC centers in our science?
Of course, as in most of the science areas,
we will see models of higher resolution.
And we have already heard from my previous speakers
that ensemble run will become an important issue as well.
This is not quite unique.
You will see it in life sciences, for instance,
as well in cosmology.
But it is, for instance, less important in plasma physics.
But this is a feature to run multiple model
and ensemble run on the same machine
will have an impact of the overall IO system
of the upcoming teraflop and petaflop systems
in the future.
The next problem is already mentioned
for my previous speaker very often.
But I try to emphasize this as well.
This is that models are of high complexity.
And if you see the European benchmarks in data
and in place, climate models and weather forecast models
are the largest application of our benchmark suites.
So this is really quite common that we have here
a very complex situation concerning applications.
And as you know, it has, of course,
to do that in the meantime, we are using coupled models
with different components.
And in this context, the application
will become more and more complex
to see the characteristic of these applications.
Models are involving multiple model components
to include more precise physical, chemical,
and biological processes.
And besides this, you need a coupling software
to combine these components.
And a third very important issue of all this climate
application is that all models have
to deal with a huge amount of data from input data,
for data for verification, and the enormous amount of data
for the resulting outputs.
Part of the components can scale very well, I will show you.
But of course, this is quite common for other models as well.
But if for climate models, it is more important
that stronger CPU is much more important
since non-decomponents save time dimension.
One critical issue is that you have,
if you couple the different components,
typically as very severe load imbalance problems.
And as you can imagine immediately,
the cabbler is a source to destroy your scalability
for the overall application.
We have already identified that climate models rely
on interconnects that allows efficient any-to-any processor
communication.
And since data is so important, the data output IOSAP system
essential or a well-performing IOSAP system must be available.
Here is only one example of an outcome application.
There is no doubt that a component of a global model
can scale very well.
This is a different story if you couple all
these different components.
I was discussing with colleagues from Max Meinke Sittel
from Montalo, mythology, the typical bottlenecks
they can identify in the current systems.
That is, of course, as already I mentioned,
the load balancing problems and the algorithmic problems.
Then they identify the scaling of collectives,
the communication latency, and the synchronization
and network is an additional problem.
OS and daemon jitter impacts is less important
since we are expecting that only over 10,000 of cores
this phenomenon will become important.
Data services, my previous speaker
have already mentioned this.
This is a very important issue for climate research.
So typically, all HPC centers in our sciences
have to store many petabytes of data.
So at DeKalb Z, we see in the next years
currently in the order of 10 petabytes per year.
And we have to emphasize that computing power
is very important, but data handling
is at least as important as computer resources.
For this reason, we need the possibility
that data can be moved between compute system,
archives system in a very efficient way.
Therefore, a tightly coupled archives system
is essential.
Furthermore, it is not only helpful
that we can store the raw data we have
in the workflow of climate runs.
We need a step to add metadata for the data
so that we can guarantee a better analysis later
with the data.
So for this reason, most of the HPC centers in climate research
have additional group that are responsible for the handling
of the data in a higher level.
And at DeKalb Z, the group model and data
is responsible for this.
And one of their expertise is, for instance,
one of their results is that they are responsible, first
of all, data climate database.
This is an activity since 2003 and operates more than a decade.
And they are planning that they are store
nearly one petabyte per year in the summer in 2009.
There are nearly 800 active users in 2008.
And they have 2 million data downloads.
Many national and international projects are involved.
Many experiments, a lot of data sets
are handled with this climate world data climate database.
And this database plays a role in the IPCC runs as well.
So Germany and DeKalb Z with his model and data group
is deeply involved in the IPCC runs and will contribute
to the IPC data federation in the next few years.
Only I think I can skip this transparency.
My previous speakers have already
mentioned the impact and the ideas of the IPCC runs
and how many models are involved in this IPCC runs.
I would like only to mention two examples here in Germany
and the idea and the resources for this run.
Say here at DeKalb Z, they will use 80 P6 nodes
with 32 cores over 400 days.
And they will produce nearly two petabyte output data.
And there's a second model with the Reno and the CCLM model.
They will produce nearly four petabyte data in the next years.
To summarize, our system models will
suffer like other applications caused by the inflation,
of course, in future architecture,
especially since the complexity of the models
with the different coupled components.
The development of our system models is complex
and takes typically nearly a decade.
Maybe I'm wrong.
Joachim was mentioning that he's expecting nearly two decades.
OK.
We need an overall support of the system models.
And for this reason, a strong synergy between theory
and computational science up in Nezu is necessary.
So beside the application support,
we need a tight, efficient data service
to cover the HPC system and the petaflop systems.
And last but not least, we need, and this I didn't mention so far,
we need strong support for the resulting data
by graphics and other methods.
Thank you very much for your attention.