Hi, great to be here. I'd like to talk over the next 10 minutes about our approach to
sustaining performance, not just peak performance. It's been a really busy year for us since
the last ISC. Last summer we launched the Cray XE6 supercomputer, and we've shipped
over five petaflops of systems. In the top 100 market where we really focus, we now have
most machines of any vendor in the top 10 and the top 50 and the top 100. But they're
not all just big machines. We've had quite a bit of momentum now in the little brother
of the XE6, what we call the mini or mid-range system, the XE6M. Most of these systems have
been shipped into research and academic environments. We just this morning announced our latest
addition to the full GE global research buying XE6Ms. And the XE6M basically is the little
brother. It's a cost-reduced version of the XE6, provides all the scalability benefits,
but at a lower price point. We've also been investing quite a bit over the last year in
Europe. We announced last year that we had previously announced at December of 2009,
the start of the Cray Exascale Research Initiative in Europe, where we've done a lot of research
projects going on with a number of different key customers. We now have several Cray Centers
of Excellence modeled after the original Center of Excellence that we built at Oak Ridge National
Lab. A few years ago, we're partnering with a number of customers in several of the big
European projects. And we actually have several of our senior top engineers now located in
Europe working on a number of the key technologies for the company moving forward.
One of the things that we've had happen recently, which is nice to announce, not really announce,
but talk about our highlight here in Germany, is our first flagship system going into a
large German center at HLRS. This will be the first large XE6 system to be delivered
with the next-generation interlogos processor going in a little bit later this year. And
the contract also included the first large cascade system order, which is our next-generation
system coming in a couple of years. Another system I'd like to highlight is at PDC in
Sweden. This started out as an XT5 mini system and has since been upgraded to an XE6 large-scale
system with the next-generation Gemini interconnect. So what excites me most about the Cray product
line is not the top 500 list. The Jaguar system, which is the largest system we've installed
today, it's been a few turns on the top of the top 500 list. But what's exciting to me is that
it is the only machine that's actually delivering petascale science. There are now 13 machines
with a peak of a petaflop or more, but only one machine that is delivering petaflops in real
applications. There are at least five different applications in a number of different areas that
are sustaining above an exaflop in performance. One of them is actually getting higher sustained
peak than LINPAC, which is pretty interesting. If we look this last year at the Gordon Bell
finalists, there were six finalists, four of them, the four on the left, the Adaptive Measure
Refinement, Blood Flow Simulation, which was the winner, Electron, State Material Science, and
Earthquakes all were running on the Jaguar system. But it's not just big science, grand challenges
that are running on our systems. BMI, which is a commercial company, approached Oak Ridge as part
of their HPC industrial partnership program to get time on Jaguar to develop airflow attachments
that fit under the bottom of long haul trucks. And they found a way to achieve improvements in
energy or gas mileage by 7 to 12%. They brought it to prototype much faster than they could have
without the large scale computational fluid dynamics simulations that they were doing on the Oak
Ridge machine. So all of these results were on, that I just highlighted, were on Jaguar, which is
an XT5 system. Last year, of course, we launched the XE6. The XE6 system packs eight of the latest
Optaron many core processors onto a single compute blade, providing a compute node with over 80
gigabytes per second of memory bandwidth and over 200 gigaflops of performance. The heart of the
XE6 system is the new Gemini Interconnect. This is available as a simple card upgrade to the
existing XT5 systems or XT6 systems. It provides a massive increase in the amount of message traffic,
the number of messages that you can send per second, drops down the message latency, provides
global address space capabilities across the machine. And we have customers routinely that are
seeing tens of percent improvement or up to 2x application improvement, just taking the existing
system and swapping out the old C star networks for the new Gemini network. So it's been a tremendous
success. This is an illustration of why a custom network is important and why, to get back to the
earlier question today, why it's worth investing a lot of money to build a custom interconnect. So
this is a chart showing a challenging benchmark. This is the HBCC random ring latency
benchmark. And everybody usually talks about, I don't know if you can see this, the very left hand
point. We all talk about the unloaded latency between two nearest neighbor points in the network.
But when you run a real application on a large number of cores, the latency does something very
different. The red line here is showing what the HBCC random ring latency looks like on an eight
core Mahalem QDR and Fin a band ring. When you get more cores and more contention, latency
skyrockets, when you add more cores, same network, but going from from Mahalem up to Westmere, where
you have 12 cores, the latency situation looks worse. The XE6 system with the Gemini interconnect,
beautifully flat network scaling with a 24 core node. That's the reason why we build custom
interconnects. So what we're planning to do now, the sort of the next interesting thing going on at
Cray is to take our focus on productive, sustained performance and bring it to the world of
accelerated computing. Just last month, we announced the Cray XK6 supercomputer. This takes the latest
interlogos AMD Optaron processor, marries it with the latest Tesla 2090 GPU, combines it with a Cray
interconnect and Cray's compute, Cray Linux environment, our scalable Linux based operating
system to build a real production worthy accelerated compute blade. You can have any number of graphics
GPU compute blades mixed in with your regular pure Optaron based compute blades in the system. This
is a picture of a compute blade. It looks like our normal XE6 is plug compatible with the XE6 system
fits into the Gemini 3D Torus interconnect. But instead of four dual socket Optaron nodes, we have
four nodes with an Optaron coupled with the GPU. You can upgrade this next year to the next
generation Kepler GPU when they come online simply by removing this SXM module and plopping in, taking
out the Fermi Plus and plopping in the, I don't know if you plop so much as screw, the Kepler GPU
module. And the software for the XK6 system is really building on the same software infrastructure
that we have in the XE system. So our full stack of debuggers and performance tools, the job
schedulers, third party software available through our cluster compatibility mode, compilers, et
cetera, are available, but they'll be extended to augment the system and support GPUs. We'll have
PGI compilers for GPUs. We'll have CUDA with the NVIDIA software development kit. But Cray is
investing a tremendous amount of our energy in the programming environment for GPUs. We're putting
a lot of work into enhanced libraries using our auto tuning framework to provide highly performant
GPU based libraries. So codes that use BLAS or other scientific libraries can just transparently
get the performance of the GPUs. We're building our own GPU compiler. Cray has a compiler called the
Cray compilation environment which has a very good performance and also native support for
global address based languages. Now it will have native support for the GPUs. And Cray is working to
provide an open standards based programming interface for GPUs that will be based on
directives. This is activity taking part as part of the OpenMP standards group. Directives for
compilers that will allow you to have one code that can work equally well on CPUs and GPUs. CUDA
is great. You can get great performance with CUDA. But we think long term we need a programming
environment that allows you to carry one code base forward and get portability between machines at
any given point of time and portability forward in time. So what we ask the users to do is simply
expose the parallelism in their applications and then let the compiler and the runtime focus on
mapping that parallelism onto the machines. And what we found is that when you go through the work
to expose the parallelism using this directive based approach, not only will it work well on
the GPU based machines, but you get better performance on the CPU based machines as well.
And so we think this is a much more sustainable path going forward that we're pretty excited about.
With that, I will stand for the Inquisition. Thank you very much, Steve. Our first question comes from
Felix Sherman. Thank you, Steve. You have a reputation to lose, right, with your sustained
performance on all the different terra, petra, and maybe exascale level. Now, a lot of that was in the
past, presumably because you offered the users a simple environment of x86 processors, and now
you're taking a big risk to essentially propose accelerated structures to continue to deliver the
same sustained performance. Now, there's a lot of programming models for expressing the parallelism.
Why do you think yours will work? Well, I think from a programming model perspective,
having an open interface is key, right? You have to have a programming model that's portable because
people, make no mistake, people are going to have to invest a tremendous amount of work
to take their applications from the petascale to the exascale, or even from the terra scale
up to the petascale. My phone is ringing, good grief. But we don't think that that work should
be done in a proprietary way. If you're going to do the work, it needs to be done in a way that has
portability across machines and forward in time. And so we think the right programming paradigm is
to ask users to expose parallelism at as high a level in the call tree as possible, and then let
the compiler and runtime take care of the mechanics. Taking care of the mechanics is not something that
you really want to ask the users to do, it's not pretty, and then it can be very specific to a
particular architecture. So that's why we think it's the right programming model. Ask the users to do
what they can do, and then let the compilers and runtime do what it can do. The question about the
architecture is maybe even a thornier issue. It's not clear that accelerators are going to take off,
how quickly accelerators are going to take off. I do believe that long term, a hybrid multicore
approach is inherently superior to other approaches. You need to have a small number of fast cores to
run serial code, and the vast majority of your cores need to be optimized for energy efficiency.
So I think that GPUs are a stepping stone in that direction. Pretty soon within the next few years,
we won't have discrete CPUs and GPUs, they'll be married together on one chip. But that heterogeneous
nature, I think, is going to be required to hit the energy goals that we have over the next decade.
Next question by Gerhard Vellein. Thanks a lot for your very interesting talk.
Krej has known and did a great job in recent years for scaling both machine architecture and
applications to thousands of nodes. One of the foundations of this success was that you kept the
balance between node performance and interconnect performance. Now putting in a GPU which increases
node performance by a factor of 3, 5, or next generation 10X, how do you address those challenges
in the network? Are you going to increase the bandwidth as well, or are you expecting a large
number of very loosely coupled GPU applications? Right, excellent question. And our response to
that is really twofold. Absolutely, we're trying to increase the bandwidth and we're trying to do
it in the most cost effective way possible. The network that I talked about that came out
last year really made radical improvements to the NIC architecture. Our next generation network
makes radical improvements to the global bandwidth and the overall bandwidth of the system. Great
architecture, I'd love to talk to you more in a different context about that. But the reality is
that flops are increasing faster than network bandwidth, and that's just physics and it's not
going to change, and we are all going to have to get used to the fact that traditional ratios of
network bandwidth per flop are going to decrease. We're helped partway by surface to volume ratios
for codes that are doing explicit 3D communication sorts of algorithms. As you grow the beefiness of
your node, a smaller fraction of your communication has to leave that sphere. There are other codes
that are more dominated by global communication, and those codes are going to be somewhat inherently
left behind. There's nothing we can do about that other than build the most cost-effective
network we can and the most cost-effective global bandwidth and then provide the flexibility so that
only the people that really need the global bandwidth need to invest in it because it will be expensive.