Ladies and gentlemen, now we are coming to the highlight of the opening session, the
keynote talk given by Andreas von Bechtelsheim from Sun Microsystems and Arista Networks.
I have thought over how to introduce such a famous man like Andreas to you, and I decided
not to give you the normal CV, quoting his CV.
You can read everything in the web if you like, so I decided to quote from an interview
that Andy has given yesterday in HPC Wire.
This is in fact the English translation of an interview conducted by Schpectrum de Wissenschaft.
This is a sister publication of Scientific American.
And let me just quote two questions, two answers, which should introduce a bit Andreas to you
if you are interested in things, what he has done with Google and so you can read everything
in the internet.
So there was a question, what drives a person who was apparently meant to pursue a scientific
career to take a path that leads him to such exceptional commercial success?
What went wrong?
And Andreas von Bechtelsheim is answering, I don't see any fundamental conflict between
science and commercial success, at least not where I work in Silicon Valley.
All in all, though, I have always been much less interested in academic research and much
more interested in how to build better products that drive a commercial success.
And quite another, there was a question, in your new company, Arista, you are focused
mainly on building network switches.
What pushed you in this direction?
And his answer is, all large web companies are building large data centers for what is
now called cloud computing, a topic we will cover tomorrow morning.
This concept used to be called grid computing, computing clusters or server farms.
There is extensive data transfer among the servers in these cloud compute clusters.
The end result of this computation work, such as a list of search results, doesn't contain
much data, but to calculate the relevance of a website, the page rank, you have to look
through large amounts of data.
The demand for bandwidth rises in proportion to the speed of the servers and the number
of servers in such a cloud.
With 10,000 servers that require one gigabit per second per server, the cloud network has
to move 10 terabits per second, 10 terabits per second.
Of critical importance is that the switches allocate bandwidth fairly to all servers and
connect them with very low latency.
So I hope this was the right introduction for
you rather than giving you the CV which the guys can read in from the web, okay?
Thank you.
No power, and the display projector.
Oh, this is always the big moment, do you get a blue screen or do you get a projector?
So I'm actually going to talk about what we hope to achieve in the next 10 years, which
is to go from the petascale systems today to the exascale or exoflop kind of systems.
And I'm not sure people realize how much high performance computing is driving the industry
these days, so let me start with a slide that may surprise you, but the good folks that
IDC, they keep track of every computer ever shipped, every server shipped, they count
how many of these servers go into enterprise, high performance computing and web, and back
in 2003, 80% of servers by units went into the traditional enterprise data centers, 13%
high performance computing and 6%, 7% web.
Last year, which is the last time they had the full year data, the enterprise market
shrank to 50%, high performance computing was actually one third of all server shipped
and the rest was web cloud data centers.
And the forecast for five years from now is that actually the majority of servers will
go into high performance technical computing, they find the broadway, all compute bound
technical applications, and the enterprise market, which we used to think of the computer
market is actually going to be smaller than the high performance market.
Now how can this be?
Well there's a very simple explanation, which is that virtualization, VMware hypervisors
have enabled enterprise workloads to be reduced to run on fewer and fewer machines.
That's actually a good thing, people save power and cost, but the net is that the enterprise
applications are actually not growing at the same speed as Moore's law.
They don't actually need that much more performance.
The payroll processing next year is going to run at the same speed as this year.
In contrast, high performance computing, the more cycles, the more performance, the better.
So the Moore's law, which is this prediction that things will double every year or so,
really benefits the high performance technical market more than the enterprise market.
And as a result, the predicted revenue forecasts for the high performance technical compute
market are still growing.
So this year of course will strengthen the recession, but again the good folks at IDC
are confident in predicting that this will grow year after year after year.
And if you add up computer storage and service, it's going to pass, what is this, $20 billion
in next year or the year after.
So the high performance market, which you represent here, is actually becoming the largest
market in the broader computer business.
And as a result, has suddenly a strange effect on how people build systems.
So for example, clusters and fabrics, which were really invented in high performance computing,
are now everywhere.
People building Oracle database clusters, storage clusters, and all kinds of grid clusters
using similar, it's not exactly the same MPI, you know, program model, but the same underlying
components as they're being used in parallel applications.
10 gigabit Ethernet, InfiniBand is shipping in high volumes, there's roadmaps to hundreds
of gigabits beyond this.
And what the industry is finding is it's the best way to scale application performance
just as we found in high performance computing.
But let me go back to the beautiful top 500 list from November 2002.
So this was the time when we had the first, what was it, 10 teraflop system showed up
on the top one list.
And if you took the lion from back then, you know, the one that's labeled N equal one,
and then you went straight out to see when would there be a petaflop system, it was going
to be June of 2008.
And right on schedule, on June 2008, it was the first petaflop system.
Now if you notice the blue line in the middle, it's not linear, it does go up as a step
function obviously every time there's a new CPU that makes a big difference.
But if we now take this line, and unfortunately I could not include the data from today since
I didn't have the slides, but if we extrapolate that straight out, it would predict the first
hundred petaflop system around 2016, and the first exaflop or thousand petaflop system by 2020.
Now there's always a good question, you know, is the future so predictable as we make it
out to be, or is this optimistic?
So what I want to share with you in the rest of this presentation is reason to be optimistic
that this is actually achievable.
And again, just to summarize the last 10 to 11 years, it took basically 11 years to get
from a teraflop peak system to the petaflop system, roughly doubling performance every
year.
Again, if this trend continues, we're going to be there on 2020.
And there is this minor problem that Moore's Law, which is the basis of all this advance
of semiconductor processing technology, is in fact predicting a doubling of transistors
every two years, not every year.
Okay, this is a big difference.
We need to double the performance every year to get to the exaflop by 2020.
So lots of challenges here, anything from the chip processing, how to package these
things, how to cool them, how much power this is going to take, how we're going to interconnect
all this, how to do the storage, how to do the software, and how to take advantage of
this parallelism.
So in the next 20 minutes or so, we'll cover all of this.
But let me start with the basic math.
Now to get to more than more, so this is the quote for how to go faster than more,
things have to grow besides the transistors.
And there are some other variables here.
Number one is actually the sheer size of the system.
Now, obviously, this is a function of budgets and power.
But if you have a bigger budget and more power, you can build a bigger system, no question.
What we have seen in the last 10 years is a trend towards larger systems.
And not clear quite why this is, but perhaps the funding agencies are moving some more
money to the bigger things.
But let's just say that trend will continue at the rate of 20% a year, and I need that
number, otherwise we don't end up at the peak of exaflop in 2020.
The course per chip, or compute module, factor two every year is 41% every year.
So that's fairly predictable.
It does depend on investment in semiconductor processes.
So assuming that the world economy does not end and this investment will continue, every
reason to expect this should happen.
Frequency doesn't look so good.
We'll explain why in a second, but let's just say we hope for a 5% increase every year.
Still an important contributor.
Instruction set efficiency, which is how many flops per clock, looks actually a lot better.
And that should be able to increase it an average of 15% a year.
Obviously, this comes in step functions, depending on new architectures.
But if you multiply all these out, yes, you end up at 100% a year, and yes, we should
have the exaflop system by 2020.
Now starting with the semiconductor roadmap, the good folks at the International Technology
Roadmap for Semiconductor Research Institute, they publish this every year.
And there are actually things that are growing faster than twice this year, in particular
flash memory.
In case you haven't noticed, the flash sticks are getting denser and cheaper every month
subsequently.
But the main line, which is the red line going through this, is the Moore's Law prediction.
The solid blue squares, or the CPU transistors, are actually a little slower growing than
Moore's Law, but not enough to get nervous about it.
And again, so between the flash and memory bits on one side and the CPU transistors,
the law is supposed to hold up till 2020 and beyond, we should add.
Now if we extrapolate this back to the present and assume a certain throughput per socket
next calendar year as a baseline in a 32 nanometer process, so assume the 100 gigaflop number,
which may sound modest, but just to emphasize what this means.
If you just add the benefit of increased scores, frequency, and instruction set enhancements,
you would end up with a 10 teraflop socket in 2020.
Put this in perspective, a current rack, like the Sun constellation rack or other competitor
strikes, are roughly 10 teraflop per rack.
So the first prediction here is we're going to have this on a socket.
In other words, if you have your personal mini computer or personal computer in this
time frame, it'll actually be a 10 teraflop computer.
But this is what the math says.
It's just straight extrapolation, and it's kind of a testimony to the power of sustainable
geometric growth over time.
Now again, this does assume that the process technologies enable this density improvement
and the power reduction and all these things to go with it, but this is the straight math.
Now to make this a little more explicit, I did two tables here, or a comparison here
rather, between where we are in 2010 and where we hope to be in 2020.
And again, some of these numbers may not pan out exactly as it's being forecast, but
I think the average will be pretty accurate.
So the current CPUs are at 2.5 gigahertz in change, 5% per year improvement, 4 gigahertz
in 2020.
And by the way, this doesn't mean that people couldn't make faster CPUs, but the problem
with that is they have fewer cores, so they actually don't have more throughput per socket.
So what we're trying to optimize here is throughput per unit and throughput per power.
Now the flops per clock, there could be some real improvements here.
So the data standard is four flops per clock with a little bit of SIMD instructions and
more floating-point units and some vector registers.
We should be able to get this to a sustainable 16.
It's not going to be easy, but that seems doable, perhaps better.
So if you then multiply out the frequency and the flops per clock, we're going from
10 gigaflop per core today to a 64 gigaflop, a nice 6.4 times improvement.
But the real improvement, of course, is the cores per socket.
Let's just say next year somebody has 16 cores on a chip.
Well, using Moore's law, that's going to be 160 cores in 2020 and maybe more.
There's some people that think they're going to have 1,000 cores per chip, but they may
be a little slower.
But let's just say that's a factor of 10, and then if you multiply all that out, here
we are at 10 teraflops per chip.
Now comes the hard part.
Today we get about 30 gigabytes per socket memory bandwidth into the CPU.
Well, if we are 64 times faster, we need 64 times more memory bandwidth just to stay
even.
And obviously we really like to have even more memory bandwidth, but let's just say
we can stay at this 0.2 bytes per flop.
If we don't have the memory bandwidth and all these flops don't do anything useful,
because we're going to be totally memory bandwidth constrained.
So we do need two terabytes of memory bandwidth.
This is just a fact.
Similar I.O. bandwidth today, we're lucky to get a QDR infinity band, three gigabytes
per second each direction to a socket.
We would need roughly 200 gigabytes per second just to stay even.
Again, more would be better.
And today, the whole module, which is CPU, memory, I.O. chips, and voltage converters
is about 200, 250 watts per socket.
It's going to go increase up to the limits of cooling.
People don't think they can cool 500-watt thingies.
We'll talk about that.
But the net is we need substantial improvements in power efficiency, otherwise the power would
be so high that nobody could pay the bill.
So all of this looks challenging and doable.
Now, if you then extrapolate this to the system level using the module performance as a baseline,
today with 16,000 modules at 160 gigaflops each, it would be a 2.5-paterflop system,
which we certainly hope to see next year in 2020, 100,000 modules, 10 teraflops each,
1 exaflop.
And again, the math is very simple.
The question is how to get there.
If you drop to the bottom of this, the power will go up.
There's probably not too many people that would have a 50-megawatt power plant right
next to the data center.
But there's at least some of them, and they know who they are.
They have somewhat of an advantage here in terms of the power cost.
Again, the biggest challenge is memory bandwidth.
How can we actually grow this?
And what you may not have realized is that there have been very few improvements in packaging
technologies in the last 10 years.
Basically we had the same number of pins per chip in 1999 as we have today in 2009.
There was no net progress in 10 years.
This is not a good sign.
What has improved is electrical signaling speed.
People have gone from gigabits per I.O. to 10 gigabits per I.O., and there's some additional
improvements there, but we're kind of hitting a speed limit as well.
So the question is how can we go 100 times faster than memory?
There's only one solution, and this is known as multi-chip packaging.
Now again, this is early research prototypes I'm showing you here, but what people are
doing is piling up these chips on top of each other, either wire bond them or use Vias that
go directly through between the chips to stack them up.
And the beauty of this is that you also eliminate a lot of power, and you get a lot of more
pins because on the chip there's actually a lot more I.O. available than we can bring
out of a package.
So these multi-chip kind of modules are essential.
They're being used today, surprisingly, primarily for consumer electronics, like your average
cell phone or iPod has these kind of chips in it at low power levels, I should add.
So this is an example of an actual chip here where there's a whole bunch of flash chips
stacked up next to the DSP, next to the RF receiver or whatever, and this is just a one-millimeter
thick package that actually includes a dozen or more components.
Now again, today this is used in very low power settings.
The trick is to build this with higher performance, higher power CPUs, and be able to remove the
power, but it seems a doable project, and there's many, many benefits.
So one is we actually eliminate power from the I.O. because when you are on the module,
you need to drive the wires a lot less long.
You can use low voltage levels, less energy required, and you get, of course, much wider
interfaces, many more memory ports, memory channels, and the goal of going to a multi-terabyte
I.O. per second on the module is actually achievable.
Of course, it will require new memory chips and new CPU chips and new interfaces, but
it is a doable proposition.
The other advantages, you can integrate the I.O. in the same module while you're at it.
Remember, today, the pins coming out of a socket is the memory ports and the I.O. ports,
but most of the pins actually is the memory.
If the memory is on chip, we can then use the remaining pins for the I.O., and we will
need all of them to achieve the I.O. scaling.
So going from a QDR today, which is just a couple of service, 10 gigabits each direction,
to these kind of multi-terabyte speeds, it requires a lot more service technologies.
They will go faster, maybe eight times faster than today, but they will actually use up
a large chunk of the I.O. field in the pin grid array.
These services then will drive either copper traces to the adjoining adjacent CPU sockets
or fiber optic cables that go either board-to-board or rack-to-rack, and there has to be enough
gates in here to drive billions of packets per second per CPU to keep up with the scaling.
It would be nice to address all the memory globally while we're at it.
This is doable.
Again, the advantage of doing it on the CPU is we eliminate another bottleneck, which
is the interface from the NIC chip to the CPU chip, which is really in the way.
So to essentially match the networking speed and the internal speed of the CPU, we have
to hook it up directly to the internal memory buses, and it vanishes, no congestion will
receive.
There's no other bus conversions, no gates required, less pins, less power.
The right thing to do, but obviously nobody's doing this today, so this will require some
additional functions to be designed.
And then we get into this whole topic of what's the best fabric here.
Now the answer is economics, meaning whatever we can afford to build that runs the fastest
per dollar or per euro, but it's going to require optimization of the NIC function,
the router function, the optical cables, and most likely, and I'm saying this with some
hesitation because it really depends on the specifics, it's going to be a combination
of meshes and tree technology.
Basically, meshes scale very well to very large clusters because it's a linear scaling,
but the hop to hop count in a large cluster gets really large and not that desirable,
whereas trees have a more constant sort of bandwidth product, and that's great, but there's
a nonlinear function to a tree.
Now if one can combine a tree and a mesh so that there's a sort of a multi-dimensional
mesh or a shorter maximum hop counts, at the right level of combination looks like the
ideal combination combined with dynamic routing so that if something fails or the load goes
a certain way, it can recover from that.
This requires a lot of work.
It may be a new standard because InfiniBand only has so many bits of addressing.
It really depends on what's on the CPU chips, but the current technology will look great
for a couple of years, but we need something much faster for the exaflop.
Now in terms of actual link data rates, today industry is shipping 10 gigabit per lane,
so a QDR InfiniBand is four of those.
We would expect to see 20, 25 gigabits in a couple of years, and maybe twice of that
by 2016.
At that point or beyond that, it's going to be really tricky with electrical communications.
We'll need optics everywhere, which requires yet another type of integration which hasn't
been quite done yet, which is how to integrate the optics directly on the module, but assuming
that can be done and will be done, the speech shouldn't be an issue.
Now let's go back to power for a second because power is a problem, not just because of global
warming, but because it's just so hard to cool things that get very, very large.
In power, everything has to get optimized.
This is a chart that I took from an IBM paper that compared what really happens on the red
curve when you take advantage of the faster transistor and just design the fastest core
versus the blue curve, which is you're trying to keep the power per chip constant, and you're
trying to increase more and more cores per chip.
The problem is when you go on the red curve, the power per core goes up dramatically, like
five-fold in the next couple of years.
That means instead of having a 100-watt chip today, it would be a 500-watt chip, and nobody
can cool that.
Whereas, if you see on the blue curve, the clock rate doesn't go up, but we can actually
take advantage of additional cores.
The answer may be in between.
There may be some modest increase in frequency, but not nearly as fast as it could have been
done with fewer cores, and the net is that it's better to use more cores, more parallelism
to get more throughput than fewer cores, higher clock rates.
Another way to think about this is the power efficiency per throughput.
This is a curve that has a knee, and the bottom of the knee will change over time, so over
time this will move to the right.
Basically, when you go slower than a certain frequency, your power based on leakage current
and other common I.O. and infrastructure will actually go up per throughput.
If you go faster, given the fact that the clock drives power and then also the voltage
square, which you need to get to higher clock rates, drives power by the square, will increase
per throughput.
There's some optimal knee point, which is the one that will deliver the most throughput
per power.
It will depend on time, but this is sort of what people will target in the system designs
to get the most power efficient systems.
What can we do here?
Well, the MCM, again, is the single biggest saving by eliminating the I.O. pins for the
memory and the power required for the I.O. prediction, so we can save half of the power
compared to today just doing that.
Keeping things cool is a good thing because it reduces the leakage current.
Optimizing transistors, materials, you name it, everything is required to reduce power.
Another thing to think about is the CPU architecture, so they are actually fairly complex.
With on-chip, on-module memory, the latencies are much lower, basically maybe 10, 15 nanoseconds
compared to 56 nanoseconds today, which actually means one can build simpler CPU pipelines,
and also by integrating the NIC and I.O. subsystems, one kind of limited a non-true amount of power
in those subsystems, but basically, power efficient design requires optimization at
each level of chip design, packaging, I.O., voltage, process, and the whole industry will
have to work really, really hard to make this achievable for the exaflop generation.
Here's some ideas how to get rid of the power today and to get rid of the power when the
power goes up per chip, so today the limit is in the 100, 150-watt per socket range.
I mentioned earlier we may be looking at multi-hundred-watt sockets.
The best idea is to actually run the liquid right through the socket by having these little
micro-channels, otherwise known as fluidic heat sinks, so there are ways to get rid of
power with liquids.
It may require a lot more plumbing than what we're used to, but it can be removed, and
it's actually the most efficient way to remove it, so one of my safe predictions is that
all future high-performance systems will use some type of liquid cooling, either in combination
with air cooling or pure liquid, and the reason is it's the most power efficient thing to
do.
It also reduces power by keeping things cooler, it does allow people to increase direct density,
which reduces other costs, and most importantly, it improves the heat removal per socket.
Basically the physics of air cooling hasn't changed ever, and we are reaching the limits
of what can be removed with air.
Now just to sort of summarize the packaging here, so the most important issue is actually
reducing these CPU boards that we have to date down to little modules, where the module
is CPU memory and IO on a little thing that may be just a two-by-two square inch thing,
and obviously this means we can pack things much more tightly than ever before, integrate
the IO, the fabric, drive the optics directly from the module.
Now at the system level, this will result in very high power densities, think 100,000
watts per rack or more, but again with liquid cooling, it's actually not a problem to remove
this, and so the physical size of the system isn't actually as bad.
Now let me take a little detour on storage, because at some point we have to save all
these bits that are being generated here.
Today people have 100 gigabytes per petaflop or so, and actually it would be nice to have
more, but that's achievable today.
Now to get to an extra flop system for get-hard disks, you would need a million disks to do
the scaling of the bandwidth, this is not a practical solution.
Solid-state storage has arrived just in time to solve this problem, and what people do
today is to plug in little two-and-a-half inch or three-and-a-half inch solid-state disks
that replace existing disk drives, they have much more IO-ops, they are actually more robust
in terms of they don't have a mechanical failure board, they are unfortunately quite a bit
more expensive per gigabyte, but they're actually cheap on the IO-ops, but there's also new
form factors emerging, at some we built this little flash dim, which is a 3.3 inch wide
micro dim that has the same functionality as a full two-and-a-half inch hard disk in
the dim form factor, and again lots of people building PCI cards that go up to the 100,000
IO-ops level, and very shortly terabyte capacity per PCI card, now what's really driving all
this is this unbelievable increase in density and reduction in cost in flash, so this is
a chart from Gartner, which the important one is the lowest line which says ASP average
selling price per gigabyte equivalent, back in 2004 the gigabyte of flash at the component
level, mind you, was a hundred or more US dollars, this year a dollar and change by
2012, 25 cents, so this is an incredible change in economics, and at the same time of course
the density per device has doubled every year, so this flash business is a really hard business
for the manufacturers, they basically have to output twice as many bits every year just
to stay even, in the meanwhile they have to invest something like 10 billion dollars a
year into new factories to keep up with the new process changes, and they're not making
any money, so I'm not sure this will continue, but assuming this does continue, we'll see a
lot of flash replacing all disk drives very shortly, and so I think it came under flashes,
you know, density doubling each year, cost falling half per year, access times are improving
rapidly, this year lasted was a hundred microseconds, now we're down to 20 microseconds,
there's a new technology now, this phase change that will actually get this down to sub microsecond,
and this means it can be used more and more like a memory, so today most basically all flash
devices are SAS or SATA moving to piece express, but at some point it will be just part of the
memory hierarchy, because it's fast enough to do actually random IO, and the speeds we're going to
see is multi gigabytes per second per controller up from the hundreds of megabytes today, so suddenly
you're not doing a hundred terabyte per second IO with the right parallel file system is actually
not a problem, believe it or not, so the storage will work as well, so going back to scaling here,
so we talked about you know frequency, cores, flops per core, frequency again modus increases,
cores very predictable increases, flops also important increases expected, and so I made a
few more choices to sort of highlight what this actually means, so today 10, roughly 10 gigaflops
for a core kind of 64, to build the exa flop system, the thousand-pater flop on the lowest row,
we would need 16 million cores, it's a large number, you know it's a stretch for the parallelism, but
it's it may be doable, whereas today with 100 million cores, you know I would say all bets are
off, so it is very important to increase the throughput per core to hit the exa flop generation,
same thing with power, today or maybe you know next year we're at a half a gigaflop per watt,
going to a couple of gigaflops per watt, we need to get roughly to 20 gigaflop per watt to make to
keep it to a 15 megawatt envelope, again the exact power number is a little hard to forecast, but
clearly things that take a thousand megawatts or more are not practical, you know this is the power
of a large city, whereas you know some people at least think they have more than 10 megawatts
available, perhaps more importantly on this slide you can notice that your peter flop system in 2020
is down to 50,000 watts from several megawatts today, so there's the same benefits of course
accrue to the smaller systems as well, and then how many sockets or CPU modules to be or we're
talking about here today or maybe next year, you know we're talking something like 6,000, 10,000
modules to hit a peter flop, to hit an exa flop in 2020 would be 100,000 modules, of course the
cost goes up with the number of modules, the reliability goes down with the number of modules,
but this seems like a achievable number that is still in the realm of a very large data center,
whereas today it would take you know many many millions of sockets, it's just not practical,
so again the color coding here is green, I shouldn't say it's easy, but this predictable
yellow is you know getting more challenging, pink is sort of a stretch goal and the solid red or
purple is beyond practicality, bottom line you know there's a steady progression between now
and 2010 that will improve each of these elements, the throughput per socket, the power per socket,
the bandwidth per socket on a very predictable fashion, and as far as anybody can tell, certainly
me, an exa flop system in 10 years is as predictable as the increase from a teraflop
system was to the peter flop system 10 years ago, so to sum this up, we're not going to run out of
Morse law for the next 10 years, it's going to get a little tougher after that, but the next 10 years
as I say in the bank to do more than more we have to have other elements of scaling including
instructions and improvements and some frequency gains and obviously some size gains, the ultimate
challenge here of course is how to deal with all this parallelism and I'm not the software guy but
the software guys know this is a interesting and major challenge and the issue is simply how much
parallelism really exists at the application and as you all know there's some applications that
will go as parallel as the world needs it to be, there's other applications that will never
get to the exa flop but clearly there seem to be enough applications and enough demands out there
based on the current experience that we should expect a good level of scaling there. There's some
other software challenges like how to deal with failures in such lower systems, you know obviously
the MTBI meantime between interruption will go down, on the other hand if we have this superfast
flash and we can recover from a failed node in seconds rather than minutes or many minutes today
that will fix that problem so very fast failure recovery is going to be essential. So in the
summary summary I'm a believer and I know you are too but it is surprisingly how well the top 500
has forecast the growth of the industry and the growth of these systems for the last many years
and it just appears every reason to believe that this will continue at the same rate.
The other good news here is not at the exa flop level but perhaps at the peta flop level the
smallest machine on the top 500 in 2020 is actually it's going to be based on this history
a 10 peta flop system bigger than any system today and just imagine that whereas the peta
flop system is down to a rack or or half a rack and you know you can think of this as your personal
computer by 2020 certainly so it is a very massive change in the industry that really benefits
high-performance computing more than any other element here and this very nice ripple effect
from the very largest of systems down to the medium size and smaller systems. So this is
all the slides I had and I know Russian is a little fast but do we have room to take some
questions or not really? Okay thank you thank you very much.
Are you okay?
Okay I'm up again.
Thank you very much Andy for this exciting talk and we still have five minutes left for questions
and this is a unique chance for you to put your questions to Andy please.
There are microphones.
Please go to the microphone say your name where you are coming from and ask and put your question
please. Okay thank you. Hello my name is Stefan Mel, I'm from Metreonix. There was one detail I
was wondering about. I agree that we will probably be needing optical interconnects
for the ExaFLOP systems. What I was wondering about is what can you tell us about the latencies
of optical interconnects? I know that the bandwidth will be good but what about latency?
Yeah that's a very good question and one thing I forgot to mention is one advantage of not having
highly increasing clock rates is that the relative latency based on clock rate you know is not going
to change that dramatically if anything it should get lower. So it would be different if the clock
rates were going through the roof to 100 gigahertz or something right and the other clockers stayed
constant and suddenly the impact of that latency on the execution would be dramatically different
but one prediction would make is that the relative ratio of ratio of latency to CPU clock rate
independent whether it's copper or optics is going to be a wash. This of course excludes speed of
light effects meaning this is in the rack in the box you know shelf to shelf. In these very large
systems there's actually a non-trivial impact starting to show up that is the
simplest speed of light which nobody can do anything about but there's no I'm not worried
about latency in any of these technologies that's not on my list of worries and I didn't
quite answer your question but clearly optics has some additional conversions and chips that
add a few nanoseconds but they can all of this can be improved and at the end of the day you're
going to see very little difference between optical interconnect and a copper interconnect.
It's a trivial little issue in the big picture here. Okay thank you. Okay thank you. Are there
other questions please Frank yeah please go to the microphone please. There is a microphone.
One comment I had while he's running the microphone is that a real benefit of the top 500 and this
focus on performance over time has been that everything that gets measured improves right
and there's a lot more focus now than perhaps 10 or 20 years ago was on how to increase performance.
So one reason I'm feeling fairly safe about this future growth is that there's almost now a worldwide
competition you know who is going to build the fastest system and how fast is your system versus
my system but the top 500 has done a great benefit to mankind and certainly to high performance
computing have been focused on this. Let's go ahead. Frank this is Frank Betke. You gave us a
lot of insight into future CPU and module designs and the challenge has ahead. Question is would you
expect those new designs also to be based on the x86 instruction set architecture or would you see
advantages real advances moving to a new instruction set or would you say that's pretty
much irrelevant. Yeah that's a very complicated question but a very good one. So number one the
instruction says have to improve in efficiency meaning flops per clock and that's true for any
instruction set right. Number two it would be advantageous to have less power per core so some
legacy things actually not that relevant right but number three and this may be the most important
issue is it comes down to which company who can actually draft the investment required to make
these changes happen. So one advantage that the x86 industry has is that they have all the volume
from other markets to drive these kind of investments which are not getting cheaper by
the way they're getting more expensive but so there's nothing philosophical about this is just
an economic reality of you have to put investment in but another topic I didn't cover is that there
are some kind of chips today that have in fact many more floating point units than the x86 which
is the GPUs or graphics chips so you're going to see GPU chips that are far faster than these
general purpose CPUs that I talked about much sooner but they tend to have limited memory you
know so they're not general purpose in that sense but clearly one could build chips that just have
floating point units and much simplified more like the graphics chips and still do something useful
so I would say the jury is actually out on which chip technology can get the most throughput per
dollar per power per die size and the x86 architecture per se does not have the advantage
natively except the advantage of economics but there may be either simplifications or GPU
extensions or other elements that help us to achieve much higher throughput per watt
than than available today and you know again I can't mention any particular vendor specific
issues here and I try to avoid any reference to any vendors but it's going to be a very interesting
competition in the market between these different architectures so I would definitely say that
jury is still out on that thank you may I also put a question first of all I'd like to thank you
for your compliment for the top 500 we appreciate very much but down to earth let me ask you the
question not 10 years ahead in 2020 when we probably will have an excess flop system
but I have a question a general question how you see the future of some microsystem as an hpc
company when acquired by oracle is this the hot seat section here it's not still the hot
season you can say no I don't like to comment on this but people are interested in this question
yeah no um what can I say about this so um I think a couple of things um the hpc market is
growing so it's an attractive market to to invest in and there's also a ripple effect between hpc
clusters and commercial applications one of oracle's product that you may not have heard about or
familiar with is a system called exa data which actually uses infinity band to cluster compute
notes and storage notes to do massive scaling of database application and oracle certainly
believes that's the future so they see a high degree of leverage surprisingly between hpc
technologies on one hand and commercial data center type technologies so they want to actually
do more of that rather than less of that so can I just leave it at that yeah okay thanks yeah
thank you very much for this exciting talking discussion