Three very interesting distinguished speakers.
The first speaker is Mark Oliver Gewaltik, Dr. Gewaltik.
He received his PhD in 1999 under the supervision
of Christa van der Malzburg in Bauchum in Germany.
And he then went on to work for the Honda Research Center
in Offenbach.
It had a lot to do with Ashimo and developed
very interesting models along, of course,
with the already mentioned Nest Simulator, which
he co-developed with Markus Diesmann,
and which is in heavy use all around the world,
including Nakamura Sensei's lab.
Yeah, he is, at the moment, like myself,
very much involved in the Human Brain Project.
And he will report on some recent results in that project.
Please join me in welcoming him.
Thank you very much, Alois, for the nice introduction,
also for the organizers for inviting me and giving me
the opportunity to present the work that we did so far
in the Human Brain Project.
And I would like to start with a few words
about the Human Brain Project, just for everybody
who hasn't yet read the newspapers.
So it is one of the Fed flagship projects.
Two of them were awarded $1 billion each.
That is 500 million euro from the EC and 500 million
from matching funding, basically.
There are two projects, Graphene and the Human Brain Project.
We are currently in what is called the ramp up phase.
And this is one of roughly five consecutive phases, which
will last two to two and a half years.
There are 112 partner institutions right now,
over 24 countries.
I, myself, am located in Lausanne, which is in Switzerland.
And there is a little arrow that you might see up there
where we are.
And the Human Brain Project is a 10-year initiative
to understand the human brain.
And the first question, of course,
we have to ask ourselves is, what does it actually
mean to understand the brain?
We all have different expectations here.
And this is probably the pure fact numbers
that you would get if you just look at what the brain is
actually made of.
So it has a certain energy value in terms of calories.
It's mainly fat, if you look at it, or protein,
depending on what you're interested in.
There are some numbers which are more interesting,
but which are also more debatable,
which is that you have roughly 2 and 1 half million gigabytes
of memory, depending on whom you ask and how it is measured.
You have some 80 billion neurons in your brain,
and some 10 to the 15, that's roughly 1,000 times
as many synapses in your brain.
Again, the numbers are dependent on whom you ask
and who had measured it when.
Also, there's a large variability.
The last two numbers that you see here on the bottom,
they're actually, from a computational perspective,
one of the most interesting ones,
namely that your brain,
which has tremendous computation power,
runs on 20 watts of power only.
This is unmatched by anything.
The K computer that we've just seen
in Professor Nakamura's talk,
it uses, I don't know, how many megawatts of power,
similar machines that are in Europe or in the States,
they eat even more.
So we are orders of magnitudes away
from this energy efficiency.
And also, if we look at the size of our brain
compared to what computers that are maybe matching
in computational performance have,
it is tremendously small.
So this would be one aspect of understanding the brain.
How can you get so much computation power
with 20 watts of energy
into roughly one and a half liters of space?
This is one aspect, of course,
all everybody sitting here in the room
is maybe interested in these capabilities,
cognitive abilities that the human brain has,
perception, attention, memory, motor skills, language,
all these are aspects where we would like to understand
how does the brain actually does this?
And much of the research in neuroscience,
computational neuroscience,
focuses almost exclusively on these abilities.
Roughly 15 years ago,
IBM has recognized that there are other abilities
that the brain is actually quite good at
and which are interesting for technical applications.
These are the non-cognitive abilities.
The homeostasis or the housekeeping of our body,
everything which is related
to the autonomic nervous system.
15 years ago, there were this buzzword self-ex properties
and they're all basically relayed
to the autonomic nervous system.
And this is also something that the brain brings about
and much less research has been devoted
on how this is actually happening.
We've seen a little bit of this in Nakamura-san's talk
about stretch reflex or the reflexes around motor abilities
but this, of course, stretches much more
or much further than that.
Last but not least,
there are the things that can go wrong with your brain.
After all, it is an organ that can be sick and ill
like any other organ
and understanding the brain also includes understanding
how the various diseases and disorders connect
to the function or malfunctioning of our nervous system.
So if in the human brain project,
we talk about understanding the brain,
it is a whole package.
It's understanding the brain as an organ,
as a piece of tissue.
So there are three aspects
or three computational, no,
three topical pillars of the human brain project
which are bundled together.
They are roughly called future neuroscience,
future medicine and future computing,
although we shouldn't treat the word future here too seriously.
It's just an indication what the future could look like
and in the neuroscience fork of the human brain project,
the idea is to integrate everything we know
about the brain into computer models and simulations.
Future medicine is contributing
to understanding, diagnosing
and treating diseases of the brain basically
by bringing in clinical research data
into the scientific community.
Currently largely human related data from hospitals,
they are locked away in hospitals
and are not accessible for science as such.
And then finally future computing means
to learn from the brain
how to build supercomputers of tomorrow.
So I won't talk too much about these last two aspects,
future medicine and future computing,
rather I would like to explain a little bit
what we mean by integrating knowledge
into computer models and simulations
and I would like to convince you
why this is actually useful.
Many of you might have the idea
that if you want to learn something from a simulation,
the simulation has to be perfect.
Yeah, you want to integrate your data,
you get a simulation
and this is the day when research can really start.
But in fact, I would like to convince you
that this is not necessarily so.
And as a metaphor,
I always use the first data integration project
that humankind actually did
and that was the creation of the first globe in 1492
by Martin Beheim in Nürnberg
and he created the Erd Abfel,
which is the word he used for the word globe
because globe didn't exist yet
and what he did was he took the various maps
that merchants and sailors and travelers had collected
and tried to arrange them essentially on an empty bowl.
And this procedure was of course very time consuming.
There were no computers at that time
and I had to clean up all the maps that he got
in order to arrive at the picture that you see here.
You can see it still in Nürnberg in the museum.
On the left side, on the upper left panel there,
no right panel, you see a drawing of the same globe
and if you look carefully,
it matches roughly what you see there
from this snapshot of Google Earth
and you will see for example that the Americas are missing.
On the very left side of the globe,
you see a country called Katai,
which is the old name of China
and on the right side you see the outlines of Africa,
Spain and also the United Kingdom.
But the Americas, they were completely missing.
Despite the fact that around this time,
actually news had reached Europe
that there is something in this area
but nobody knew whether it's a continent or an island
or just a few rocks in the water.
Despite the fact, the globe was used
for a couple of hundred years
and the exercise of compiling this first globe
resulted in a consistent view of our knowledge of the world
and likewise, the idea of putting all the data
that we have about the brain will yield
a consistent view of our knowledge of the brain.
Where do we actually know things about
and where are the uncharted territories?
Where are the white spaces on the globe?
For Beheim, this was very easy.
For Europe, you will have a pile of maps.
For other areas, there will be hardly any.
Same thing is true if we look about the brain.
In recent years, there has been a revolution
in imaging techniques,
meaning that it is possible to obtain image data sets
from an entire brain of one individual.
And this allows us actually to approach modeling
of nervous system in a very different way
than we've done it, let's say, five to 10 years ago.
So the idea in the human brain project
is to use these types of data sets
to almost automatically produce whole-brain models.
And the strategy is actually relatively simple.
So you say the first models, they will be very coarse,
but at least you have a model and you can start validating it
against other experimental data,
which is then used to refine the model.
New data will improve the model,
and this way, it brings you into cycle
of model-building, model-validation, and model-refinement.
And since this is largely a technological problem,
integrating the various tools that we have
into something that you can repeatedly run
rather than just focusing on a particular model,
we start here with the mouse brain
because it's much smaller than the human brain.
Also, various genetic methods are available
to stain and to highlight different types of neurons,
different types of connections.
And in the following video that I will try to convince you
how these techniques help us to get at least
draft models of entire brains.
So here you see the outline of a mouse brain,
and the first step is that from staining data,
you can reconstruct the positions of individual cells.
So these are the blue dots.
You can convert these images into density maps,
which gives you the probabilities of finding a neuron
or a cell at a given position.
And this is the first step where you can do virtual anatomy
and compare basically the model generated to the original.
The next step is to use so-called gene expression data
to give the different cells that you have an identity.
So you can distinguish between glia cells and neurons,
between excitatory and inhibitory cells
and various other cell types.
All these data sets are available
and you can integrate them into a common model.
The next step is reconstructing the connectivity
between the cells.
There are different data sets that help us here.
There's one technique called diffusion tensor imaging,
what you see just here,
and the other one uses viral tracer injections
to determine how local regions of the brain
are connected to other parts of the region.
So here you see first an injection site
and then how the connected areas light up.
And from that you can get a consolidated picture,
a model of the brain, of a mouse brain in this example.
And the techniques allow us also to scale the size
of the model to anything that we can handle
on the computers that we have.
Now, we have a mouse brain model now
and the challenging part is to turn this into something
that is useful.
So we have to put the brain back into its natural habitat,
which is the body.
And a good part of what we do in the human brain project
and the neurobiotic sub-project
is creating a body for the mouse brain
and trying to reestablish the different abilities
that the mouse has.
On the left side you see the sensory parts
and on the bottom side you see the motor parts.
And I will walk you through some of the steps,
but due to the time restriction,
I can just highlight the results
rather than tell you exactly how it is done.
So the first step is creating the body here
and we are actually working on a collaboration
with Professor Nakamura to come up with something
that is even better than what I'm going to show you.
So the current approach is very straightforward.
It is basically, well that was a bit too forward,
so this is already the result,
but the steps are basically that you create
what is called a mesh, which is a top figure
that you see, then you create a skeleton inside the mesh
and then you have to connect the two
so that if you move a part of the skeleton,
the body will also move.
Then you have to create what is called collision meshes
from the body mesh.
And then finally create joints and muscles
to have this little avatar here
being able to be controlled.
So this is what you get then.
This is a passive body as you get it.
You see it's a soft body physics,
so the meshes that describe the outside of this mouse
that have to be recomputed regularly.
And this provides then a problem
if you want to detect, for example, touch.
So if this mouse touches the body,
how do you actually determine where does it touch the floor
and how would you then route these signals
back to the brain of the mouse?
So the first is a technological problem,
which is solvable by sufficient computational techniques.
The other one is a biological problem.
So what you see here on the left side
is a typical delineation of the really state of touch
on the surface of a mouse brain.
And the bluish and the green areas,
they correspond to the mouth and the snout.
The reddish parts, they correspond to the limbs,
forelimbs and hind limbs.
And then you have these stripes, which are the whiskers.
And the smallest part, this purple part,
is actually the trunk.
For our mouse brain, we know exactly where these regions are
because the model that we created is annotated.
So for each cell, we know exactly where it sits.
So step two is relatively easy,
mapping this biological data
of the somatosensory cortex
back onto the surface of the mouse brain.
Step three is a little bit more difficult
because here we have to create a correspondence
between the surface of the mouse
and the representation on the brain.
And since the size difference is huge, for example,
the trunk has on the body the biggest surface,
but on the brain, it takes the smallest area.
You have to use mapping techniques from computer vision
to establish at least a global correspondence.
So local correspondence would have to be solved
or could be solved by map formation algorithms
and other learning techniques.
Once you've done this,
you can do the following types of simulation
that you can have the mouse, in this case drop,
and you record where it touches the ground,
and in this case also where an external object
treats the mouse and then observe how the activity
is routed onto the brain model,
which you see on the right side.
You see here neurons that are individually flickering,
they're actually not connected to each other,
otherwise you wouldn't see much.
So it's just for illustrating that technically
this is possible, we switched off the connections
between the neurons.
So this is a somatosensory reconstruction
and this is a prototype.
The next step is to reconstruct also the movements
and this is again something where you want to provide
a technological basis and then fill in the details
of how it is solved particularly.
So in this case, what we have to do is,
first of all, so on the one hand we have the brain model,
then we have the spinal cord,
and then we have the muscles,
and Professor Nakamura nicely explained the relations here
and there's a whole zoo of different models
of how to model spinal cord circuits
and how to model muscles.
We've taken a rather straightforward approach
and said basically, from the perspective
of the mouse avatar here, what it counts is that
you have to produce a talk at the respective joint
and that has to be antagonistic,
so you have two sets of muscles
and you use characteristic functions to do a translation
from neural activation to angle dependent talk.
And these characteristic functions you can obtain
from your favorite model of choice for the details
and with this approach, you can do something like this.
So you can have the mouse do, for example,
this rowing motion here, which articulates exactly two joints.
And what you see on the upper right corner here
is little bubbles lighting up
and each of these bubbles is a population of neurons.
So here we don't model each neuron individually,
but rather we group them into populations
and the rowing motion is realized
by a wave propagating over these population of neurons.
Of course, now you can orchestrate this
and create the following
that you force the mouse to do the stepping here
and then you see that the correct regions
in the somatosensory areas are activated.
Yeah, I have to say here that this is now not
any type of voluntary movement of the limbs,
rather it is really just two sine waves
going into the neural populations of the spinal cord
forcing the mouse to do the stepping motion
and if you just watch this video often enough,
you will see that you get resonance effects
because there is no posture control and nothing else.
It's basically a dead body being driven to move.
So this kind of closes the loop
between the brain and the body model that we have.
Of course, this is a draft.
It has to be filled with details
and a particularly challenging problem is that
the brain model here is largely derived purely from data.
That means as such, it doesn't do anything.
For example, we don't exactly know
the precise synaptic setup of our brain.
So it is basically void of any knowledge, any experience.
Anything that in our brains is acquired during development,
during our growing up phase is missing here.
On the other hand, we have a large set of functional models
that tell us quite convincing theories
about how certain functions are realized in the brain.
And the big challenge is now to bring these two sides together.
On the one hand, we have functional models
which make no relation to the particular neural substrate
that they have to live in.
On the other hand, we have the neural substrate
and we have no relation to the functional capabilities
that it actually implements.
So the big task that we have to solve
is that we have to bring these two sides together.
And this is the research program largely for the neuro robotics
part of the human brain project in the coming years.
So the brain models, they set boundary conditions
for functional or cognitive architectures.
Why is it that a mouse cannot speak?
If we have the constraints taken care of,
then maybe it will help us to select
from the large number of candidate theories
about brain functions for motor task or cognitive task,
the ones that are the more plausible for a particular system
that we talk about.
On the other hand, so the body models
provide the boundary conditions and brain models
set the boundary conditions again for, did I repeat myself?
I guess I did.
So I think I said anything that is to be said here.
So the big problem is bringing the two sides together.
The body model actually sets the boundary conditions
for any type of kinetic and kinematic models
that are to be realized.
For example, if we talk about why a mouse cannot speak,
because it also doesn't have the vocal cords that we have here.
But the body and brain, they relate to each other
and they have to adapt it to each other.
And this we can use to zoom out again a little bit
into the program of the human brain project, namely
the question of how do you embody a brain
and how do you use information about the body models
to inform the brain models and vice versa.
And this is a slide that shows a little bit
where the different sub-projects of the human brain project
come in if we see modeling a particular task as a workflow.
So let's say you want to run a particular experiment
in a simulation.
So you want to replicate an animal experiment
in a simulation.
So you actually have to choose what type of body environment
and task you want to have and what type of brain model.
And you see here this little bubble showing up
with this SP and the number.
And these are the various sub-projects
that we have in the human brain project.
So the brain models here are created
by their own sub-project, which has a number six here.
And once you've done this, the next one
is that you have to choose a desired level at which you
actually want to look at that system.
What I've shown you so far are so-called point neurons,
the same ones that Professor Nakamura used
in the previous talk.
But they're also much more detailed models
which will help you, for example, to understand how,
for example, metabolism links in with neural processing,
which is very important for the medical side at one point.
And then you have to decide where do you actually
want to run your simulation.
There are different platforms that the Human Brain Project
provides from high-performance computer, which
are in Eulish, in Germany, in Spain, and in Italy,
to neuromorphic hardware that is specific chips built
to accelerate the execution of brain models.
And then the biggest problem that I highlighted to you
a little bit, that is what is called instrumentation
and alignment here, that is recreating
the correspondence between areas of the brain
and sensory or motor input and output.
And finally, to use this to map functional and cognitive
models onto the data-driven reconstructed brain
models that we have.
And with that, I thank you very much.
Yeah.
That was a very abrupt ending.
I can go.
Are there questions from the auditorium?
Wow, it was obviously perfectly clear.
So I'll ask a question over here.
Is there?
Yeah.
And you're using the wireless microphone.
The new microphone.
Excellent.
So I thought it was an extremely cool project.
Very interesting.
The question I was asking myself the whole time
was, how would an adult mouse brain
compare to a newborn mouse brain?
And could you discover, using these same techniques,
where learning comes into play?
How structures are developed within the brain
between the two phases of the mouse life?
Yeah, that's an extremely interesting question.
So there's always this question of whether you should actually
grow the brain rather than just doing a snapshot,
like we are doing it.
In particular, if you do a snapshot of a brain
at a certain age, then you're missing out
on the developmental part.
And so currently, your question can't
be answered with these techniques.
This is a caveat that we have.
The difficulty is, if you were to simulate
or to look at the development and also the learning phase
with the same modeling techniques,
you would need several orders of magnitude more work.
Because you would also have to simulate, essentially,
the early life of your animal.
And that is beyond what is even dreamable currently.
So what we have to do, the only chance that we have
is that we do several snapshots at different ages,
reconstruct them, and look at the differences.
And then use what is already known
about learning, map formation, and so on to fill the gaps
and to understand the gaps.
OK, there's one more question I think we can take.
Yes, microphone three.
You seem to have a really complicated model,
but you probably don't have enough data
to justify this model, to fix the parameters of this model.
Isn't that a big problem?
Because as long as your model doesn't
do catastrophic failures, you can never
be sure that it is close or really good,
or at least how well it represents the actual biology.
Well, I would turn this argument a little bit around.
So the way I look at models is often
I see them as a canvas on which you can draw.
And what you're doing with this drawing
is you try to reconstruct a photograph of which you only
have pieces.
And in that sense, your model is a container
for what you already know.
And what you don't know is the missing bits.
But at least you keep everything you know together
in a tight place.
And when the data comes in, you can always correct.
You can always go back.
You can always constrain a little bit more.
Actually, if you think about it long enough,
there is no alternative to this approach,
because everything else would mean
that you accept fate of not knowing anything
and putting in a lot of assumptions
where you typically lose track of.
If we look at these hyper-simplified models
that are around, you don't really know which assumption
have gone into these models.
So they are as wrong and as right as any other.
But they don't even have a place
where you could put in the new data once it comes in.
But you could build up things bottom up instead of top down.
You're now putting up an extremely complicated model,
which is not in any way grounded in reality, which
just has nice computer graphics.
So what we are doing is a combination of bottom up
and top down.
That is the model that you've seen.
It is grounded in reality.
So you can verify it or validate it
against other experimental data.
So we have a forthcoming publication
where this is at least shown for the cell numbers in the brain,
where you can make relatively good predictions
about the cell numbers for the regions
where there is data available.
Of course, for the regions where there are no data available,
you can only put it forward as a suggestion.
But you have a very good idea about how
to integrate the data from various orthogonal sources
to get an integrated model.
And if you look at other branches of science and geology,
the same techniques that have been applied 15 years ago.
There are optical methods to determine,
for example, the composition of drill kernels, which otherwise
needed manual intervention.
And so I would contest your view a little bit
that what we are doing is not grounded in reality.
The orbit is quite well grounded because you can validate it.
You can look at our model, and it
will give you a prediction for an experiment
that you could run, not hypothetically run,
but that you can actually do.
And if you do the experiment, and then it
is a point where you can say, OK, are we closer or further
away from what the experiment tells us?
And this will bring the model forward by refining it.
So you have to embrace the error that is in the models,
because no model will ever be correct.
There will always be faults.
The question is whether it's useful.
OK, stop here.
We have a pre-final question.
There's been some very exciting research
over the past few years and decades,
actually, about the spatial reasoning
capabilities of the hippocampus and the entorhinal cortex,
which incidentally won the Nobel Prize relatively recently.
And I'm wondering, are you incorporating
these kinds of things into your model
because they would seem to connect very directly
to the behavior of the mouse, which
is where a lot of this work was done,
and it connects very directly to the kind
of mapping and localization work that's been done widely
in the robotics community?
Yes, so.
OK, that was the answer.
Yes.
And a final question?
A final question?
OK, so I don't know if it's too late.
Sure, sure, sure.
OK, so the brain of each and every one of us in this room
is very different from each other in a number
of neurons and connectivity.
So how important are these variations?
What makes a model generic with respect
to the model of a single individual?
Yeah, that's actually a very difficult question.
And we are looking at it.
And I think just in the realm of time,
we have to discuss this offline.
But this is a very crucial point, I agree.
OK, so that's the starting point for the follow-up discussion.
Let's thank Mark Oliver again.
Thank you.