Okay. My name is Oriel Torres.
I am one of the founders of the company Power Robotics.
I will explain you a little bit about
navigation in a crowded environments,
which is one of the main topics that we are working on.
First of all, I want to
explain you a little bit about our company for those who don't know about us.
We are a company that started in 2004 creating biped robots.
Biped robots of one meter, 50 of height.
We created for four years,
we were working out with them.
Afterwards, we went to real world scenarios and then we removed the legs of the robot.
So because legs, they are very nice.
Everybody is dreaming about to have a robot with legs,
but if you put a one meter 70 or one meter 60 robot with legs and it falls
down it's okay, there can be a problem there.
So, but we have specialized on human size and human shape robots.
What I want to talk about?
I want to talk about crowded environments,
ramigation in these environments,
what challenge are in this kind of navigation?
Future proposal for this kind of navigation and environments.
First of all, I want to talk about navigation in what is crowded.
I want to say, what do we consider crowded?
This is a robot rim.
It's one of our robots that it was navigating in a fair in 2011.
It was providing information to visitors and it was trying to interact with different ones.
It was working autonomously.
But people interact in some strange ways.
The robot is performing a task and nobody cares about it,
or one cares about realize that robot is there,
push the screen and stops the task that it was performing.
Everybody works or make tasks with robot in different ways.
But if you think this is crowded,
if you think this is crowded,
you will see what happened last week in Barcelona Science Museum.
The robot rim was working autonomously for one week in Science Museum in Barcelona.
Now, you see the robot is here.
It's trying to reach a goal.
It was going from one point to another point in the Science Museum.
Usually, if we were with nobody,
it takes about less than one minute to arrive from that point and reach a goal.
It took about 10 minutes to reach a goal.
Because people were stopping, taking photos, taking pictures.
All the children, they wanted to play with the robot.
This is a nightmare of crowded environment.
But finally, it reached a goal.
What did happen here?
It's not only about navigation,
it's not only about how the robot delivers with a lot of people and different variables.
It's also about robustness.
Children were pushing the sensors to try to destroy the robot.
They were hitting the robot.
This is crowded and this is real world.
You will find when you put a robot in real world,
and they didn't realize that someone was monitoring all these things.
They performed very strange things with the robots.
What is the goal of navigation?
To make the robot move by itself autonomously,
in a human environment without any human intervention.
This is crucial.
But what is important here, what makes it crowded, is the human environment.
You can have a human environment with one person in one room,
you can have several persons,
you can have thousands of persons in the same place trying to interact with the robot,
and the robot has to interact with each one or say something to each one,
say, okay, now I'm performing a task for the person who is beside me,
and I cannot deal with you.
You have to wait for the next, for one minute,
or I can provide an information to several persons in the same room, for instance.
This is important.
Look at the face of these children.
I know that quesera is for Asian people,
but these old people, they have grandsons,
and they will come to play with the robot.
They have a lot of bad ideas, these children.
They will interact with the robot.
They will alter the environment.
They will move things from, they will move the chairs,
they will put the chair in front of the robot to see if the robot can deal with that,
and they will perform any non-preview action.
I have two kids, and when we were in the science mission,
they came to visit the robot,
and my two kids, they have seen the robot so many times,
they sit down in front of the robot and start playing with the robot,
but like another kid, if the robot would be another kid.
This was not preview at all.
The steps to perform the navigation.
Here in University of Hamburg, you know very well,
because you are working with navigation,
but the first thing is to create a map of the environment,
locate the robot itself, plan safe trajectories,
and avoid obstacles as they appear.
This is quite important, these safe trajectories.
I mean, when you deal with a lot of people,
all the movements of the robot have to be safe.
In the science museum experience that we made,
the robot had enabled the possibility to make movements with the arms.
It couldn't at any time make these movements,
because there were so many people around it that it couldn't move their arms.
Okay, so you have to take into account these safety issues.
Each one of these steps has different challenges.
For instance, to create a map.
The creation of the map has to be done autonomously.
If there should be a person creating its time, a new map,
then it's a cost, so it should be done autonomously.
The map should be working for a long time,
and another challenge is the mapping of large environments.
What I mean with large environments, it's about more than 10,000 square meters.
This is what we consider a large environment.
This is a map of a 60,000 square meter center.
To localize, it has two main challenges.
First, the initial localization, where the robot is.
When there are a lot of people around it,
it's difficult to find what is the initial position.
This is done, okay, we are working with laser scanners,
and this is very difficult to find in a big map if there is plenty of people.
Maintain localization in crowded environments.
A lot of people moving around, the map is changing all the time,
so it's one of the challenges to maintain this localization.
Path planning.
This is planning large environments.
It's one of the first challenges.
I know that you are working more in a small environment,
but you can think also about multilevel environments.
In a house with different levels.
Another challenge about path planning is
the robot has to preview where people will work.
So it has to preview, which is the action of the people that is coming
and then change the trajectory for a safe trajectory.
It has to take profit of the human in shape.
So it has to be if the robot can move the head to have some more information,
for instance, for visual mapping, or can move some part of the torso
to know more about the environment, we have to take advantage of this.
Finally, about obstacle avoidance.
There are two main challenges to increase the detection of obstacles
and the distance of this detection.
And handle with strange situations.
Even you can imagine the best robot ever with the best sensors.
The robot will find it will find a problem sometime somehow.
Maybe because the environment has changed in that moment,
maybe because a kid has come and has put chairs all around the robot.
And then the robot has to handle with strange situations.
And has to know what to do with that.
And also, we have to think about multiple robots in the same environment.
Because it can't interfere the sensors of different robots in the same environment.
Finally, I just want to talk about one slide out our robot.
What with which sensors we are working with?
We are working with cameras for person detection, face detection, object recognition.
We are working with proximity sensors for obstacle avoidance.
We are working with lasers for map environment or lasers also for obstacle avoidance.
But the future.
One of the feature environments where the robots will have to work is in domestic.
Like you have been working with inside the Kecera project.
And so this is a simulation of home environment.
So we use tools for simulate home environments.
New sensors will be required.
So to put a kinetic type camera, for instance, we put it in there as a helmet of the robot.
Okay.
We have to, but the robot has to learn some abilities.
Has to, for instance, to open doors.
Has to learn to recognize actions like the falls that we were seeing before.
Object recognition.
Has to recognize important objects inside the domestic environments.
And improve the interaction between persons and the robot.
Through speech recognition, through face detection, face tracking, image recognition,
environment, the interaction is critical.
But what is also critical is to test it in real scenarios.
This is not a real scenario.
This is a scenario we created to test the robot.
But this is the step before.
So we have to, okay, we can create simulations.
We can deal with the data from the sensors.
But the next step is to create scenarios where a robot can be working.
And then the next step is to put the robots in real world scenarios.
And real world scenarios from our experience with all the robots that we have created in
part robotics changes everything.
So even for the bipeds, for the rim series of robots, real world scenarios, they change a lot
of things because you cannot imagine how the people will interact with the robot.
So this is quite important.
And that's all.
Thank you very much.
Any question?
How many ultrasonic sensors do you have around the robot?
We have 15 ultrasonic sensors.
But, okay, we would like what we recommend, for instance.
We like to work with a sensor fusion.
So as the lasers are, we got two lasers on the robot.
And the lasers are in the frontal part of the robot.
So we always try to move the robot in one direction, going forward.
And not going backwards because there are less sensors.
Yeah, the question was how many cameras and what kind of cameras do you have?
Okay, we got stereo camera.
Well, we got a synchronized stereo camera on the head.
And we got one back camera here because we want to know if, because one of the features
of the robot is the guide, okay?
So we want to know if the people guided is following the robot.
So we got one very basic camera here on the back of the robot.
And also the stereo camera.
We use it for visual grasping or more complex things, okay?
Are you still using the color information for the depth sensor device or just using it for depth information?
For depth information.
But, okay, this is, we are currently developing this kind of applications.
And for now, just the depth information.
So you said that when the robot navigates in the cloud environment and we see a person
coming, then it should predict what the person will do.
So do you, I mean, if this is a research topic, do you do that?
And if yes, if it's more, then also...
No, no, I can, we are working on that.
We are trying to preview the movement.
There are two kind of, okay, there is an obstacle avoidance part.
So if the action of the person is not previewed,
it will work the obstacle avoidance sensory system, okay, to avoid the collision, okay?
This is what's happening in the science museum.
So it was impossible to preview any movement because it was the robot camera.
There were 40 or 50 persons around the robot.
But when the robot see one person coming, it doesn't wait until the person is in front of it.
It avoids it, change the trajectory to get them to the goal.
So we are working on that, yeah.
I asked you if the management of the children and the persons in wheelchair is a difficulty
or it is also solved by the stereoscopic vision you have already.
So because the robot is high, the children are small and person in wheelchair could be also small.
Okay, these children, they were not treated as persons, they were treated as obstacles.
Okay, so now that's it.
I mean, and then, okay, there is one challenge on that for human-robot interaction with children
and people with wheelchair is that the screen is on the torso of the robot.
So this is one of the main interaction systems.
So this is one of the challenges that we have to face.
The way to interact with people is quite difficult because
also we are doing research on speech recognition.
But okay, I have not put the sound of silent vision but it was like, imagine.
There were 50 people around the robot but I guess there were about 100 people talking
and in Spain we talk very loudly and it's quite impossible to make a speech recognition.
So the touch screen is one of the main interface tools for people
and it's one challenge to move it to different heights for children or for wheelchair, for instance.
So, questions?
And about the laser, is it not dangerous for children if you have children around the robot?
Okay, if the children just, okay, it's about, I think it's two centimeters or three centimeters
on the floor, okay. They have to do very strange things to be dangerous but they will,
they can do it. But okay, it's true that there should be some safety measures
and people should be aware of that. But okay, we would like to be working with
Visual, for instance, for Visual Islam, for instance. But it's quite difficult in some
buildings, for instance, to work with Visual Islam. I like hearing how they were working
with a sailing camera to detect the persons and the robots and everything. But if you go to an
exhibition center that the sailing is about 15 or 20 meters high, then you should have a very good
camera too. And it's in public areas, it's quite difficult. So we have to deal with these laser
problems. But, okay, we try to avoid as much as possible the possibility of interaction
with children with laser.
Okay, no, there are two lasers, one on the bottom and another one here
to detect obstacles. But it's not only about tables. Imagine about stairs that they are going down.
So a laser will not detect that. An ultrasound will not, there will be, there is no obstacle there.
So, but there is an obstacle because the robot cannot move there. So it has to be detected.
Okay, the easy ones are the obstacles that are there. The difficult ones are the ones that are
not there. Okay. One last question, perhaps.
So maybe you say that the navigation that you, how did the robot actually localize itself wasn't
based only on the sensors of the robot? No, one of the, okay, one of the premises, one of the
conditions that we are very hard on that with ourselves is that we don't want to install
anything on the, on the client's house. So the robot has to compute by itself,
everything, and it has to, should have all the sensors by itself. Okay, it's possible to connect
to any other sensor, it's possible to, but then the robot should work by itself. And this provides
also a high degree of autonomy in the way of installation of the robot.
Okay, thank you very much. You're welcome.