For my presentation title, let me talk a little bit about our company.
We started in the 90s with programming rendering software for the entertainment industry.
It's having a plug-in for 3D Studio Max and some other projects.
And that was the time where the Internet business came up.
And we had a lot of clients having a lot of money
and asking us for special projects like avatars on the Internet side.
And so we think, excellent, great opportunity to make some money
because we also did some real-time character animation
and we could offer the clients some very cool avatars.
And so they wanted women avatars on their side
so that their customers could ask this female avatar about their products.
And we made the character, we made the animation
and we needed a software to do the speech and the dialogue processing.
And so we programmed a little chatbot
and then we had an editor doing the dialogue
so that when I asked the chatbot, for example,
hey, my name is Rudy, what's up here?
That the lady can say to me, oh, fine, nice to meet you, Rudy.
We have some cool projects or cool products here.
And then I could talk a little bit to the lady
and maybe get some information about their products.
But mostly talking to this chatbot.
So we had some very cool projects at this time.
It doesn't make really much sense for the companies we did for.
But when we look at the log files,
there was a lot of talking to the chatbots
about if the avatar has a boyfriend
and if one can meet this girl sometime.
And so this was a little bit bad
because there was no real functionality in it.
So we think about, okay, we put some cool technology in it
like parsing and learning algorithms from the artificial intelligence stuff
and maybe we can make the robot more intelligent.
Unfortunately, in 2001, the .com bubble implodes.
So our clients decided not to give us more money to make cool robots.
So we close down this project and go on with normal e-commerce things.
Two or three years later, the first affordable human who eats robots
came out for a few thousand bucks.
And we think, okay, we have a chatbot software.
How can we put this software in the context of a robot?
And then we think, oh, what can we do?
We can make him understand to go over there.
But no one needs this.
So we say, okay, actually, there's no need for a chatbot in the robot context
because you only need some speech recognition, planning tasks,
motor control to act a little bit.
So a few years later, another question arises at the horizon.
So what happens if we get older?
So when I get older, my body goes bad and bad and bad.
So the idea of a service robot is natural.
So we think, okay, what can we do in the context of a service robot?
But because it would be cool if I could say to him,
bring me a beer or bring me a coffee or maybe bring me to bed.
And we think, okay, but this is not our focus
because we are not able to assembly such a robot.
And it makes no sense to have a dialogue engine which accepts these basic commands.
So we think again, and we remember there was a US sitcom in the 80s.
It was about A.I.F.
A.I.F. was an alien landed on the earth and was in the family.
And the only thing I was doing is bringing excitement to the family.
So if we put A.I.F. in a robot, in a service robot,
we would have a robot without service but bringing excitement to the family or to me.
Because my problem is when I get older, one day I will be alone.
So I retire from my job.
Then I meet some friends more often.
And then the older I get, only every few weeks.
And then I get older and older.
My friends are getting older.
So I lose my social contacts.
And then I have a robot. I have a service robot.
So that's great.
But the service robot will only bring me some stuff
and check if I get my pills and stuff like this and make me like this.
And I think, but I'm alone.
And we are not built for being alone.
We want someone around us.
So I can buy a dog.
But a dog is complicated because I have to go out with a dog
and maybe I'm not so good at going out.
So maybe I can buy a cat, but it's not good.
But having something like A.I.F. in our service robot would be a great idea for me.
So I can have someone who's around me and not making A.I.F. me,
but also having his own personality.
So we think about, okay, what can we do or what kind of algorithm we need
that A.I.F. can entertain us?
And what do we have to do to make this work?
And this was our project idea two years ago.
When we said, okay, this is a good idea of making a robot who entertains us,
who brings some excitement to us.
This is our project, Virtual Living Machines.
We started two years ago.
And we said, okay, we need a hat with a mimic, so a face.
And we need all the other stuff you normally need for a robot,
like object detection, recognition, face detection and face recognition,
emotions, speech recognition, common sense knowledge,
learning of user behavior and the behavior engine.
So it's a very big task.
People come to us and say, oh, it's too much for us.
And because each of these topics are so complicated and they were right,
so we said, okay, we look around what kind of frameworks exist outside.
So there are a lot of frameworks out for object detection.
And the whole stuff, which are listed there,
and we put it together and make them work.
And maybe we make it.
So we put together neurons, speech recognizer, object vision stuff,
point cloud library, Kinect, and stuff like this,
putting it together and detect all words that don't work very well.
So because when I speak to the microphone and I give some commands to the robot,
the speech recognizer will maybe, or in most cases, understand me.
But if I have a speech recognition in a dictation mode,
it gets a little bit worse.
So when I go away and talk to the robot, it maybe gets more worse.
So what do we want?
So the question was, how can we make this head acting to us without necessarily
understand us what we are saying?
And so we come to the behavior engine where we have to think,
what can the robot do when he don't understand us?
So and we say, okay, if I have Rob, for example,
this is our robot's head.
It's a model currently we're making the animatronics for it.
What happens when Rob is here and I talk to you and he don't understand me?
And he wait and wait and wait.
And when I'm finished talking to you and maybe say to me,
Rudy, you haven't talked to me for three hours.
And I say, hey, that's not true, Rob.
I talk to you, I ask you a question a few minutes ago and Rob say, no,
this is action because Robert, Rob, forced me to act.
He say something to me, whatever it is, I have to say something to Rob
because otherwise he say, like in Tamagotchi, no, Rudy, that's not fair of you.
I don't talk to you.
And this is, I think, very important thing when we talk about getting older.
We need someone forcing us to act, giving us trigger, giving us input so that we can,
if we stay in contact with social creatures.
So the question is, how can we make Rob know what to do?
Because either if Rob would have a buddy, he would never go out and drink a beer
or make his own experience in real world.
So maybe we have to give Rob this experience.
How can we make this?
When we look at the research area, what is about social interaction with Robert,
we find a lot of things like gays and mimik and gestures.
And we think, okay, this is very cool, but it's only one part.
We have a Robert which is around us for several years maybe.
And for this time, this Robert has to give us trigger input that we can react,
that we do something with Rob.
So we look around and say, okay, what do we have?
We have games.
What do we have in games?
We have stories.
We have movies.
We have stories.
So we have actors having a script and making their story.
So what is we give Rob a script?
What to do?
A story.
And in this story, we put some dialogues.
For example, I say, Rob, Rob, today, three guys are coming to play card with me
at 10 o'clock in the morning.
And Rob say, oh, cool, entertainment.
What will you do for them?
Will you serve them?
Will you cook for them?
And I say, no, no, no.
I don't think so.
So one hour later, Rob came, come to me and say, oh, this is not good.
You have to give them some meal.
And I say, no, no, no.
They drink beer.
I don't want to make some food for them.
So shortly before they come, Rob goes to the door and waiting and say to me,
yeah, it's 8 o'clock.
They come.
I am so happy.
And then the doorbell rings.
Rob knows that now the guys were coming.
I opened the door and people came, came in, and Rob talked to them.
What's your name?
What are you doing here?
And stuff like this.
And then we play a card game.
And Rob maybe asks some questions and tries to participate on the play.
At midnight, people are going home.
And the story for Rob is finished.
Next day, I say again, Rob, friends of mine, are coming.
And then he would restart again this dialogue.
But this would be bad.
So what do we have to do that he makes some variations with it?
You have to store this story with the information you get in this memory.
There are a lot of memory stuff out there.
And it is called the episodic memory where you put stories in.
So we can make Rob that he put the story in his episodic memory.
And again, today, another three people are coming.
And they have different names.
They act differently.
And maybe this time, I cook for them.
So after a few weeks when a lot of other guys are coming to me,
he knows a lot about playing card game with Rudy.
Some of the friends of mine will not talk to him.
Others will talk to them.
So he can have some variations in how he selects the dialogues of this particular story.
If we have a lot of stories for daily activities,
and if we put more and more stories,
we can make Rob that he will stay in contact with me over a very long period of time.
If I have birthday, he can ask, Rudy, you have birthday next week.
What are you planning?
And I can say, oh, maybe some people are coming and stuff like this.
And then I keep in touch.
If I only buy a robot which do something for me,
I would say after a few weeks, it's boring.
But with this type of interaction, I stay in contact because he's a vital living creature,
forcing me to act always.
So this is our approach for doing the service robots without beyond service.
So we don't have the service part, but we have the entertainment part.
So to give you a small outline of the future of Rob,
next year we put it onto a body and put him on the wheelchair
so that he can move around a little bit more faster than the human who eat robots can do,
so that he has the ability to move around.
So when I'm in the kitchen or in the living room and I'm boring with him,
he can move around or he can go out doing something, call me,
and then I have to, oh, Rob, Rob, where are you?
And so he will act with me like a child, a companion, whatever.
And maybe one day he can also play cards with some now robots.
Thank you.
The model is finished.
We are now actually making the animatronics for the heads.
So we have the mechanical parts and putting it together.
Yes, indeed.
He can move the eyes, the eyebrows, the mouth, and can make something like this.
Yes, because he has to look away if you're not good with them.
We put one camera into the eye and we put another, we put it connect over his head
a little bit beside the head.
This version stands on the table and is connected with the power.
And we have a PC doing all the processing.
We don't put the software on an embedded system currently.
Maybe one day, but not currently.
Yes, we have a state-based approach for the storyline.
So when you have a story like, for example, playing the card game,
you think about how can we start the story.
When I say to friends of mine coming this evening for playing cards,
I can put this in a dialogue and triggering the start of the story.
So then someone has to think about, okay, how are the single phases of this storyline
and put in some dialogues.
He has to estimate what can the user ask.
And with the episodic memory, he also has to take into account
what happens if this story starts again.
Because we have to look in the episodic memory, what kind of story already happened.
So that I can say, okay, yesterday he didn't cook,
but now I can ask him again or say something like,
oh, you're a bad entertainer because you don't cook.
So we can put a little bit of ultimateism with the storyline
and with the episodic memory in this stuff.
But sure, we need someone who thinks about what can the robot say,
how can the robot act.
In this case, when we take this example, we would store the title of the story.
We would say which are the names of the people who are coming.
For example, Steph Matias and Antonio are coming.
The robot would also store that I won't cook for these guys,
that we only drink beer, for example.
And if these guys which are coming talk to him or were unfriendly for him.
So that he have for the next time when they come enough information
to relate to this information.
So he's very connected to the story.
Yes.
For whatever you do, you have a story.
Not forever, because if you ask him what time it is, you don't need a story.
But for longer things, you need a storyline.
In terms of bringing in this common sense knowledge in your system,
you need to get this perceptual input into some kind of representation,
like cases, frames, or scripts.
We have concept nets.
We get the conceptual knowledge we're getting from concept nets.
And we can relate to this.
So if we say, my friends are Steph,
we create instances of, it's a human, it's a friend of Rudy,
and everything else what the robot will collect on information
and store this in the global context framework in concept nets.
Good.
Okay. Thank you very much.
Thank you.