Working towards a future where humans and robots work together

Maarten Steinbuch and Elena Torta share a great passion for robots. For Steinbuch, this started at a young age. "In 1971 - I was eleven at the time - I went to visit the Evoluon with my parents, the iconic technology museum here in Eindhoven. When you went in there, you saw a five-meter long robotic arm with three microphones. And when you said something, that arm came to you. I really loved that!”

It was the beginning of a lifelong fascination with smart machines, and more generally with "things that move", as Steinbuch puts it. When he had to choose what to study at university a few years later, it was soon clear to him that it had to be mechanical engineering. "Mechanical engineering combines exactly the things I like: thinking with your head and making with your hands!”

Most people think of robots as humanoid machines that can walk around and do all sorts of clever things. But robotics is a very broad field, ranging from packing robots at Amazon and self-driving cars to operating robots and construction robots.

In love with feedback

"They all have in common that they are moving systems that perform autonomous functions using smart intelligence," Steinbuch explains. "In robotics, all sorts of disciplines come together, from electrical and mechanical engineering to computer science and physics, but perhaps the most important is control engineering. This ensures that the machine adjusts itself based on information from sensors and smart intelligence, so that it does exactly what you want.”

In robotics, all sorts of disciplines come together, but perhaps the most important is control engineering.

"I've always found that feedback loop to be a beautiful principle, I really fell in love with it. Also because you see it everywhere, not only in robotics, but also in humans, our sugar balance is one big control system. How we listen, and look, and move, those are all control systems. But also in management theory, thinking about feedback systems is very important.”

Steinbuch was able to indulge his fascination with control engineering in his PhD, where he figured out what the best pitch was for wind turbines, and at Philips, where he worked on the rotating arm of a CD player after graduating.

Robbie

While for Steinbuch it was movement that triggered his interest in robotics, for Elena Torta - born in 1985 in Fano, Italy - it was the interaction between the robot and the human, an aspect of robotics that still fascinates her immensely.

"When I was 10 years old, I first read something by Isaac Asimov, the famous science fiction writer. I didn't know anything about that author at the time, but I started reading, about robots that can do human tasks. The story was about Robbie, a friendly robot nanny who is part of an American family. You see how this robot grows up with Gloria, the daughter of the family, you can follow exactly the development he goes through. After that I devoured all of Asimov's robot stories," says Torta.

It won’t come as a surprise, that Torta went on to study control engineering a few years later. There she soon discovered how complex robotics really is, and that we are still a long way from Asimov's visions of the future.

"It is very difficult to get a machine to do things autonomously, especially in a dynamic environment. I very much recognize what Maarten is saying: to make a good robot, all kinds of disciplines have to work together. You are dealing with sensors, with data that you have to get from those sensors, with feedback loops, to correct any errors, you have to make sure that the robot moves in the right direction, and so on. For me, as a 21-year-old student, that was really a revelation."

Oops, a human!

Not to mention an environment where people also walk around.

Steinbuch: "Take Elena's robot. The real problems only start when someone walks in front of it. Then you have to know: which sensor is suitable for detecting this person, how fast is this sensor, what should I do next? You can't possibly map out all of those situations.

Look, in the car industry we have seen production halls full of robots since the 1980s, but these are all robots that work in an environment that is deterministic. They can be programmed flexibly, but in the end they do the same tasks over and over again. The challenge over the past 20 years has increasingly been robotics that has to work in unstructured environments, where things change that you don't know about.”

Interaction with humans was also the subject of Torta's PhD, which she chose to do in Netherlands. From 2010, she spent four years in Wijnand IJsselsteijn's group doing research on care robots for older people, so that they can live independently longer.

Goal!

The Eindhoven robot researchers have learned a lot from TechUnited, a TU/e student team founded by René van de Molengraft (in Steinbuch's group) in 2009, a year before Torta came to the Netherlands. The TechUnited soccer team has been successfully participating in the international Robocup competitions for years.

Steinbuch: "That soccer team is really a living lab. What we learn there, we can apply in our service robots, but also in our autonomous cars. Because the same principles apply there. Our goal is not to make the perfect soccer robot, but to use the knowledge needed to make that good soccer robot in other relevant applications."

Torta is closely involved with @home, TechUnited's service robot team, which also scores highly in international competitions. While the challenges for a soccer robot are similar to those of a care robot (both must be able to move autonomously in a complex environment), there are also differences.

Torta: "With care robots, the playing field is less structured, the environment can constantly change. You have objects with different colors, people who have different clothes on all the time, or different shoes. So there the understanding of the world has to be greater."

A formula for succes

That TechUnited's robots are so successful, has to do with the unique approach to robot research at TU/e, Steinbuch says. It combines the smart use of world models and artificial intelligence with years of experience in hardware and systems engineering and collaboration with industry. 

Steinbuch: "A lot of robotics groups originate from computer science. It’s something you see at Robocup too, there most teams are computer science teams. We tend to win because our engineering is better, we make better hardware. We are mechanical engineers who also like software, instead of the other way around.”

When you talk about robotics, you are also talking about artificial intelligence. Artificial Intelligence, like robotics, has been burdened for decades by a combination of overwrought expectations and dystopian predictions of the future.   

The promise of AI

Both researchers see the promise of AI, especially as robots increasingly have to function in the "real" world. Think of self-driving cars in the busy city traffic, or care robots that must be able to interact with people. But Steinbuch cautions against overblown expectations.

“If you only use data to learn your system, it will take a lot of iterations. That's fine in a simulation, of course, but you don't want to let a soccer robot try a shot 10,000 times, then it will be broken before it has learned anything. That doesn't work. So where we have models, we use them. And where the models have their limitations, such as in object recognition in a dynamic environment, we use data."

It's not either models or AI. We just choose the best way to solve a given problem every time. Without bias.

In this regard, Steinbuch agrees with Yann LeCun, Facebook's AI chief, who spoke about this at TU/e three years ago. "He sees the combination of AI with modeling as the big development in the next decade."

Torta, who increasingly uses machine learning techniques such as reinforcement learning in her research, agrees: "It's not an either/or situation. We just choose the best way to solve a given problem every time. Without bias." 

Robot realism

So there are plenty of successes to celebrate, but how do the scientists see the future?

Steinbuch: "We have come a long way, thanks in part to the increased processing power of computers, the use of lots of data, and increasingly better sensors. But we're not there yet. To give you an example, we started with our soccer robots 12 years ago, and every year we make huge progress. But it's still the case that our robots don't win over a human team."

Torta and Steinbuch have high hopes for a multidisciplinary project they have set up together with colleagues from Delft University of Technology and Twente University, the VU University Amsterdam, and Erasmus University Rotterdam.

Torta: "We want to experiment with a soccer team that is half robot and half human. You can imagine, that is not a trivial matter. The soccer robot has to sense what its human teammates want. It wouldn't be nice if a soccer robot was always on the ball and the only one to score goals!"

Steinbuch: "The great thing about this project (which is still waiting for funding from the Dutch Research Council, ed.) is that we will be working with psychologists and sociologists. They think very differently from us. I am convinced that a combination of our technical expertise and knowledge from the alpha and gamma world is the way to go forward."

The indispensable robot

Ultimately, both believe that robots will become indispensable in the society of the future. If only because, due to the aging population, we have too few people to take care of everyone.

Steinbuch: "If, as predicted, the aging population in the Netherlands will continue to increase, care will eventually become unaffordable. We need twice as many people in home care, from 400,000 in home care to 800,000, and we don't have those people. So the only thing we can do is make sure those people can stay at home much longer. And the only solution to that is technology."

Not to mention the crucial role that robots can play in the operating room (through precision surgery) and in logistics, where there is still a world to win in terms of quality and efficiency.

Ethics

But it won't happen automatically. If robotics is to serve people and not the other way around, both believe it is of great importance that the people who make these smart machines think about ethics at an early stage.

Torta: "As researchers we want to contribute to a better future. The technology we make is intended to help people in 10 years, not to replace them. You have to take that into account now. Equally important is that you involve users in the devices you make for them. Include them in the design." 

Steinbuch: "Don't forget education. It's crucial that ethics is part and parcel of our educational program, so that our students learn to deal with these problems at an early stage. They may not always like it, but it's extremely important. Because it’s their generation that is going to change the world in the next thirty years!"

The language of industry

Eventually both believe that the success of the Eindhoven robots stands or falls with the degree to which they solve real problems. In this respect, they stress the importance of collaboration with industry. Both have worked in industry for years (Steinbuch at Philips, Torta at ASML), and now also work intensively with companies in the Brainport Region and beyond.

"Collaboration with industry brings real-life problems to science," says Torta. "Companies have a good grasp of what the real problems are. Moreover, they force researchers to make sure their results are reproducible. It has to work.”

I think that as a university we have the task of being the long-term conscience of industry.

Steinbuch agrees, but also sees a reverse interest. "I think that as a university we have the task of being the long-term conscience of industry. Now that companies no longer have extensive facilities to do fundamental research, as they used to at Philips’ NatLab, they pay us through PhD students to come up with out-of-the box solutions. And of course, Elena and I also speak the language of industry, that certainly helps. You won’t find that everywhere.”

Steinbuch is not afraid of conflicts of interest. "We often have more freedom in our research with PhDs paid by ASML than with the PhDs we get funded through the Dutch Research Council. We do for ASML what we think is relevant to them in the long run.”

Frankenstein?

In the Asimov story that Torta read when she was 10 years old, Robbie ends up saving Gloria's life. It was Asimov’s way of dealing with the age-old fear that robots will turn against their creator.

Torta and Steinbuch also don’t believe in this Frankenstein scenario. "As engineers we are ultimately techno-optimists. We believe that technology can offer solutions to a lot of social problems. And that certainly also applies to the robot."