ASU’s AI-powered robot is great at pingpong, but its ultimate goal is to help around the house

What is the relationship between playing pingpong and doing household tasks? For Heni Ben Amor, an Associate Professor at ASU, the answer is found in a robot.

Ben Amor works on robotics and machine learning. He and a team with Google DeepMind have been building and training a robot that plays table tennis. But, the goal is not to rule the basement or rec room — the goal is to help people with chores around the house, especially people who can’t do some of those tasks on their own.

Ben Amor joined The Show to discuss how this project is working.

Full conversation

HENI BEN ARMOR: Absolutely. So this is a project together with Google DeepMind. And one of the big milestones in AI has been something that was called the AlphaGo moment. A couple of years ago, Google was able to beat the world champion in the Chinese game of Go, and that was considered really sort of a major milestone in AI.

However, all of that was sort of virtual. No one was actually moving the chess pieces, you could say. And so what we wanted to do is to have a similar moment for robotics — meaning that you have robots that can learn and engage and have maybe human-level capabilities, but all of that is happening in the real world. They’re actually moving the chess pieces.

But in our case, it’s not chess pieces. It’s table tennis.

MARK BRODIE: Well, playing table tennis is a lot more sort of quick movements than chess. So what kinds of challenges does that pose for you and your team who are trying to do this? 

BEN ARMOR: There’s a variety of challenges, but actually, in our case, we were really looking forward to these challenges in order to tackle them and then see sort of how we can fill this knowledge gap.

So one of the challenges is that in table tennis, if you’re just watching the ball, by the time you actually get to the ball, it’s too late. What human players do, at least good players, is to anticipate where the ball is going and already be ready to kind of hit the ball back in the right way.

Now, another challenge here is that the ball is — since it’s very fast — it’s actually not easy, dexterity-wise, to hit it with the paddle in the right way in order to get it back on the table. And even more challenging is to hit it, for example, with a spin, or make sure that the ball lands at a particular location on the table.

So all of that requires really complex physical skills and dynamic reactions. But these reactions are more than just reactions. They’re really in anticipation of the future and then acting on what’s going to happen rather than what’s happening right now.

And it also involves incorporating both perception challenges. So we need to see the ball and anticipate where the ball is going. But, we also need to reflect on what the best response is, what the robot should be doing. So quite a number of challenges. And if you make even a tiny mistake, well, the ball is going to go somewhere else.

BRODIE: So obviously the main point of this project you’re doing is not to create a world champion table tennis player. The goal is to ultimately be able to use robots like this to help folks to live independently, to help with tasks around the house.

I’m curious, what is the relationship between teaching a robot how to play ping pong and helping them with some of those other tasks?

BEN ARMOR: It’s this aspect of human-robot interaction. Ultimately, what we would like to have is robots that engage with humans in a meaningful and safe way. And so for that, we need to have a way to measure whether robots are actually engaging and whether they are reacting to a human player at all.

And table tennis is kind of a really interesting benchmark, because you can very quickly figure out whether the robot is responding the right way to a human or anticipating the human’s actions the right way.

Now, the same technology can be used to help people at home, meaning we would like to have robots that anticipate your needs, for example, at home. And according to that, then plan their actions so that they can help you at the right time.

Now, one of the things that, probably you are aware of is that we don’t have robots at home. And one of the challenges for that is really that they have a hard time coping with, sort of, all of the changes and the dynamic aspect of our home.

So robots need to take their surroundings into account, both the inanimate surroundings — you know, Lego pieces on the floor, a book or a moved chair — as well as humans and the actual family inside the home.

BRODIE: So is the goal then to, for example, if the robot was going to be in the home, to not have to have somebody program or ask the robot, “OK, hey, can you change this light bulb? Can you set the table? Can you sweep up, you know, this mess?” But to have the robot somehow know that those things needed to be done without being told?

BEN ARMOR: It’s a combination of both, actually. On one hand, we would like the robot to understand your instructions, to execute those instructions. But potentially, even if they fail at that, they could self improve and adapt and learn. So machine learning is really a critical part of what we’ve been working on. And so that’s one part of it.

And then the other part is actually having robots that can create something like a capability model of the human partner. So the robots start to understand what the sort of weaknesses and capabilities of my human partner are. And they can then incorporate that into their actions. And so that leads automatically into some sort of anticipation — anticipating what the human partner needs, what the weaknesses are, and then based on that, sort of engaging in a meaningful and safe interaction.

BRODIE: I have to say that listening to you describe that, I’m reminded of the TV show “The Jetsons,” who of course had Rosey the Robot sort of doing all the housework. It kind of sounds like that’s the direction you’re heading in here? 

BEN ARMOR: This is great that you bring this up. Right now I have a follow-up project that I’m working on, and the title of this project is actually Rosey the Robot, which is the name of the robot in “The Jetsons” TV show. And kind of in our case, we say RoSI for robot self-improvement. So indeed, what we’d like to have is robots that can engage in interactions with humans. But at the same time, they need to be able to perform physical tasks.

Right now, many of us are using ChatGPT, right? And Gemini and all of these large language models. And that has led to sort of a hype around AI. But what we’d like to have is something like that, like these Gemini models and LLMs and so on, but they can actually execute something in the real world.

You ask them to prepare breakfast, and the robot goes out and prepares breakfast. Maybe the robot already knows what your preferences are. Maybe they know, for example, if someone is diabetic, not to include sugar and so on. So this actually has sort of a meaningful purpose behind that.

And in my specific case, at ASU, I’m using similar, if not the same, algorithms to create the prostheses, a lower leg prosthesis for people with amputations. What we would like to have is to have these robotic technologies that assist them but can also learn a profile of the user. And they know that you have a certain walking gait, that a certain walking gait leads, for example, to pressure on your body. And they try to avoid that.

And so this technology really has sort of a much deeper underpinning. And what we are trying to do is to fill this knowledge gap by starting with table tennis — because there it’s very easy to assign a number and say whether we’re making progress or not — and then that knowledge, that body of knowledge that we then create, we use it for a variety of different tasks that have a much deeper sort of societal value.