This Robot Can Guess How You're Feeling by the Way You Walk

Maybe you’re sad, plodding along with your head slumped and shoulders sagging. Maybe you’re angry and alert, hurrying along upright. Or maybe you’re annoyingly happy, skipping down the street. Whatever your gait, that combined with your facial expression is nonverbally signaling to other people how much space they should give you—and being able to accurately read these cues is an essential skill for our social species.

And perhaps for robots, too. Researchers at the University of Maryland have developed an algorithm called ProxEmo, which gives a little wheeled robot the power to analyze your gait in real time, and to take a guess at how you might be feeling. Based on that perceived emotion, the robot can choose its route to give you more or less space. This might seem like a small matter for robot-human interactions, but we might also imagine that one day, when machines are sophisticated enough, they might read a sad person’s gait and try to help them.

“If somebody’s feeling sad or confused, the robot can go up to the person and say, Oh you’re looking sad today, do you need help?” says Aniket Bera, a robotics and AI researcher at the University of Maryland, who helped develop ProxEmo. Or, the ability to interpret gait might help the robot navigate around a person who wouldn’t want to interact with it. “Like if an angry person were to walk towards it, it would give more space, as opposed to a sad person or even a happy person,” he says.

A caveat right off the bat: This emotionally intelligent robot is basing its actions on perceived emotions. It can’t truly read anyone’s inner state. Not even a human can look at another and say with 100 percent confidence that they know if that person is happy, sad, or angry. But as a social species, we’ve learned to read telltale clues in a person’s gait that indicate their emotional state. That’s useful for not pissing off an already pissed-off fellow human.

Courtesy of Aniket Bera

So to build this system, the robotics researchers began with humans. They had a group of people look at other people walking, and asked them a series of questions about what they thought the walker’s emotional state was. This exercise was largely subjective, of course. But the researchers took this data from people observing their fellow humans and then correlated it with data about each person’s gait.

In order to understand how a gait looks, though, the robot needs objective data, not subjective judgements. So the researchers used algorithms that analyzed videos of the people walking, with each person’s image overlaid by a skeleton with 16 joints, including at the neck, shoulders, and knees. Then, they used deep learning algorithms to get the system to associate certain skeletal gaits with the emotions that the human volunteers associated with those walking people.

Courtesy of Aniket Bera

They ended up with the new ProxEmo algorithm. They load this into a cute little yellow four-wheeled robot (the Jackal, from the robotics company Clearpath) with a camera mounted on it. As the robot rolls along, the camera watches passing pedestrians, while ProxEmo overlays each human with that 16-joint skeleton—the objective measurement of gaits, which the algorithm has learned to associate with certain emotions. That’s how ProxEmo can take a guess at your emotional state, and direct the robot to respect your personal bubble, which may grow larger if the system thinks you’re angry, and shrink if it thinks you’re happy.

But this bubble isn’t a perfectly rounded sphere—it’s oblong, a kind of ellipsoid. “So there’s more space ahead of you, there’s some space at the sides, and there’s less personal space behind you,” says Bera. When the robot is approaching a human head-on, it needs to give a lot of space up front, but less so as it passes on the side or when it resumes its original course once it’s behind the human.

The tricky bit, though, is that the robot trained on clean data extracted from people walking in a lab environment—that is, their limbs were all visible at all times. But that isn’t always the case in the real world. So to prepare the robot for a chaotic world, the researchers had to incorporate a little bit of chaos in the form of “noise.” In this context, it doesn’t mean sound, but training it to handle variables like clothing and if the person is carrying something that might cover their hand. “Adding noise won’t change the emotion, but we’ve added different kinds of noises, and then trained the system to understand even if one hand is missing, you’ve still been looking at that person for a while,” says Bera.

One day, ProxEmo might be paired with another system that reads facial cues, building an even more complex emotional intelligence, which makes it more likely to know if it should ask to be of assistance, or give the person a wide berth. Robots have to get much, much better at picking up such nuances, if we want them to walk and roll among us without hurting people or themselves.

“When robots behave in an unnatural way, not only can people become uncomfortable, but also it can lead potentially to collisions, because the person cannot guess how this robot is moving,” says MIT computer scientist Dina Katabi, who wasn’t involved in this work. “So if a robot, for example, can avoid making people uncomfortable because it’s getting too close, that would be beneficial.”

No sense, after all, in turning a fancy robot into a very expensive punching bag.

This Robot Can Guess How You’re Feeling by the Way You Walk

Author: sgadmin

Related Posts