In 1997, Hiroaki Kitano, a research scientist at Sony, helped organize the first Robocup, a robot soccer tournament that attracted teams of robotics and artificial intelligence researchers to compete in the picturesque city of Nagoya, Japan.
At the start of the first day, two teams of robots took to the pitch. As the machines twitched and surveyed their surroundings, a reporter asked Kitano when the match would begin. “I told him it started five minutes ago!” he says with a laugh.
Such was the state of AI and robotics at the time. It took a machine minutes to interpret its situation and work out what to do next. But much has changed, with AI increasingly helping machines, from self-driving cars to surveillance cameras, perceive and behave in clever ways.
Kitano now leads a new effort at Sony, announced in November, to infuse cutting-edge AI across the company. The Japanese giant believes AI will create smarter cameras, more cunning videogame characters, and even the first helpful kitchen robots. Kitano says Sony believes AI is making such rapid progress that the company needed to make the technology central to its strategy.
“We have decent AI researchers and engineers at Sony, and we have a good sense of what’s going on,” says Kitano, who was attending the Association for the Advancement of Artificial Intelligence conference in New York this week. “We decided now is a moment that we should really push.”
Sony’s move stands out among big companies’ efforts to embrace AI. It lags behind Silicon Valley giants in researching and harnessing AI. It also has different aims: Sony is more focused on content creation and entertainment than the likes of Google, Facebook, or Apple. The Japanese giant is now looking to match America’s AI titans by betting heavily on a powerful but still relatively experimental approach to AI known as reinforcement learning. Google parent Alphabet and Amazon have made notable investments in this technology too.
Alphabet’s DeepMind famously used reinforcement learning to create a program capable of beating one of the world’s best Go players in 2016. Inspired by animal behavior, it involves an algorithm refining its behavior in response to positive or negative feedback.
“We consider reinforcement learning is equally or possibly even more important,” than the technologies that have driven most progress in AI to date, Kitano says. “It’s going to be the key.”
Besides research demos, reinforcement learning is being tested in areas ranging from finance to logistics. It is also emerging as a powerful way for robots to learn to deal with the real world and for training software agents to behave intelligently in simulated environments. So it may have huge potential to generate compelling videogame characters and scenarios.
Reinforcement learning has been part of AI for decades, but its promise has become apparent thanks to powerful neural network algorithms, roughly modeled on the way learning happens in the brain, as well as far more powerful computers and large amounts of training data. Even so, it is experimental and notoriously difficult to get right. Research has shown, for example, how reinforcement-learning algorithms can sometimes fixate on a reward that results in repetitive and useless behavior.
Sony will focus its AI on three domains, Kitano says: gaming, sensors, and, more curiously, culinary arts. These areas reflect the company’s current business focus and an aspirational direction for the future.
Sony is well known for making the PlayStation and games, but it also gets a large share of its revenue from digital sensors and imaging technology. It isn’t hard to see how AI could improve these areas, by making games more compelling or lively or helping cameras perceive the world more intelligently.
The effort to put AI to culinary use is about advancing robotics. So far, Sony has demonstrated a robot capable of placing food items on a plate in an artistic but preplanned way. Future systems might be able to recognize and grasp things without careful coding. Handling food is especially challenging for a robot because items are often irregularly shaped and arranged, and need to be handled with care.
Sony is, of course, no stranger to robots. A few years after the first Robocup, the company released Aibo, a doglike toy that gained a cult following but was canceled in 2006 amid a corporate streamlining. A new version of Aibo, released in 2018, includes some AI capabilities such as object and voice recognition. But it’s still relatively dumb. When I met with Kitano at the AAAI conference, one of the robots, brought by a Sony rep, explored the room and yapped away behind him.
Some outsiders see big potential in Sony investing in reinforcement learning. “It makes a ton of sense,” says Pieter Abbeel, a professor at UC Berkeley and cofounder of Covariant.ai, a company using reinforcement learning to make more adaptive warehouse robots.
Abbeel points out that it’s expensive and time consuming to create videogames, and he notes that reinforcement learning has shown potential to take on much of the drudge work. He points to a project called DeepMimic, which shows how virtual characters trained with reinforcement learning can exhibit lifelike behavior. A character placed in a physically accurate environment and given a specific goal, like climbing over an obstacle, will eventually work out how to vault it. This could automate the process of programming videogame characters or even allow behavior to emerge on the fly in a game. “It has the potential to facilitate much faster content creation,” Abbeel says.
Abbeel says robots preparing meals are probably some way off, but he expects reinforcement learning to change the way these machines are programmed. “It will be really exciting to see a push there,” he says.
To fuel its AI endeavor, Sony last year acquired Cogitai, a company cofounded by Peter Stone, a professor at the University of Texas at Austin. Stone has successfully pioneered the use of reinforcement learning in a virtual version of the Robocup contest with trophy-winning success. Stone now leads the Sony AI operation in the US.
Before being acquired, Cogitai launched a platform designed to make reinforcement learning easier to use. Stone says this and other tools will now be made available to researchers and engineers throughout Sony. A game developer or a hardware designer should be able to use these tools to explore new ideas and innovations. He says the focus on reinforcement learning reflects Sony’s desire to get ahead of the curve in AI—by betting on what looks likely to be the next big thing.
Today’s Robocup matches show how rapidly things are moving. Players can be seen passing, moving, and shooting with remarkable speed and skill.
According to Stone, this also points to the next stage of progress in AI. “There’s been a revolution in perception and supervised learning,” he says. “The whole premise of Sony AI is that there are huge opportunities for automated decision-making in AI too. It’s really everywhere, and in some sense it’s untapped.”
More Great WIRED Stories
- The ragtag squad that saved 38,000 Flash games from internet oblivion
- The tiny brain cells that connect our mental and physical health
- How to get the most out of your smartphone’s encryption
- Vancouver wants to avoid other cities’ mistakes with Uber and Lyft
- The eerie repopulation of the Fukushima exclusion zone
- ? The secret history of facial recognition. Plus, the latest news on AI
- ✨ Optimize your home life with our Gear team’s best picks, from robot vacuums to affordable mattresses to smart speakers