Google Has Started Adding Imagination to Its DeepMind AI
The robots have plans now.
Another way to put it would be imagining the consequences of actions before taking them, something we take for granted but which is much harder for robots to do.
“When placing a glass on the edge of a table, for example, we will likely pause to consider how stable it is and whether it might fall,” explain the researchers in a blog post. “On the basis of that imagined consequence we might readjust the glass to prevent it from falling and breaking.”
“If our algorithms are to develop equally sophisticated behaviours, they too must have the capability to ‘imagine’ and reason about the future. Beyond that they must be able to construct a plan using this knowledge.”
We’ve already seen a version of this forward planning in the Go victories that DeepMind’s bots have scored over human opponents recently, as the AI works out the future outcomes that will result from its current actions.
The rules of the real world are much more varied and complex than the rules of Go though, which is why the team has been working on a system that operates on another level.
To do this, the researchers combined several existing AI approaches together, including reinforcement learning (learning through trial and error) and deep learning (learning through processing vast amounts of data in a similar way to the human brain).
One of the ways they tested the new algorithms was with a 1980s video game called Sokoban, in which players have to push crates around to solve puzzles. Some moves can make the level unsolvable, so advanced planning is needed, and the AI wasn’t given the rules of the game beforehand.
The researchers found their new ‘imaginative’ AI solved 85 percent of the levels it was given, compared with 60 percent for AI agents using older approaches.
“The imagination-augmented agents outperform the imagination-less baselines considerably,” say the researchers. “They learn with less experience and are able to deal with the imperfections in modelling the environment.”
The team noted a number of improvements in the new bots: they could handle gaps in their knowledge better, they were better at picking out useful information for their simulations, and they could learn different strategies to make plans with.
It’s not just advance planning – it’s advance planning with extra creativity, so potential future actions can be combined together or mixed up in different ways in order to identify the most promising routes forward.
“Further analysis and consideration is required to provide scalable solutions to rich model-based agents that can use their imaginations to reason about – and plan – for the future,” conclude the researchers.
Dr. Hans C. Mumm