Whether it’s plotting a driving shortcut or planning a pickleball serve, humans can use visual simulation to solve everyday problems. It turns out, monkeys can, too, according to new research published in Current Biology by a group of scientists affiliated with the Robert J. and Nancy D. Carney Institute for Brain Science at Brown University.

Using experimental and computational methods as well as fMRI imaging, the research team, led by Professor of Neuroscience David Sheinberg, showed that monkeys also use a “mind’s eye” approach to problem solving. In this case, monkeys learned to play a game called Planko, successfully predicting where a ball would travel down a game board through a series of obstacles. Behavioral, computational and imaging evidence suggests that the monkeys used visual simulation to accomplish this task.

Sheinberg said the work is a win for brain scientists, giving them a new framework for studying complex thinking. There may also be implications for human health. The more scientists understand imagination, Sheinberg said, the more they might learn about when it goes awry. For example, the circuitry involved in visual simulation could shed light on the origin of hallucinations that can come with schizophrenia or bipolar disorder.

“This work gives us a foothold for exploring neural circuits at a level of explanation we never had before,” said Sheinberg, vice chair of the Department of Neuroscience. “We know that visual systems in the brain are involved in this kind of complex cognition, and that humans aren’t the only species with the capacity for this behavior.”

Visual simulation is a way to imagine. To predict the future and plan behavior, humans can “see” a scenario in their mind’s eye – to visualize events that aren’t happening. It’s a sophisticated trick of the mind used to solve problems, make decisions or achieve goals.

But is this ability uniquely human? And how do brains – any brains – perform this task?

Sheinberg’s team decided to find out, working across disciplines to create a reliable animal model of this complex behavior. 

To design the behavioral experiments, Sheinberg worked with Aarit Ahuja, a former neuroscience graduate student and postdoctoral researcher at Brown. They set up a series of scenarios in which two macaque monkeys played a classic strategy game called Planko. 

The object of Planko is to look at an obstacle-filled display and predict where a ball will land – right side or left. The researchers trained the monkeys, and when the animals guessed accurately, they were rewarded with a sip of juice. Sheinberg and Ahuja found that the monkeys’ behavior and eye movements indicated they were simulating the ball’s path to predict how it would fall and where it would land – information consistent with the lab’s previous studies with humans.

To develop a computational explanation for playing Planko – with and without simulation – they turned to Thomas Serre, the Thomas J. Watson Sr. Professor of Science in the Department of Cognitive and Psychological Sciences and the Department of Computer Science and associate director of Carney’s Center for Computational Brain Science. 

Serre developed the computational experiments with Alekh Karkada Ashok, a Department of Cognitive and Psychological Sciences graduate student in his lab. They trained two neural networks to predict where Planko balls would land, then dug into the data to see how the different networks made predictions. They found that a recurrent neural network, or RNN, developed in the Serre lab most closely matched the behavior of monkeys while they played the game – play that was most consistent with visual simulation.

So, if game-play and computational modeling both pointed to visual simulation as the strategy monkeys were using while playing Planko, what was happening in their brains? To find out, Sheinberg teamed up with Theresa Desrochers, the Rosenberg Family Assistant Professor of Brain Science and an assistant professor in the Department of Neuroscience. They adapted the task as a neuroimaging study so a monkey could play Planko in a functional magnetic resonance imaging machine, or fMRI, which detects changes in blood flow and oxygen levels to measure brain activity. 

These neuroimaging experiments, performed by Ahuja and former Brown neuroscience graduate student Nadira Yusif Rodriguez, showed that when the monkey studied static Planko boards to predict how the ball would fall, motion-sensitive brain regions responded as if the ball’s motion was literally being seen. This activity mirrored the activity in their brains when the balls were actually set in motion. Again, these findings were consistent with Sheinberg’s work with humans.

“We showed that monkeys have the capacity for mental simulation – that they can predict outcomes of events they’ve never seen before – which is a huge leap in our understanding of animal intelligence,” Sheinberg said. “This opens up exciting avenues for brain science research, and hints at a rich, imaginative mental landscape in the animal kingdom.”

The work was supported by two Carney Zimmerman Innovation Awards in Brain Science as well as grants from the National Eye Institute (R01EY014681, R21EY032713, T32EY018080); The National Institute of Mental Health (T32MH115895), the National Science Foundation (1632738) and the Office of Naval Research (N14-24-1-2026).