Researchers Use ‘League of Legends’ to Gain Insights Into Mental Models

Psychology researchers have used the game League of Legends to advance our understanding of how people build “mental models” – the mental tools that allow people to make use of complex systems. The researchers found that mental models are developed in three distinct stages – findings that have the potential to inform everything from workplace training to video-game development.

“We create mental models to help us interact with systems, whether that’s driving a car or playing a video game,” says Caleb Furlough, lead author of a paper on the study who did the work while a Ph.D. student at North Carolina State University. “These models allow us to predict how the system will react when we perform different actions. For example, if I turn the steering wheel to the right, the car will turn right.

“Our goal with this study was to get a deeper understanding of how people structure these mental models in their memories, and how those structures change as we gain expertise,” says Furlough, who is now a human factors consultant. “In other words, how does the structure of a mental model differ between a novice, a journeyman and an expert?”

“We chose a video game for this study because it presented a new arena for mental model research, and because it requires people to use different types of memory – not just retaining facts, but also memory related to skills and actions,” says Doug Gillan, a professor of psychology at NC State and co-author of the paper.

For this study, researchers first recruited several people who played League of Legends. This small group served as a focus group, which the researchers used to develop an in-depth survey that could be used to capture a player’s understanding of various concepts in the game, such as strategies, characters and locations.

Researchers then recruited 158 study participants with varying levels of experience in playing League of Legends. Novices were classified as those who had played the game for 100 hours or less; journeymen were classified as those who had play the game for between 101 and 1,000 hours; experts were classified as those who had played the game for more than 1,000 hours. All of the study participants completed the survey, which focused on the extent to which key game concepts were related.

The researchers then analyzed survey responses using the Pathfinder analysis tool to create visual networks that highlight the connections between various concepts. The researchers then compared the networks of novices, journeymen and experts in order to identify differences in how each mental model was organized.

The researchers found that novices organized their mental models based largely on “natural language connections,” or the way that words are used in everyday conversation – as opposed to the way language may be used in the context of the game.

In contrast, journeymen mental models were based largely on procedural connections. This means their models connected concepts based on how they could be used to perform actions or accomplish goals within the game.

Expert mental models displayed even more pronounced procedural connections, but also demonstrated two other characteristics: abstract connections and subnetworks.

Abstract connections rely on theoretical links, rather than obvious, external characteristics. For example, if an ambush relies on splitting one’s forces, the link between the concepts of “ambush” and “divided forces” is an abstract one.

“And subnetworks are little groups of highly interconnected concepts,” Furlough says. “We only see a lot of subnetworks once someone has achieved a level of expertise, granting them significant insights into the interconnection of myriad concepts.”

These insights into how people build mental models have potential applications for developing training programs in general, and possibly for game design in particular.

“For training, this work offers insights into which characteristics a training program may want to focus on at each stage of the process,” Furlough says. “For example, you may want to start with a focus on language, then move to procedural connections before introducing abstract connections. And, if subnetworks are relevant to a given skillset, a training program may want to teach the relevant concepts together as part of a single package.

“As for gaming, these three stages of mental model development could offer a distinct approach to game development – with games introducing challenges that force players to change their mental models as they gain in expertise.”

The paper, “Mental Models: Structural Differences and the Role of Experience,” is published in the Journal of Cognitive Engineering and Decision Making.