Google is training an AI to play video games — why this could be problematic

Computer-controlled video game characters aren't anything new. Non-playable characters or NPCs are longtime staples of gaming. 

However, Google's AI research lab, DeepMind, has announced work on something distinct and new in computer-controlled gameplay: SIMA, which stands for Scalable Instructable Multiworld Agent.

Tweet may have been deleted

Unlike NPCs, Google DeepMind's AI-controlled gaming companion isn't programmed into a specific game to act a certain way or respond to certain human-provided actions. SIMA is, as DeepMind describes it, a "generalist AI agent for 3D virtual environments." 

SIMA is not an NPC

SIMA plays along with gamers, acting as a human player, by following verbal commands. As Google explains, "SIMA isn't trained to win a game." It's primed to work alongside the human player and perform actions based on natural language prompts.

This element is crucial to what makes SIMA unique. Again, SIMA isn't an NPC, designed specifically to a certain title, which is meant to provide a challenge to the player. SIMA acts as a fellow gamer trying to accomplish whatever the human player is telling it to do.

"SIMA needs only the images provided by the 3D environment and natural-language instructions given by the user," Google says. "With mouse and keyboard outputs, it is evaluated across 600 skills, spanning areas like navigation and object interaction - such as 'turn left' or 'chop down tree.'"

According to Google DeepMind, SIMA was trained and tested on nine different video game titles of varying gaming genres, including No Man’s Sky, Goat Simulator 3, Teardown, and Wobbly Life. The training does not require use of any sort of access to the games' source code or APIs. Google DeepMind says it focused on open-play games over narrative ones to facilitate SIMA's training in more general gaming skills.

There are some positives, and some very obvious negatives, with SIMA. An AI gaming companion could be nice when an individual wants to play a multiplayer game or an offline two-player story-mode without another human around to pick a controller.

But, the ideas driving SIMA's research could obviously be used in bad faith, too. Gamers could use SIMA for an unfair advantage in online gameplay. Games where players have to perform monotonous tasks for resources or complete quests to gain XP could obviously be manipulated by a human just telling an AI companion to do all of that work for them.

As for right now though, SIMA is not ready for primetime. It's still in the midst of learning to play video games.