This adventure game tech demo with AI-generated art is impressive and disconcerting
The demo is a convincing look at how an actual artist can use generative programs as a tool in their process, and that's what troubles me.
Jussi-Petteri Kemppainen, a veteran game dev based out of Finland, has recently started sharing early demos of a 2.5D point-and-click adventure game with art and assets sourced from Midjourney and Stable Diffusion, two machine learning text-to-image tools. The project convincingly shows how an individual artist with limited resources can use these controversial, questionably ethical tools to punch above their weight when developing a game.
In his first blog post discussing the prototype, Kemppainen describes wanting to see how far he could push these tools in the creation of a videogame prototype. To that end, he chose to make a 2.5D point and click adventure, allowing easy generation of 2D background art with a more challenging experiment in turning AI-generated assets into textures for 3D models.
Kemppainen used AI tools to generate a number of character concept art sheets, the likes of which you may have seen in a collector's edition artbook. He ultimately settled on a weird old bearded man he got out of Stable Diffusion—the character basically looks like a stock "old coot but make it cyberpunk," someone I can imagine doing a sidequest for in Borderlands. Kemppainen then roughly aligned this art with a 3D model he made and touched it up by hand.
To turn AI-generated location art into a playable space, Kemppainen created 3D collision data in modelling software Blender, overlaid that on the 2D background, and lit the scene in Unity. The result is impressive, but in a discomfiting way. There's still some AI image gen weirdness—our dirty Borderlands Santa's favorite convenience store seems to be named "PAICKTPPARRCTK: MIATIIKK PICAIKANKT," and Stable Diffusion birthed him with a rather incongruous set of pigtails. As a rough proof-of-concept of the labor-saving potential of AI tools, however, it's convincing.
It helps the project's credibility that Kemppainen isn't a fabulist producing images of dead-eyed anime ladies with eleven fingers on each hand and claiming that these tools will replace human artists, he's a veteran videogame developer with experience on the art side of things and credits on multiple games, including Remedy's Quantum Break. All together, he thinks his use of generative tools saved him at least five days of work vs creating similar background and character art by hand.
"[AI generation] enables the creation of games that would otherwise be left unmade because of budget or time constraints for sure," Kempainnen says.
The unresolved problem here is that PAICKTPPARRCTK and Dirty Borderlands Santa are a slurry of real people's artwork—individual artists who may or may not have found success with the craft, but whose work has been fed to these AI algorithms to train them, probably without their knowledge or permission. Kemppainen himself is not precious with the AI generations that resulted from his prompts, stating that "they can be freely used by anyone, as replication of these images is pretty trivial." He also acknowledges that these assets derive from other artists' efforts, explaining that, for what it's worth, he avoids referencing specific artists, instead opting for as broad or generic results as possible. Whatever my own concerns, though, Kemppainen's success is further evidence that AI art isn't going away.