Automated Sound and Vision

Art direction is a big thing in videogames. Nothing sells a game for me like great sound and vision; I just finished playing Portal 2 and one of the most affective aspects of the game was its immaculate presentation, the crispness of the architecture and the cleverness of its layout. There’s a big difference between placeholder ‘programmer’ art, and real artwork chosen and selected by someone who knows what they’re doing.

To make an obvious point more obvious, here’s a screenshot from a prototype of Jonathan Blow’s Braid:

And here’s a comparable game section after an artist got involved:

It’s my intention to add audio/visual design components to ANGELINA soon, but I’m torn over whether they should be evolutionary components, or something more straightforward. Why? It really comes down to external fitness – how we evaluate individual game components and their contribution to a game design as a whole. When we’re considering a ruleset, there’s plenty we can talk about by getting ANGELINA to test the game for herself. Things like:

  • Did I die every time I played this level?
  • Was it too easy or too hard to gain score?
  • Were some areas of the map impossible to reach?

In other words, we have some ideas for heuristics that might guide ANGELINA towards fun, playable games. But when it comes down to visual design, evaluation is more complicated. Off the top of my head, here’s some questions we might ask about visual design (I’m using the phrase ‘visual design’ here kind of loosely, because ANGELINA will be choosing images from a stock set, not drawing them herself):

  • Do the game elements make sense? Is a dragon trying to save a knight from an evil princess, or does everything fit together logically?
  • Does the music fit the mood? Am I picking flowers to a war march, or battling ogres to the sound of Yakkety Sax?
  • Are the sound effects contextual? Does it sound right when I pick up a gold coin?

These are problems that don’t feel particularly suited to evolution right now. If ANGELINA were designing the music or visuals herself, then maybe, but all she’s really doing is ‘artistic direction’ – putting pre-made assets in the right place. For instance, the excellent tool sfxr generates sound effects according to templates like “Player Hurt” or “Powerup”, which solves the contextual sound effect problem. Similarly, there’ll be a tagged database of music, which mitigates the problem of choosing an inappropriate soundtrack. Finally, the choosing of sprites for various game objects must ultimately appeal to the player’s understanding of what the sprites are trying to depict. So eventually, we’ll be appealing to a knowledge base of facts. In all three cases, the problems boil down to simple constraint solving rather than full-blown evolutionary search.

Which puts me in a tricky position! Because whilst we want sound and vision to improve ANGELINA’s games, it’s not getting us anything in return in terms of research gain. Despite this, I’m hoping I can find time to add in some of these features before CIG 2011 in September.

Leave a Reply