We talked to the game developer Ear Games about their upcoming iPhone/iPad game Ear Monsters. Brian Schmidt explains how they overcome some technical and gameplay issues while making the game. You can purchase Ear Monsters in the iTunes for $1.99.
Ear Monsters sounds like an interesting game. Can you tell us more about the game and how it came to be?
Ear Monsters is a casual, ‘bite-sized’ arcade style audio game that uses 3D sound instead of visuals for its gameplay. Think Fruit-Ninja or Whack-a-mole, but where you can’t see the enemies that pop in, you can only hear them. It’s an arcade-style game in that it’s fast paced and timed and it’s a scored game (as opposed to a “play it til the end” game); the better your skill level, the better your score will be. To keep you challenged, the better you get at the game, the harder it gets in return in a number of different ways…
I’d always been intrigued by the notion of sound-based videogames (or audio games as they’re sometimes called), and I’ve worked quite a bit in 3D sound over the years. Ear Monsters is loosely based on an old technology demo I did in the early 90’s when I was designing a 3D audio chip for Capcom’s arcade game system. In that demo, you’d fly around a helicopter with a joystick and try to shoot audio enemies as you heard them. We used to sit around and play that, trying to best each other’s scores.
Fast forward 20 years and iOS is out there, so I decided to look at what the iPhone/iPad could do, and it turns out to be a great device for audio games. Although I’d been poking around with the iOS SDK for a few months, I got serious about Ear Monsters about 4-5 months ago; to really dot the I’s, cross the t’s, try out different rules and game scenarios, and make it a game, and not just a tech demo. And I created EarGames (www.EarGames.com) as a company to officially handle the details and logistics.
It says Ear Monsters is a fast paced action game driven by sound. How does that effect overall gameplay and sound design?
It was fairly challenging to design. Games like precise actions and reactions, but it turns out our hearing—localization in particular—isn’t really all the precise on its own. There are a couple of very good 3D sound based games out there, but they are generally slower and more deliberately played. I really wanted something that you could pick up, try to beat your high score (or your friend’s ) and be done within a few minutes. So I went for the fast-pace. A Monster quickly appears, and you need to kill it as fast as you can. You hear a bomb, and you need to defuse it in a few seconds or it’s game over. There are moments where you’re tapping the screen as fast as you can. So the gameplay is short, quick, and the sounds themselves need are as well. You can’t very well have a 8-second “monster death” sound when you need to move on to the next monster before time runs out.
I also wanted to try very hard not to have to have to explicitly tell the player what to do—to have the game’s rules be self-evident if you just jump in and play. I found that dumping the player into a purely audio-only experience was too tough; they didn’t understand what they were supposed to do. So I cheated utilized some creative game design techniques. When you first start the game, you can see the enemies as well as hear them, but each monster that appears is dimmer than the previous one until after a few, you can’t see them at all. That let me ‘train’ the player so they would know what to do. So being driven by sound, and having to bring the player from their visual-oriented world into the sound-oriented game definitely had an impact on gameplay.
What technical limitations did you have to overcome (considering this is an iOS game) to make the game fun and interesting?
The biggest challenges were not technical so much as psychoacoustic.. iOS is actually a great platform for audio games. You’ve got a small, compact, powerful platform with a headphone jack! Perfect for 3D sound. There were a few technical gotcha’s, but not really much. On lower power devices, I run the sampling rate down (22k instead of 44k) and limit the number of sounds, but on anything 4S or higher, I didn’t really have to worry about any tech.
That said, as I mentioned above, it turns out that even in real-life, our 3D localization isn’t all that precise. So the challenges were figuring out what was reasonable to expect the player to be able to do. So I ended up focusing on those psychoacoustic 3D sound aspects that are pretty robust, such as lateral positioning and distance, and less on those that we have trouble with, such as front/back distinction. Also, it turns out what we can localize broadband sounds more easily than others—so for sounds where it was important for the player to locate the sound, I’d make the sounds more broadband.
Is Ear Monsters designed as a game for the blind?
Ear Monsters wasn’t designed to be a ‘game for the blind’ per se, but I definitely wanted to make sure the game was accessible to the visually impaired community. It turns out that the iPhone/iOS is the preferred device for the visually impaired. iOS has a technology called VoiceOver designed to make the iPhone/iPad and apps more accessible. So I definitely made good use of that. I also made sure to have aural feedback for things. So as the time’s running down, you her the ‘tick tick’ get faster; as you get extra time awarded, there a special sound for that, etc. A challenge was providing that aural feedback in such a way as to not distract from the main enemy sounds.
Were samples for the game recorded using binaural recordings? If not, how did you manage to put the player in the middle of the action?
Here’s another example of where I cheated used some creative alternatives. Even the best run-time HRTF doesn’t generally sound as good as a great binaural/dummy head recording. So I mixed and matched. For the main monster and player sounds, it’s all run-time HRTF. Those sounds are stored as mono and then processed in real-time by a 3D Sound engine that applies HRTF processing to place the sounds in 3D space. But for some aspects of the game, I play some originally recorded binaural sounds.
In fact, by achieving some game milestones, you can unlock the ability to hear some of those binaural sounds outside the context of the game itself through a 3D Sound Gallery on a separate screen, because binaural sounds are just plain cool to listen to!
Does the game use any sort of music system (dynamic, interactive audio)?
I’d experimented with this, and in the end decided not to have interactive music. The game is relatively short (just a couple of minutes), and I found that changing the music was a bit distracting. The music in general during gameplay is quite sparse—just bass ad drums and almost mono; that’s deliberate because I really didn’t want the music to hinder the gameplay itself, which at times requires some pretty careful listening.
That said, my background ambiences are an important part of the game. I wanted the game to get harder as you got better. One of the ways I do that is to gradually introduce more sonically intrusive ambience layers. When you start the game, it’s very light, but as you progress, the ambience gets denser and busier. Since success depends on being able to hear the Monsters, a denser ambience makes it harder to pick them out; that’s one of the ways we make the game harder as you get higher and higher scores.
Could you tell us more about the HRTF (Head Related Transfer Function) and QSound Labs technology?
This is a technology that QSound designed originally for records and arcade games, but later brought to mobile. I’d had a long working relationship with QSound and worked out an arrangement for EarGames to use it. The engine is full-fledged 3D positional HRTF engine—every sound has its own x,y,z coordinate in 3D space. It’s fairly straightforward—you give it a number of mono sounds as well as source and listener positions for them, type of reverb, low pass filtering you’d like and it takes care of the HRTF filtering, mixing, distance calculations and the like. It was fairly easy to design a game engine around it.
What’s next?
I’ve already got 3 or 4 audio games mapped out on paper with some similar and some new gameplay mechanics and themes. Now that the Ear Monsters engine is solid, we’ll be working on the next ones. One cool thing about games these days it that it’s easy to collect information on how people actually play your game—what they found hard, what they found easy etc. So I’ll be taking that feedback and incorporating it into future games. And I’m looking at porting to some other platforms.
Screenshots
