Composing Music for VR - Part I
Most people that have tried VR with a good headset can agree that even if we have a long way to go, AR and VR are surely going to be huge in the future. Gaming, education, business, all have huge possibilities in the virtual immersive space as well as augmented reality. And we haven't really even began to fathom the endless possibilities these platforms have to offer when technology catches up, which it will.
And then, there is music.
Now, it's been a long time since we got a new music listening format. 5.1 surround never really caught on for music, if we exclude concert films. People didn't want these setups in their houses, and when they did, didn't set them up right. So not a lot of music has been deliberately mixed for 5.1. Sure, the titles are out there but the mainstream market isn't.
Which is a terrible sin, really, because good 5.1 music mixes are wonderful. Like the 5.1 mix of Agent Fresco's great album Destrier. If you have a 5.1 listening setup, I highly recccomend it. Apart from being a great album, it was really mixed properly for 5.1, and when I say mixed properly, I mean musically and with the intention of fully utilizing the medium, not just adding some sloppy reverbs to the back speakers. It was totally reworked for 5.1. surround.
Which leaves us with good old stereo. And look where we are now with that. We mix our music for precise stereo imaging. Next thing we know it's streaming through a single source bluetooth speaker. Music might still sound fine, but the original stereo intention of the mixing engineer is often not portrayed correctly, even with all the modern magic that these otherwise great speakers use for sound projection.
So a virtual 360 mix sounds like a great opportunity, doesn't it? The user has headphones, so no speaker placement problems, level balances or room acoustics to think of. Sounds great. I decided that I wanted to be a part of this revolution after an impressive tech demo from a friend working at CCP Games here in Iceland. Bought a VR rig, and a computer to match.
I started scouring the VR world for titles that incorporated music elegantly mixed for 360. And with very few exceptions, I came up empty handed. Music in gaming is mostly stereophonic, and very often not an integral part of the gameplay, more for cutscenes or opening scenes. When I asked developer friends why that was, they responded with what I thought to be interesting facts. They found that when they mixed music with sound effects in gameplay, our brain had problems processing everything right, so that the effect of the immersion would actually be less, not more. This problem is still as of this writing unsolved, and I have still to find a good VR title that uses 360°mixed music fully, in a way that I like.
But what about straight forward music titles? Turns out there is a lot af music videos packaged for VR. And what do they all have in common? You guessed it. The music is stereophonic while the visuals utilize the VR technology for immersion.
The first thing that I understood immediately when I started composing for 360°is that you have to throw out the rulebook. You have to compose for a new space. Deliberately. It's not enough just to make a track like normal and distribute it around the 360°space. The possibilities are almost endless, and the limitations and challenges are new. You have to decide where to place instruments so that they make sense for the composition, and for the listener, and manage to sound tight and proper in this almost endless space. You have to admit that it sounds like great fun for a musician. And the result?
I'll be quite honest. It dawned on me that audio and music has been an afterthought for a long time in the VR world, where innovation is mostly fuelled by graphics. All the music software solutions I was using made my music sound terrible. Add to that subpar headphones being shipped with most goggles today, and the result was dismal. Phasey sound, without any real definition and feeling of real space. I'll quickly add that real progress is being made here every day, but the material I have composed for 360° is still sounding too crappy for me to release it, using the software solutions I've tried.
But I'm still working on composing it, so someday I will hopefully release a science fiction themed album for 360°, if I find the right technology to make it sound good enough. So let's send the developers of VR music mixing software good vibes and hope they get enough coffee. Hopefully you will hear more about this project later. I won't give up, but we have some way to go. But being the optimist that I am, I named this article Composing for VR - Part I.