Spatial Audio for the Audio Metaverse
About Spatial Audio…
So what’s this all about, Spatial Audio? How does it elevate a story or engage an audience, in a way that other types of audio can’t? Why should you consider using it? No, spatial audio is not the same as surround. It’s also totally different from a game with surround sound rendering, like 7.1 or 8D, 11D, or whatever marketing-term audio experience is presented. Apple uses the term Spatial Audio for their protocol, for example, but I think it’s really more of a collective name. I’m not sure if there's one single right definition for it, but one thing most experts agree on, is that spatial audio or 3D-Audio, is an audio reproduction form that takes place and is presented in a virtual space around the listener. It creates the illusion of being surrounded by audio sources which are not physically present in the same room, but reproduced by a speaker setup or headphones. The sound remains in its designated place, regardless of the moving position or viewing direction from the listener… So in fact, it’s a reproduction of how audio behaves in real life. It’s the most natural way of listening, exactly like we’re used to doing in the real world.
We have gone from mono speakers on the phonograph, to stereo systems such as hi-fi and headphones and multi-speaker systems like surround and Dolby Atmos. Each new form came a little closer to reality. Now with VR and all new kinds of immersive media and applications, we’re now able to present an auditory form of telepresence through the use of spatial audio. That ís fantastic and great if you ask me, as an audio fanatic with a passion for spatial audio design. But why should you consider using spatial audio in your next production?
Immersive Experience
Spatial Audio provides an immersive experience. Yup, so what?…
Well, an immersive experience is an experience that completely absorbs us and where we have the feeling of being part of that environment. The better this feeling, the more beautiful the experience, and the better we can achieve the goal of the experience. Especially applications like training, education, eHealth applications (such as stress-reducing experiences) can really benefit from an experience that is as immersive as possible to optimally utilize their goals. In this way, we can create a form of telepresence, giving the feeling that you are somewhere else than you really are.
We want to use as many of our senses as possible in the experience. Because the more senses we use, the better we understand the environment we are in, and therefore the better we can anticipate in that environment. This also allows us to learn procedurally and interactively, and be more focused on our environment. And because of that, we can store the experience in our memory much better. So if you want to create a world in which we are completely absorbed, with the goal of learning something new, remembering important experiences, training soft skills, or completely surrendering to distraction or relaxation purposes, we need to use as many senses as possible.
Of course it can’t be distracting nor dissociating from the experience in the virtual world. This means that for audio, we want to use sounds taking place in the virtual world to behave just like it does in the real world, to optimize the sense of telepresence. So we want to use sound that is heard in the space around us, and not ‘in’ our head. We also want to hear the reflections of the sound sources adapted to the environment and we want to hear a sound that remains at its source, regardless of the viewing direction or the position of the listener. Now, given the above-mentioned definition, Spatial Audio is currently the best method to apply, because it is the only one that meets all these conditions.
Sound around your head
Spatial Audio is in fact nothing more than a technique that ensures audio is played in the space around you. The main difference with surround sound is height and the adaptation to the listener's movements in VR. As mentioned before, it’s the most natural way of listening to audio, just like we’re used to in the real world. With all of these sound sources around you, far or near, you can create a total audio image of the virtual world around you. And in this way, provide information about the environment, add structure and a sense of presence in this virtual environment. The reproduction of spatial audio is best used over headphones and in combination with VR, or a mobile app that monitors our head movements and adjusts the sound accordingly.
You will experience spatial audio through headphones as if the sound is playing outside your headphones, instead of 'in' your head. That’s because a so-called Head-Related Transfer Function (HRTF) filter is used. The way in which we locate audio in the space around us strongly depends on a number of physical characteristics. The size of our head, our torso, for example. But our pinnae is also a personal filter that we use when converting the sound waves into useful information about the direction and distance of the sound. Therefore, this hrtf-filter is applied and mimics the physical characteristics that help us to localize sound. This process is also known as binauralization. There are different sets of filters available and the closer a set matches your appearance, the better your ‘out of the head’ experience. I expect in a few years everyone will have their own personal hrtf-filter set available.
Now there is a difference with binaural audio, which could be called a light version of spatial audio. As soon as these filters have been applied and the file is rendered to a two-channel format, without taking head tracking or position from the listener into account, we speak of Binaural Audio (not to be confused with binaural beats). There are several examples available on streaming services, with the Virtual Barbershop as one of the most famous. The biggest difference with real spatial audio, is the lack of capability of adaptation from the sound image to the position and head movements of the listener.
With true spatial audio, we can simulate audio sources playing around us using an hrtf-set and, in combination with head-rotation metering, making sure it adapts to the listener's position and head movements.
Spatial Audio for everyone
Spatial Audio can actually be applied to any form of audio listening. Also, many consumer manufacturers are embracing and supporting this ‘new’ form of audio experience with their hardware, like Apple’s Airpods Pro, Sony 360 Reality Audio headphones, Samsung Galaxy Buds Pro, and many more to come. Listeners can experience added audio value as an extra dimension to podcasts, music, events, stories, simulators, art routes, museums, games, or any experience that uses the power of sound and needs to immerse the listener even more. And of course, any serious VR production or 360Video will use Spatial Audio to prevent dissociation.
All you need for a Spatial Audio experience is at least a pair of earbuds or headphones and a device that plays the experience, like a mobile phone or HMD. Since nowadays everyone has a mobile phone in their pocket, it’s really easy to listen to spatial audio anywhere. And within a few years, spatial audio will be as common as listening to stereo music in years past. Headphone manufacturers and software companies will increasingly adopt the ability to play spatial audio in their products. And content creators have to adapt their workflows and clients have to adapt their strategies to use the power of sound and apply spatial audio in their media releases or for their productions in any form for the Audio Metaverse.
It’s only a matter of time before we start listening to audio digitally in its most natural form and everyone is able to listen to a perfect personal spatial audio reproduction.
Amaze, Immerse and Impress
With spatial audio, you can amaze, immerse and impress. You can make a great impression and create a truly immersive experience that no one will forget. You only need to make sure you use the best audio for your visual presentation. So if you want to stand out, amaze, and create wow moments, you’ll need to use the power of sound to enhance your visuals, and hear how spatial audio can help your story come alive.
Spatial audio helps people to feel better, learn faster, memorize better, and make sure they are fully immersed in the virtual environment. This will provide better focus, more engagement, and better use of memory, but can also be used for optimal relaxation, brain entrainment, and full immersion for any desired pain distraction. On top of that spatial audio can help develop professional soft skills in VR in a safe and realistic environment, and create improved qualitative experiences that are not only cost-effective but also more environmentally conscious.
Spatial audio provides an optimal telepresence experience with convincing realistic auditory environments. In this way, you not only make more sense of an impression, but you offer an environment where people can learn procedurally and act interactively in a safe and effective way.
I am convinced that spatial audio has already become an integral part of any future production and that these experiences will only get better. With spatial audio, we make the digital world much more beautiful to enjoy, because audio is so important for a really good experience and so functional when applied really well!
Do you want to apply Spatial Audio in your next project and need some advice or a helping hand to integrate the audio workflow within your budget?
Feel free to send me an e-mail or give me a call or just contact me here.