3D sound is another illusion we cast at the listener in order to further trick them into believing that our virtually generated world is real. In fact, 3D sound has been used extensively for years in movies, TV, and of course, video games in order to trick the listener into a more immersive experience. In a movie, for instance, the listener is stationary, so 3D sound can be mimicked by setting up multiple speakers. However, in an AR or VR mobile app, the sound needs to come from a single (mono) or double (stereo, headphones) source. Fortunately, numerous smart people figured out how our human ears hear using a technique called binaural sound to map out sounds in 3D. The next diagram goes into a little more detail on how binaural audio works:

3D sound visualized

Since then, we have figured out not only how to record binaural audio, but also how to play it back, thus giving us the ability to play sounds that fool the brain into thinking that their source is different from reality. However, most of the current technology assumes that the user is stationary, but, of course, that is far from the case in an AR app. In an AR app, our user (listener) is moving in our virtual world, which means that the 3D sounds around the listener also need to adjust. Fortunately, Google has again come to the rescue and developed a 3D sound API for AR and VR, called Resonance Audio. We will explore more about Resonance Audio and how to use it in the next section.