Magicians spend hours in front of a mirror, watching and studying every angle of their performance in order to get it just right. They realize that every detail needs to be perfect in order for the audience to believe in the illusion. Even a single mistake can ruin not only the illusion, but the entire performance and credibility of the magician. As harsh as it is, this is no different to what it's like building an AR app. If your app will immerse a user in your world, you need to make it as believable as possible. This includes ensuring that all the virtual objects in a scene look like they belong. Magicians use lighting and perspective tricks to fool the user into believing that something is real. We have already seen how we use perspective, so now we need to cover and enhance our use of lighting.
In this chapter, we will cover how ARCore uses light estimation techniques to make the AR experience more believable to the user. We will then go on to extend some of those basic techniques in order to improve our future AR apps. Here are the main topics we will cover in this chapter:
- 3D rendering
- 3D lighting
- Light estimation
- Cg/HLSL shaders
- Estimating light direction
We will use Unity in this chapter because it provides an easier platform for learning about the rendering process, lighting, and more about shader programs. The shader programs in Unity are a different variety and are definitely worth taking a look at.