In this chapter, we took a proverbial dive into the deep end—or the deep learning end—of the pool. We started by talking about the importance of ML and what applications we can use it for in AR. Then, we looked at how ML can use various methods of learning from unsupervised, supervised, and reinforcement learning in order to teach an ML agent to learn. We then looked at a specific example of learning ML algorithms, called neural networks and often referred to as deep learning. This led us to build a simple neural network that you can also use to learn the intricacies of neural networks on your own. NNs are very complex and not very intuitive, and it is helpful to understand their basic structure well. We then trained this network on a very simple dataset to notify the user if they get too close to an object. This led to a further discussion of how NNs train with back propagation using the gradient descent algorithm. After that, we looked at an enhanced example that allows you to train the network to recognize an area or object. Finally, we looked at the current king of ML, TensorFlow, and looked at a quick example of what is possible and what is coming soon.

In the next chapter, we get back to building a practical example with ARCore. We will build a simple design app that lets the user virtually decorate their living space.