In this chapter, we built an app that is really designed to augment the real world. There are many uses for apps like this, in architecture, design, and retail for example. Our app lets you decorate your walls with framed photos. We developed this project first for HoloLens wearable AR smartglasses, and then we moved it to mobile AR devices using ARKit and Vuforia.

We created an object hierarchy for a picture that includes a frame, an image, a toolbar, and a couple of menus for selecting frames and images. We built a 3D UI framework from scratch that supports toolbars, action buttons, manipulation tools, and modal menus. Then we took a pretty deep dive into the Mixed Reality Toolkit Input Manager and Gaze Recognizer input event systems, as well as the HoloLens spatial mapping room mesh features. Along the way, we used more features of Unity via C# scripts, including tags, audio clips, animations, colliders, and textures.

Then we reworked the project for mobile AR devices. First, we ported to iOS and ARKit. It was a fairly simple process. Most of our implementation architecture was device independent, and we just had to replace the holographic gesture input events with more conventional touch-screen mouse events. Then we ported the project to Android and older iOS devices using Vuforia. In this case, we used image targets to identify walls in lieu of spatial mapping. (And using Google ARCore is similar; see the project at https://github.com/ARUnityBook/).

In the next chapter, we look at another dimension of augmented reality-using physics to interact between the virtual and physical environments. In the project, we will build a ball-toss game that will be fun to play!