Preface

The mid-1990s was when I first encountered virtual reality, and it was a transformative experience. My father, a marine geologist and oceanographer, had just finished mapping the ocean floor off the east coast of Australia with sonar. Using the data they had collected and a giant Silicon Graphics computer, they were able to recreate a 3D rendering of the ocean floor that could be projected onto a large screen. Armed with 3D glasses and a joystick, the user could control a real-time 3D “fly through” along the coast of Australia. I was blown away by this experience. In the mainstream at that time, VR was very uncommon, and AR was virtually unheard of. For most people, their only exposure to these technologies was through their portrayal in futuristic Hollywood movies.

Skip ahead twenty years and VR is still as enchanting as it was then, but it is no longer the exclusive domain of academia or Hollywood fiction. VR and AR are now low cost and accessible to anyone with a mobile phone.

With Google’s powerful suite of AR and VR technologies, the time is ripe for fresh ideas and large-scale innovation in the VR and AR ecosystems. Over the next five years there is set to be a new wave of focus on how these technologies can be used to help solve real-world problems. Creators should be thinking now of ways to utilize AR/VR for positive social change across all fields of endeavor: certainly gaming and entertainment but also education, health care, government, and the environment. I hope this book acts as an enabler to motivate professionals and hobbyists alike to build, not only enchanting, but also socially responsible VR games and apps that produce a positive impact on their users and on the world around us.

Whether you are a UX professional, visual designer, or come from a different design field altogether, this book is a great practical introduction to VR and AR. UX designers and researchers will learn how to build high-fidelity prototypes for evaluation of hypotheses and the validation of designs. As well, they will also learn conceptual information: UX, human-centered design; best practices; and tips, tricks, and insights from my professional experience. These constitute universal themes that apply not only to Google’s products but to all of those in VR and AR.

Software engineers and game developers with preexisting programming knowledge will find this book in-depth enough to be a beneficial resource and focused enough to quickly ramp up in Google’s VR/AR platforms. If you are experienced with Unity, C#, or another programming language or game development platform, it will be a bonus but it is not essential. I cover every step in sufficient detail that beginners will be creating end-to-end games and apps in no time. For advanced developers this book is fast paced enough and sufficiently packed with enough knowledge to keep you interested.

People on the production, management, and accounts side of the product team with experience in managing mobile or webs-based projects will find this book insightful. Running teams of AR/VR developers and designers require specific domain knowledge and an understanding of the full panoply of the product pipeline in the breadth in which this book lays out.

For students, casual creators, and hobbyists with little programming experience but who really want to build some AR or VR games for fun in their spare time, this is the perfect book. They will gain a clear and working introduction of the technology, specificities of the limitations, and, by the end, will have built some entertaining games and apps.

For Daydream development, you will need either:

• A Daydream-ready Android phone¹ and a Daydream View headset

• Or a stand-alone headset²

See Chapters 1 and 2 for more information on the Daydream platform, available headsets, and on setting up a Daydream development environment.

For ARCore development, you will need an ARCore-enabled Android phone running at least Android 7.0 (Nougat).³ All the recipes in this book were built and tested using a Google Pixel XL phone. See Chapters 1 and 7 for more information on ARCore and getting things set up in Unity for AR development.

AR and VR present complex and tricky problems that novices and people unfamiliar with the technologies often find too intimidating to pursue. This book’s aim is to assist and motivate people to continue by propelling them over these early VR humps and arming them with enough tips and tricks to feel comfortable to begin building their own applications.

The chapters are set out in such a way as to gradually introduce concepts that build on each other over time. Frequently, these programming concepts rely on knowledge learned from the previous chapters. However, it is not absolutely necessary to follow the chapters in numerical order. Each chapter stands on its own and all steps are described in detail for every recipe.

The chapters start with an in-depth introduction to the topic of discussion focusing on high-level concepts that encapsulate the core discipline under examination. These concepts are then applied to Unity, Daydream, and ARCore as key technical approaches are introduced. The chapters then jump into as many useful examples as possible. Each recipe builds on a previous one, and most require the supplementary material as a starting point.

• Chapter 1, “VR and AR”: This chapter introduces Google’s VR and AR platforms and places them in an historical context.

• Chapter 2, “Daydream and Unity”: This chapter introduces you to Unity, getting Unity set up, getting your Daydream developer environment set, up as well as installing the Daydream SDK and the Elements Unity package. You then build a template Unity scene to use as the starting point for all the book’s VR recipes.

• Chapter 3, “Getting to Know the Daydream Controller”: The controller is arguably one of the most important things for a Daydream developer to know about. This chapter takes you through absolutely everything you need to know about programming with the controller: from tapping into the various sensors, to using the event system to grab and toss objects in 3D space.

• Chapter 4, “Building UI in VR”: This chapter introduces fundamental interaction design principles. These are important design considerations that should to be taken into account when building useable user interfaces in VR. The chapter takes you step by step from the very basics of using the Unity UI system all the way through to creating a complex UI system that scrolls with the pointer and asynchronously loads the games levels into a scene.

• Chapter 5, “Video and Audio”: The video and audio chapter attempts to demystify much of the complexity around video in VR. It starts by building a simple rectangular video player, going through all the different types of possible video players: 360, stereoscopic, monoscopic, and streaming. The video section ends with a recipe that creates a video player that can dynamically switch between any type of video being presented, all loaded in from a JSON file. The audio section introduces the user to various considerations around audio in VR by showing how to build out several spatialized audio recipes.

• Chapter 6, “Physics Games”: No developer’s knowledge of Unity and VR is complete without knowing how to build physics-based games. This chapter introduces Unity’s powerful physics engine by providing steps for building a whimsical pancake-flipping game, then moves onto some basic weapon mechanics for shooting games, and finally provides steps for building out an end-to-end alien pop cap game.

• Chapter 7, “ARCore”: This chapter introduces everything that you need to know about ARCore: installing it; setting up your development environment; and some essential tips, tricks, and recipes that you’ll be using as boilerplate code for all of your AR apps moving forward.

• Chapter 8, “AR Apps”: This chapter reinforces the importance of building AR apps that focus on solving real user needs. It starts by building a drawing app that lets users draw with their phone in three-dimensional space. The chapter then shows the user how to use Google’s Poly Toolkit for Unity, a repository for developers to download 3D models into their games and apps. You then use the Poly Toolkit and Poly API to build a catalogue-style app that places furniture in the user’s room.

• Chapter 9, “AR Games”: This chapter introduces you to the essential concepts of AR games. You then build out an end-to-end tabletop arcade-style AR game that asynchronously loads in game levels.

• Chapter 10, “Optimization and Performance”: This chapter provides essential but complex knowledge for optimizing Unity Daydream projects using the Profiler and the various tools provided by Daydream.

Listing P.1 Example Script

Click here to view code image

using UnityEngine;

public class ExampleClass: MonoBehaviour {

    public GameObject myGameObject;
    public Text helloText;

    public void Start() {
        helloText.text = "Hello World!";
    }

}

You can find links to all the code and any updates to the book here:

https://www.daydreamVRbook.com/code

Please visit this site before embarking on an epic coding session through the chapters. It is highly likely that there will be regular updates to these platforms. Any required updates to the SDKs, Unity game engine, and bugs that have been encountered will be documented for the affected chapters at the preceding URL.

It is important to note that some of the chapters build apps and games that asynchronously load in other scenes. This means that some of the scenes do not work on their own as they have been designed to load into another scene, or load in another scene (see Chapter 9, “AR Games”). If you come across any scenes that don’t seem to work on their own, please refer to the chapter’s recipe to make sure you are building it correctly.

Each Unity package needs to be imported into a single, new project. Do not try to import multiple chapter packages into a single Unity project as this will cause conflicts. To import the supplementary Unity package into Unity follow these steps:

1. Download the package for the particular chapter from the website: https://www.daydreamVRbook.com/code.

2. In Unity create a new Unity project and then choose Assets > Import Package > Custom Package and select the downloaded Unity package. Select all the assets and import.

3. To Build and Run the scene, you need to set up Unity for either Daydream (see Chapter 2) or ARCore (see Chapter 7).

Many of the C# classes in the supplementary Unity files are wrapped in what is called a C# namespace; however, the code in the book is not. For example, the supplementary code looks like this:

Click here to view code image

using UnityEngine;

namespace ThisIsAnExampleNamespace {

    public class ExampleClass1: MonoBehaviour {

        public GameObject myGameObject;
        public Text helloText;

        public void Start() {
            helloText.text = "Hello World!";
        }

    }
}

A namespace, in this case ThisIsAnExampleNamespace, protects against naming conflicts between class names. Unity does not let you create a class with the same name as an existing class, unless you wrap it in a namespace. Namespaces have been added to most of the classes in the supplementary code so that if you are following along with the book you will not run into any naming conflicts with the finished examples.

The only other thing to know about namespaces is that if you want to reference a namespaced class from another class you will need to add the using directive to include it. For example, the following code includes the namespace ThisIsAnExampleNamespace from the previous block of code and is then able to create an object of type ExampleClass1 that is contained within that namespace:

Click here to view code image

using UnityEngine;
using ThisIsAnExampleNamespace;

public class ExampleClass2: MonoBehaviour {

    public ExampleClass1

If you are new to programming and this code makes you feel anxious, the basics are covered in Chapter 2. This is simply a useful piece of information for developers coming from other languages that don’t use namespacing, such as Objective C.

https://www.daydreamVRbook.com/code

Please make sure to visit this URL before starting a recipe to see whether there have been any updates to the code examples.

1. See the full list of Daydream-ready Android phones at https://vr.google.com/daydream/smartphonevr/phones/.

2. At the time of writing the Lenovo Mirage Solo is the only stand-alone headset available for Daydream. See Chapter 1 for more information.

3. See the full list of ARCore-enabled Android phones at https://developers.google.com/ar/discover/.