The Panorama

As with preparing a delicious meal, the success of our immersive experience depends almost entirely on the quality of our ingredients. Some camera phones allow users to capture a 360-degree panoramic photo natively, whereas others require additional hardware, software, or both. My phone, for example, only allows me to capture 180-degree panoramic images. Of course, in VR, where our canvas is completely immersive, presenting a user with only half of a 360-degree environment isn’t a successful recipe for good VR content.

To follow along with this exercise, feel empowered to use your own original 360-degree panoramic image, or feel free to use the assets included in the repository for this book. You can also make use of the assets available to download through the Unity Asset Store, which you can reach directly through the Unity Application (Figure 2-8).

I have included three different 360-degree images in the Assets folder for this project at the GitHub URL. http://​www.​apress.​com/​source-code. You are free to use these images for this exercise, but if you plan to use them for your own purposes please review the licensing terms provided by the original artists.

The 360-Degree Image vs. the HDRI

A 360-degree image simply describes the orientation of our photo asset. What truly makes a 360-degree image useful for inclusion in our GlobeHopper application is its quality. In our current context, we define the quality of an image as the amount of information an image file contains to help the Unity Engine render a surrounding as similar to reality as possible. We achieve this high quality through the use of high dynamic range images (HDRIs).

The dynamic range of an image measures the amount of color information an image contains. Digital cameras create HDRIs by snapping several frames of a subject at different exposures. This is called bracketing. For example, when your parents took a photograph with an analog, manual camera, they set the exposure level for the subject determined by the level of available light. Today, digital cameras, some by default, capture several photos at different exposure levels ranging from darkest to brightest at a single press of your finger. Although these files are larger in size than traditional JPEG images, the color information they pack is essential to creating the illusion of reality required for an immersive experience.

The steps in this exercise assume you are working with the images provided in the Assets folder for this book. If you are using an original image, please follow the instructions from your camera manufacturer on how to create HDRIs. The final settings for the image we import into our Unity project as an asset will be a 4096 × 2048 image. This is known by some as a 4K resolution asset. As of this writing, 4096 × 2048 is the largest file format allowed by Unity for skymap images. The Unity best practices guide suggests using an image no larger than 2048 × 2048 because that is the maximum size handled by many graphics cards.

The Cubemap

Photographs are two-dimensional, of course. So how are we going to use them to effectively wrap a three-dimensional space? Enter the cubemap. A cubemap is a folding pattern that arranges a set of two-dimensional images into a three-dimensional geometry (Figure 2-9). Unity wraps a cubemap version of our 360-degree HDRI as a texture around a skybox material. Although there are advanced ways you can create a cubemap of your own, Unity provides a convenient feature to handle the manipulation for us. Let’s give it a try!

Let’s start from the very top, as if we’re going to make a recipe from scratch, so we can reinforce what we’ve already covered about integrating VRTK with our Unity project.

Create a new Unity 3D project. Name the project GlobeHopper and save it in a file destination of your choice. It is important that you are able to navigate to the saved destination of your project later in this exercise so note the file path if you must.

Now that we have our 3D project created, let’s make it interactive by importing VRTK. Navigate to the Assets folder of your GlobeHopper project in your File Explorer or Finder. Double-click the Assets folder to make sure you are within the directory GlobeHopper/Assets. If you see a folder labeled Scenes, you are in the right place.

After you’re sure you’ve installed the Git client on your computer and have navigated to the inside of your GlobeHopper Assets folder, right-click in File Explorer and select Open Git Bash Here.

For future reference, you can always find the steps to import VRTK at the VRTK GitHub page: https://github.com/ExtendRealityLtd/VRTK.

Return to Unity and wait as Unity compiles the VRTK scripts you just cloned from GitHub.

If a pop-up menu appears that says “The VRTK Example Scene requires additional Unity Input Axes to be defined,” then click the Add Input Mappings button at the bottom of the menu. If you did not install the VRTK input mappings at this point, it’s okay. We review them in more detail in Chapter 7.

If you haven’t already, also be sure to update the Legacy XR Inputs in the Unity Package Manager to make sure VRTK is fully compatible with your current version of Unity.

The third and final step for our project setup is to confirm that our VR settings are correctly set in our project. To do this, navigate to Edit ➤ Project Settings ➤ Player. On the Player settings tab, navigate to Other Settings and make sure the Scripting Runtime Version is set to .NET 4.x Equivalent, and the Api Compatibility Level is set to .NET Standard 2.0 (Figure 2-10). If you are running Unity on a Mac, then select Mono as your scripting back end. If you are using Windows, select the IL2CPP back end. If you are unsure which scripting back end applies to your project, refer to the Unity web site for further information.

Under XR Settings, select the Virtual Reality Supported Check box. If you do not see your Virtual Reality SDK listed, click the + button on the bottom right of the Virtual Reality SDK menu and select the device for which you intend to build your experience. If the SDK for your headset does not appear in the list, then visit your headset manufacturer’s web site for specific information on how to connect your headset’s SDK to Unity. If you continue to have trouble finding your headset listed as an option, then confirm the build settings of your project (Menu ➤ File ➤ Build Settings (Ctrl+Shift+B) are set to PC, Mac, & Linux Standalone; Universal Windows Platform; or even Android if targeting mobile VR devices like the Oculus Go. If you are targeting Universal Windows Platform in your Build Settings Unity might only give you the option IL2CPP as your scripting back end, which is fine. Check the Build Settings section of the Unity documentation for further reference. When this step is complete you can close the Player Settings window. You might see some yellow exclamation marks in your Console window. As long as none of them are red, you can click Clear at the top left of your Console window and proceed.

Now, we drag our UnityXRCameraRig object from our Project window into our Scene Hierarchy and delete our default camera object. As always, if you do not have a headset mounted to your computer you can select VRTK’s SimulatedCameraRig instead of the UnityXRCameraRig prefab.

Finally, press the Play triangle at the top of your Unity project. If you are using an external headset, move it around to verify that the Unity Game Window tracks your headset’s movement. If you are using the VRTK simulated camera rig, you can confirm your setup works by following the onscreen instructions for manipulating the viewfinder on your virtual camera.

As is, our default scene in Unity is pretty plain. All it is is a camera, a light object mimicking the sun, and a blue sky called a Skybox (Figure 2-11). Let’s see what happens if we swap out that sky for a more immersive representation of the real world.

In the Project window of your GlobeHopper project, right-click the Assets folder and select Create ➤ Folder. Let’s name this new folder Images. Double-click the Images folder to open it. If you’ve downloaded the assets from the repo associated with this exercise, then navigate to the location of the rocky_dawn_4k.hdr file . Drag and drop rocky_dawn_4k.hdr into the Images folder you created in your Unity Project. If you downloaded HDRI content from the Unity Asset Store, the file extensions of your assets might be *.exr and *.mat. Although the file formats are different, the data they represent are the same.

Click once on the file name in your Project window. The file’s details appear in the Inspector window to the right of your scene view in the Default Unity Layout. We’ll learn more about Unity Textures soon. For now, we will leave the Texture Type of our rocky_dawn image asset at its default settings. Its Texture Shape, however, we will change from 2D to Cube (Figure 2-12). Recall that Unity converts our 2D environmental images into a 3D representation by reconfiguring the image into a six-sided cubemap.

After you change the Texture Shape from 2D to Cube, new parameters appear in your Inspector. One of them is labeled Mapping. Using the drop-down menu change the Mapping value from Auto to 6 Frames Layout (Cubic Environment). Leave the remaining values in the Inspector at their defaults (Figure 2-13).

We’ve got our cubemap for our environment, so now what? Now, we have to replace Unity’s default sky with our own image. Let’s navigate to the Skybox settings and see what we can change. In the Unity menu select Window ➤ Rendering ➤ Lighting Settings (Figure 2-14).

The Lighting menu appears with three tabs at its top. We select Scene and turn our attention to the first category listed on the Scene tab, which is Environment. If the Environment drop-down list is collapsed, click the arrow to the left of its name to expand its properties. The first is labeled Skybox Material (Figure 2-15). Excellent! The value for this field is currently set to Unity’s Default-Skybox. Let’s drag and drop the cubemap we created from our Images folder in our Project window into the Skybox Material field of the Lighting menu.

Uh-oh. What happened? Nothing? Good! When you dragged your Rocky Dawn image into the Lighting window, Unity gave you the “not allowed” symbol of a circle with a line through it. What gives?

Well, remember when we set the Texture property of our Rocky Dawn image to a Cube? Unity, as a 3D engine, uses a graphics pipeline common to all 3D asset programs, and in that pipeline there are three unique (although interdependent) categories that inform the way an object looks in 3D space. One is the Texture property, which helps us to render the difference between textures such as sand and brick, for example. Another is the Shader property, which primarily defines the way our objects interact with light in our virtual space. Different shaders can transform the same object from a highly reflective mirror to a muddy, matte puddle of sludge. The third property is an object’s Material. Personally, I find it helpful to imagine a property’s Material as the fabric pulled over the chicken wire skeleton of an object, called its mesh. It is a material to which we can add texture to transform a cylinder from an aluminum pipe to a bratwurst. We’ll play much more with these parameters in later exercises. For now, though, let’s create a Material to replace our current default Skybox material, a new material onto which we will project our Rocky Dawn texture asset.

Let’s repeat the process we performed to create a folder for our images to create a folder called Materials in our project’s Asset folder. Navigate into the Materials folder you created, right-click in the empty folder space of your Project window, and select Create ➤ Material. Let’s name our new material mySkybox. After renaming your new Material asset, select it and turn your attention to the Inspector window to view its properties. The very first property listed in the Inspector is labeled Shader. Unity defaults to its Standard shader, but let’s see if we can find something more specific.

From the Shader drop-down menu, select Skybox ➤ Cubemap (Figure 2-16). Beneath the Rotation property of our material there is a property called Cubemap. To its right is a thumbnail window that allows us to select the cubemap we’d like to project onto this material via the Skybox/Cubemap shader. Click Select in this window, and a new window will appear listing the assets that fit the criteria to act as a cubemap on our new Skybox material (Figure 2-17). Select the rocky_dawn_4k file. In the preview window at the bottom of the Inspector, we see what our Skybox will look like in our scene. Click in the preview window and drag the image left, right, up, and down. Well, it’s definitely a six-sided cube! Let’s add mySkybox to our Skybox Materials field in the Lighting menu (Hint: Window ➤ Rendering ➤ Lighting Settings).

Now, when we drag our new mySkybox material into the box presently occupied by our Default-Skybox material, we receive a + sign that lets us know we can proceed (Figure 2-18). Let’s test this out!

Close the Lighting settings window. If you’re play-testing on an external headset make sure the UnityXR Camera Rig is the only camera in your Scene Hierarchy on the top-left of the Unity Default Layout. Further, verify that the check box to the left of the UnityXR Camera Rig game object is selected in the Inspector when you highlight the UnityXR Camera Rig in the hierarchy. If you’re using the VRTK SimulatedCameraRig, then make sure it’s the only camera game object activated in your Scene Hierarchy. Ready? Press Play and enter your immersive, photorealistic world.

Welcome back! How did it go? Oh no, it’s not right? I’m sorry for leading you astray (not really). We’ll iron out all the wrinkles right now, I promise.

If you play-tested the GlobeHopper app in the previous step you noticed that our six-sided cubemap is quite literally a six-sided cube (Figure 2-19). Well, that might work if we have a cubemap comprised of images of the inside of a box or a near-featureless room, but it’s definitely a nonstarter for an immersive experience intended to re-create the great outdoors. Let’s amend our texture settings to get things Goldi (as in Goldilocks—“just right”).

In the Unity Project Explorer, navigate to the rocky_dawn-4k asset you saved in your Images folder. Select the asset, and in the Inspector window change the value of the texture’s Mapping property from 6 Frames Layout (Cubic Environment) to Latitude-Longitude Layout (Cylindrical) (Figure 2-20). We’ll leave the other settings at their defaults. Click Apply in the Inspector window to save the changes we made to our texture’s Mapping type. Now, let’s try things again, shall we?

Confirm your camera settings are to your preference in your Scene Hierarchy, and press Play.

Hey, that’s much better, isn’t it (Figure 2-21)?

Texture mapping is one of the many tools Unity offers us as a 3D game engine to quickly create photorealistic, immersive experiences. It took humans until the Renaissance to master the complicated mathematics that create the illusion of not only depth, but also curved space. Unity does the same for us in a matter of seconds and a few clicks of a mouse. For further practice, try swapping out our desert setting at dusk for the other HDRIs included in the GlobeHopper repo.