Developing an XR product in Unity involves importing one of several XR software development kits (SDKs) into our Unity project and making some specialized API calls to configure and use the platform at runtime. Each SDK is different in its own way and offers a different set of features. For instance, the Oculus Rift and HTC VIVE SDKs provide APIs to control VR HMDs and their respective controllers, whereas Apple's ARKit provides utilities to determine spatial positioning and superimpose objects on the display. Unity Technologies have been working hard to create APIs that support all of these variations, so the APIs for XR development in Unity have changed a lot over the past few years.

The early days of Unity VR development meant dropping native plugins into our Unity projects, importing SDKs directly from an external developer portal (involving all kinds of annoying grunt work in the setup), and applying updates manually. Since then, however, Unity has incorporated several of these SDKs directly into the editor. In addition, since AR has become more popular recently, the main API has been renamed from UnityEngine.VR to UnityEngine.XR in Unity 2017.2.0 and later, and modified so that it can work with several AR SDKs.

The development experience of working on XR products is a bit of a mixed bag right now. It involves working on some top-of-the-line hardware and software, which means that there are constant changes, redesigns, breakages, patches, bugs, crashes, compatibility issues, performance problems, rendering artifacts, a lack of feature parity between platforms, and so on. All of these problems serve to slow down our progress, which makes gaining a competitive advantage extraordinarily difficult in the XR field. On the bright side, pretty much everyone is having the same issues, so they get a lot of attention from their developers, making them easier to develop with all the time. Lessons are learned, APIs are cleaned up, and new features, tools, and optimizations are made available with every passing update.

Performance problems limit an XR product's success, perhaps more so than non-XR projects because of the current state of the medium. Let's take a look at a few of these performance problems:

• Firstly, our users will be spending significant amounts of money to purchase VR HMDs and sensor equipment or AR-capable hardware. Both of these platforms can be very resource intensive, requiring similarly expensive graphics hardware to support them. This typically leads users to expect a much higher level of quality compared to typical games so that the investment feels worthwhile. To put it another way, this makes poor user experiences understandably less forgivable due to the monetary investment required by the user.
• Secondly, perhaps more so for VR projects than AR ones, poor application performance can lead to serious physical user discomfort, quickly turning even the staunchest advocate into a detractor. In particular, if the frame rate of the XR application is not enough, there will be a discrepancy between the motion the players feel (for example, by rotating the head) and what they see (we will learn about this in more detail later). This leads to the common issue of motion sickness that, in some cases, can last for hours. 
• Thirdly, the XR platform's primary draw is its immersiveness, and nothing breaks that faster than frame drops, flickering, or any kind of application breakdown that forces the user to remove their headset or reboot the app.

Ultimately, we must be prepared to profile our XR applications early to make sure we aren't exceeding our runtime budget, as it will be stretched thin by the complex and resource-intensive nature of the technology behind these media.