Picking Up Objects

With the VRTK, as with all VR tool kits, you can’t just pick up any object. For an object to be grabbable, it must be set up as an Interactable object. In the last chapter, we created an Interactable cube game object that could be picked up. In this chapter, we’ll set up a couple of Interactable objects that you can pick up using either of your controllers (hands).

An Interactable game object can be notified about Interactor actions, such as an Interactor touch. It can also react to input actions, such as a Grip press. Interactable game objects can perform two actions depending upon which Interactor is acting upon it. In the case of the Interactable white cube we set up in the last chapter, either our left or right Interactor could grab it depending on which Interactor grabbed it first.

The first grab attempt from an Interactor is called the Primary Grab action . While this is occurring, any Secondary grab attempt from a different Interactor is called the Secondary Grab action . For example, if you had a long weapon such as a sniper rifle, you would ideally need both hands to manage it. If you were to grab the trigger hold of this sniper rifle with your right hand, then a Primary Grab action would have been performed by your right hand, or right Interactor. To ensure that you can aim well using the scope on the sniper rifle, you will need to support it with your left hand, which means your left hand needs to grab the rifle somewhere on the front to stabilize it. This left-hand grab, in addition to the ongoing right-hand grab, is referred to as the Secondary Grab action. It occurs via a different Interactor; namely, your left Interactor. You may or may not want a secondary grab to occur, the choice of which can be configured on the Interactable via its Interactable Facade component .

As you’ll be testing out most of the properties of your Interactable objects via the Spatial Simulator, ensure that you have deactivated the Oculus-provided hands—namely, the Custom Hand Left and Custom Hand Right—within the hierarchy. Also, ensure that your Hand Proto Left and Hand Proto Right hands have been activated to represent the hands that the Camera Rigs, Spatial Simulator and Camera Rigs, Unity XR can use. Last, make sure that Camera Rigs, Spatial Simulator has been activated and Camera Rigs, Unity XR and Camera Rigs, Oculus Integration have been deactivated.

Select the white cube Interactable object in your Demo scene, available in the Environment game object in the hierarchy. Double-click it to enable it to obtain focus. Scroll down to the Interactable Facade component in the Inspector. You’ll notice that there are two actions an Interactable object can perform: the Primary Action, which is visible within the Primary Action Settings section , and the Secondary Action, which is visible within the Secondary Action Settings section.

First, let’s examine the Primary Action property, which allows you the functionality to pick up (grab). Locate the Primary Action property within the Interactable Facade component in the Inspector. Note that its drop-down value has been set to Interactable Grab Action Follow. This indicates that it should follow its grabbing Interactor (hand) once the Interactable is grabbed. We tested the ability to grab an Interactable object in the last chapter, and it worked well. However, if you want an object to be un-grabbable, probably at a certain point during gameplay, you can change this property’s setting to Interactable Grab Action None. This will ensure that your Interactors can no longer grab the white cube, see Figure 8-1.

Let’s test this out now. Set the drop-down value for the white cubes’ Primary Action property to Interactable Grab Action None. Now playtest the Demo scene using the Spatial Simulator and pick up the white cube sitting on the table with your left or right hand. You won’t be able to, as its Primary Action property has been set to Interactable Grab Action None. Reset the Primary Action property to Interactable Grab Action Follow.

The drop-down action, Custom, available to both the Primary and Secondary Action, is utilized if you intend to code the Grab action yourself. In this case, the Interactable custom editor won’t override any custom changes.

Now, look at the Grab Action Settings section located above the Primary Action Settings. Here, the Grab Type property has been set to Hold Till Release. This indicates that you must have the Grip button on your controller pressed continually to hold onto an Interactable object. The moment you release the Grip button, an ungrab will occur and the object will fall out of your hand. You’ve already tested this out in the last chapter and know that it works.

Next, set the Grab Type property to Toggle and let’s test it out. We can playtest your Demo scene using the Spatial Simulator. Pick up the white cube using your right hand by clicking the right mouse button. But don’t keep the right mouse button pressed down continually; release it once you have grabbed onto the cube. Note that the cube stays attached to your hand. To ungrab the white cube, simply click the right mouse button again. You’ll notice that the cube falls from your hand. That’s how the Toggle option works. A single mouse button click or Grip button press grabs the Interactable object, while a subsequent mouse button click or Grip button press ungrabs it, see Figure 8-1.

The Follow Tracking Property

With the white cube selected in the hierarchy and its Interactable Facade component expanded in the Inspector, locate the Follow Settings section and then the Follow Tracking property that has been set to Follow Transform. The option Follow Transform will allow your cube Interactable object to pass through other objects with colliders on them. Let’s try it out.

Hit the Play button to playtest your Demo scene using Unity’s editor. Grab the white cube on the table and walk forward using the W key on your keyboard to approach the container in front of you. If you try pushing the white cube through the container, you’ll notice that it passes through easily. Using the W, A, S, and D keys, maneuver yourself close to the ladder. Now try pushing the white cube through the sides of the ladder or any of its rungs. You’ll find that it passes through easily. Both the ladder and container have been fitted with colliders. However, your white cube doesn’t collide with them; instead, it passes through. This may be the effect you desire or possibly not. Now, hit the Play button to stop your Demo scene from playing.

Suppose you don’t want your Interactable objects passing through other objects that have colliders on them. In that case, you’ll need to set the Follow Tracking property on your Interactable object to Follow Rigidbody . Let’s try it out now. After setting the Follow Tracking property to Follow Rigidbody, hit the Play button to playtest your Demo scene using Unity’s editor. Grab the white cube on the table and walk forward by pressing the W key on your keyboard to approach the container in front of you. Try pushing the white cube through the container and you’ll see that it won’t pass through. Using the W, A, S, and D keys, maneuver yourself so that you’re close to the ladder. Now try pushing the white cube through the sides of the ladder or any of its rungs. You’ll see that it won’t pass through. Instead, it collides and rolls about as if being dragged around by your hand. This may be the outcome you desire. Figure 8-2 shows the Follow Tracking property set to the value Follow Rigidbody. The other options available in the Follow Tracking property are used with the Angular drives provided by the VRTK that we’ll explore in a later chapter. Now, set the Follow Tracking property to Follow Transform.

Picking Up an Interactable Object Using Precise Grab Points

A precision grab gives you the ability to grab an Interactable object at the point your Interactor collides with it. To set this up, all you need to do is set the Grab Offset property to Precision Point. To demonstrate a precision grab, we’ll use a lifelike object, which you’ll need to download from the Unity Asset store at https://assetstore.unity.com/packages/3d/props/industrial/workplace-tools-86242. Log in to the Unity Asset store and search for the freely available asset “Workplace Tools.” Download and import these tools into your project. After that’s complete, you’ll see that a new “Workplace Tools” folder has been added to your “Assets” folder.

Before looking at the Grab Offset property, let’s explore the white cube Interactable game object available in the Environment game object of the hierarchy. Select and expand your white cube game object and then expand its Mesh Container game object, and you’ll notice the cube game object you’ve been interacting with so far. Select this object in the hierarchy and look at the Inspector. You’ll see that it comprises a cube mesh with a default material and a Box Collider. To make any object Interactable, you must change the Mesh property here and use an appropriate collider that correctly encompasses the new object’s mesh shape. Another way to make an object Interactable is to drag a prefab of the object (that contains a mesh) onto the Mesh Container game object nested within the white cube object, making a child of the game object. Then, you need to disable the cube game object nested within the Mesh Container game object. Let’s set this up now so that you can replace your white cube mesh with the Drill Machine prefab available in your “Workplace Tools” folder.

Within the “Assets” folder in your Project tab, navigate to the “Workplace Tools” folder and select the “Prefabs” folder . You’ll see that the right pane contains a Drill prefab. Ensure that the Mesh Container game object, nested within the white cube game object, has been selected. Drag and drop the Drill prefab onto the Mesh Container game object selected in the hierarchy, making it a child. Finally, disable the cube object nested within the Mesh Container game object. Now, you’ve replaced your default dull-white cube mesh with a more realistic Drill machine. In the hierarchy, select the parent white cube game object and rename it “Drill Machine.”

Let’s now scale, position, and set up appropriate colliders for your Drill machine. Select and expand the Drill game object nested within the Mesh Container game object. With the Drill game object selected, right-click it, and from the context menu that pops up, select Prefab ➤ Unpack Completely. Then, select the Drill Machine game object in the hierarchy and set the Transform Scale property for X, Y, and Z to 0.25. Next, set its Transform Position values as follows: X = -12.831; Y = 0.8436; and Z = -4.213. This ensures that your Drill machine will sit well on the table. Double-click the Drill Machine game object in the hierarchy to enable it to obtain focus so that you can ensure that it is sitting well on the table. You may want to fine-tune these values to your liking.

Now ensure that the Drill game object nested within the Drill Machine has been expanded in the hierarchy. Select its Body, Top Front, and Trigger Main child game objects in the hierarchy. From within the Inspector, remove the Mesh Collider component from all of these child game objects. (Mesh colliders are expensive.) Now select the Body child game object in the hierarchy and add a Box collider component to it. Use the Edit Collider property within the Box collider component to ensure that the Box collider encompasses the Drill Machine well. The default Box collider settings worked well for me. You don’t need to set up any collider for the Top Front and Trigger Main child game objects, even though you can if you want to. Figure 8-3 displays the setup for your Drill machine.

Let’s now proceed to set up the Grab Offset property so that you can grab your Drill Machine at a precise point. With the Grab Offset property value set to None, grabbing an Interactable object will result in the Interactable’s origin point snapping to the Interactor’s origin point. If you were to playtest your Demo scene now, this is how grabbing the Drill Machine would work. Go ahead and give it a try.

You’ll see that this works well for basic grabs, like grabbing your cube. However, you may want to grab an Interactable object from a precise predefined position. You may even want to orient it to your Interactor in a particular position and rotation. You can achieve a precision point grab using the Grab Offset property available in the Interactable Facade component. Now that we have a Drill machine that has a distinct shape, let’s set up its Grab Offset property so that it can be grabbed at a predefined precise point. Currently, with its Grab Offset property set to None, you can grab it on its handle, which is its origin point.

Precision Point allows you to grab your Drill Machine Interactable object at the precise point at which your hand’s Interactor collides with it. Select the Drill Machine game object in the hierarchy. In Interactable Facade component of the Inspector, locate the Grab Offset property and set its value to Precision Point, as shown in Figure 8-4.

You can playtest your scene in Unity’s editor by hitting the Play button. Once your Demo scene loads, approach your Drill Machine and try grabbing it at its Base using either your left or right hand. Note how you can grab the machine at a precise point. Now try grabbing the Drill Machine from the top, somewhere up-front, and note again how you can grab it precisely at the point you choose. Finally, hit the Play button to stop your Demo scene from playing.

Custom Pickup Placements

To make it easier to interact with your Interactable item and make it feel more realistic, you may want the grabbing Interactor to position and rotate the Interactable item in a specific way when you pick the Interactable item up. This is an effective technique to use when you want your grabbed Interactable items to be oriented differently when grabbed.

To achieve the specific orientation you desire, you’ll need to set the Grab Offset property value on your Drill Machine to Orientation Handle . Go ahead and set this property value and you’ll see that a Show Orientation Container button immediately shows up below the Grab Offset property setting. Click this button and within the hierarchy window, your Drill Machine game object will automatically expand to display its Orientation Handles child game object, as shown in Figure 8-5.

Now, expand the Orientation Handles game object in the hierarchy until you see the Generic Orientation Handle game object, as shown in Figure 8-6. Select this game object, and set the Transform Position values in the Inspector as follows: X = 0; Y = 0.5; and Z = -0.5. Set the Transform Rotation value for the X-axis to -90.

Test your scene in Unity’s editor by hitting the Play button. Approach your Drill Machine, and grab it with either your left or right hand. You’ll see that you have grabbed it by its rear top end.

You can fine-tune the grab position to your liking by adjusting the Generic Orientation Handle game object’s position in the scene editor. This way, you can ensure that your hand grab looks more realistic, without being embedded within the Drill machine when grabbed.

With the Generic Orientation Handle game object still selected in the hierarchy, rename it “Generic Orientation Handle Right.” This will be the grab orientation of your Drill machine when you grab it with your right hand. For your left hand, we’ll set up a different grab orientation by duplicating the Generic Orientation Handle Right game object, renaming it “Generic Orientation Handle Left,” and selecting it. It will remain a child of the Orientation Handle Collection game object. Set the Transform Rotation value for its X-axis to 90.

You now need to provide your orientation handles logic for using the Generic Orientation Handle Left and Right game objects whenever you grab the Drill machine with either hand.

In the hierarchy, select the Orientation Handles game object. Ensure that the Game Object Relations component has been expanded in the Inspector. Then, expand its Elements property , and you’ll see that its size value is set to 1 by default. Change the size value to 2, and you’ll see that two element slots, Element 0 and Element 1, are made available. Expand both slots. You’ll see that the Value property for both slots has been populated with the Generic Orientation Handle Right game object. The Value property for Element 0 is fine. Now, drag and drop the Generic Orientation Handle Left game object from the hierarchy into the Element 1 slot Value property.

You now have two different orientation handles that you can choose from when grabbing your Drill Machine, see Figure 8-6.

You now need to instruct the Game Object Relations component, shown in Figure 8-6, about which Interactor needs to be paired up with which orientation handle. You’ve set up the Generic Orientation Handle Right to work with your right hand and the Generic Orientation Handle Left to work with your left hand. To let your Interactors know which orientation handle they should be paired up with, you need to associate your Interactions Interactor Right, available in your Right Controller Alias, with the Key property of the Element 0 slot.

Likewise, you need to associate your Interactions Interactor Left, available in your Left Controller Alias, with the Key property of the Element 1 slot.

Expand the Camera Rigs, Tracked Alias game object in the hierarchy until you locate its Interactions Interactor Left game object nested within the Left Controller Alias. Next, with the Camera Rigs, Tracked Alias game object still expanded, locate the Interactions Interactor Right game object nested within the Right Controller Alias.

Within the hierarchy, you should have your Orientation Handles game object for the Drill Machine still expanded. Select this Orientation Handles game object. In the Inspector, you need to populate the Key properties for the Element 0 and Element 1 slots. Scroll down within the hierarchy so that you can see the Camera Rigs, Tracked Alias game object. Drag and drop the Interactions Interactor Right game object from the hierarchy into the Key property of the Element 0 slot of the Game Object Relations component. Then, drag and drop the Interactions Interactor Left game object into the Key property of the Element 1 slot in the Game Object Relations component.

Now, look at the Game Object Relations component in the Inspector. It clarifies things to know that the Interactions Interactor Right game object has been paired with the Generic Orientation Handle Right game object, as well as that the Interactions Interactor Left game object has been paired with the Generic Orientation Handle Left game object. This can be seen in Figure 8-7.

Using this simple setup, you can create many more orientation handles and then associate them with the Value property of the Game Object Relations component to grab an object so that it may be oriented differently.

Now, playtest your scene within Unity’s editor by hitting the Play button. Once your Demo scene loads, approach your Drill Machine and grab it using your right hand, noticing the way it is oriented (facing downward). Drop the Drill Machine from your right hand onto the table and then grab it using your left hand. You will notice that it oriented itself differently (facing upward) than when you right hand grabbed it. The orientation for each hand changes based upon the way your generic orientation handles are set up.

Adding Secondary Grab Actions to Interactable Objects

So far, you have set up a Primary Grab action on your Interactable object. The first grab attempt from an Interactor against an Interactable object is called a Primary Grab action. A secondary grab attempt from a different Interactor against the same interactable object that takes place while the primary grab is occurring is called a Secondary Grab action. The next section will show you to set up some commonly used Secondary Grab actions against your Interactable object.

Swapping Objects between Hands with a Secondary Grab Action

The most common Secondary Grab action that you can use is the Interactable Grab Action Swap. This Secondary Grab action allows you to swap an Interactable object back and forth between your hands.

For testing out these Secondary Grab actions, you need to use your VR headset. I would advise that your switch to the Camera Rigs, Unity XR setup, as you can test both Oculus and HTC Vive devices using this setup. Ensure that you deactivate Camera Rigs, Oculus Integration and Camera Rigs, Spatial Simulator. The Camera Rigs, Tracked Alias should always be active.

Select the Drill Machine located within the Environment game object in the hierarchy. Ensure that its Interactable Facade component has been expanded. Locate the Secondary Action property, which can be found within the Secondary Action Settings section. Select the Interactable Grab Action Swap option from its drop-down, as shown in Figure 8-8.

Now, hit the Play button in the Unity editor to playtest your Demo scene using your VR headset. As soon as it starts playing, mount your VR headset. Grab the Drill Machine with your right hand, and then, using your left hand, grab the Drill Machine out of your right hand. You will notice that it snaps to your left hand. Thus the Interactable object is now being swapped between your hands. Swap the Drill Machine back and forth between your hands to get a feel of the Swap Secondary Grab action working. Once satisfied, exit the Play mode.

Now set the Secondary Action property to Interactable Grab Action None. Hit the Play button within the Unity editor to playtest your Demo scene, again using your VR headset. Mount your VR headset as soon as your the scene starts playing. Grab the Drill Machine with your right hand, and then, using your left hand, try grabbing the Drill Machine out of your right hand. You will notice that you won’t be able to. This is because you have set the Secondary Action property to Interactable Grab Action None.

Performing a Two-Handed Grab with a Secondary Grab Action

A two-handed grab is handy to use on longer objects, like a rifle, that need to be supported by a secondary hand. It provides secondary hand support and allows you to guide the direction of the object using your secondary hand.

Before setting up this two-handed Secondary Grab action, ensure that the Grab Type property for your Drill Machine has been set to Hold Till Release. Also, set the Grab Offset property to Precision Point.

With the Drill Machine game object still selected in the hierarchy, set its Secondary Action property to Interactable Grab Action Control Direction, as shown in Figure 8-9. This Interactable Grab Action Control Direction setting allows you to perform this two-handed grab.

Hit the Play button within the Unity editor to playtest your Demo scene using your VR headset. As soon as the scene starts playing, mount your VR headset. Grab the Drill Machine with your right hand, which is now your primary grabbing hand, and then grab the Drill Machine somewhere at the front with your left hand. After that, move your left hand up, down, left, and right. Notice how you can control the direction your Drill Machine faces by simply maneuvering your left hand. This Secondary Grab action would be beneficial if you created a First Person Shooter (FPS) game that used long weapons that require a two-handed grab.

Scaling an Interactable Object with a Secondary Grab Action

You may find the need to scale the size of an object. For example, if you have a virtual tablet device within your app, you may want to provide the user the flexibility to scale this virtual tablet to make it either larger or smaller. This is easily achievable by setting the virtual tablet game objects’ Secondary Action property to Interactable Grab action Scale. In this section, you’ll set up the Secondary Action property on your Drill Machine to allow it to be scaled either up or down in size.

With the Drill Machine game object still selected in the hierarchy, set its Secondary Action property to Interactable Grab Action Scale, as shown in Figure 8-10. This Interactable Grab Action Scale setting is what allows you to scale your Drill machine in size.

Hit the Play button in the Unity editor to playtest your Demo scene using your VR headset. As soon as the scene starts playing, mount your VR headset. Grab the Drill Machine somewhere at the top with your right hand. Then, perform a secondary grab somewhere at the bottom of the Drill Machine with your left hand. After that, move both hands (controllers) away from each other to see your Drill Machine scale up in size. Now, move both hands (controllers) toward each other to see your Drill Machine scale down in size.

Creating a Unity Layer for Interactable Objects

Finally, let’s set up a Unity layer to which all Interactable objects within your scene will be assigned. We’ll call this layer “Interactable.” All interactable objects in your project must be assigned to this Layer.

From within the hierarchy, select the Drill Machine game object. Within the Inspector, click the Layer drop-down and select Add Layer. Ensure that the Layers drop-down is expanded in the Tags and Layers dialog box in the Inspector pane. In the Text Box for the first-available vacant User Layer, type in “Interactable” as the layer name. In the hierarchy, select the Drill Machine game object again. Note that within the Inspector, the Layer still shows “Default,” as you have only created the Layer so far and not yet assigned it to your Drill Machine game object. Now, from the Layer drop-down in the Inspector, select the Interactable layer. You will be asked whether you want to set the layer to Interactable for all child objects. Click No, This Object Only.

For this newly created Interactable layer, you need to set up its Collision Matrix, which determines whether game objects assigned to different layers and to the same layer can collide.

From within Unity Editors main menu, select Edit ➤ Project Settings to bring up the Project Settings dialog. From the pane on the left, select Physics. You’ll see your Layer Collision Matrix on the right with your newly added Interactable layer showing. Ensure that you uncheck the boxes Interactable and Ignore Ray Cast at the intersection of layers. You don’t want the objects in these two layers colliding with each other, which you’ll learn the reasons for when you add a Pseudo body in a later chapter. Also, uncheck the boxes Interactable and Interactable at the intersection between layers. You may keep these checked if you want your Interactables to interact with each other. For now, though, leave them unchecked, as shown in Figure 8-11.

Note that you can access the Tags and Layer dialog to create new Tags or Layers from within Project Settings.

Last, navigate to the Example Avatar game objects, nested within the Avatar Container of your Interactions Interactor Left and Interactions Interactor Right game objects. Select these Example Avatar game objects, and note that they have been assigned to the Ignore Ray Cast Layer by default. The layer setting for these Example Avatar game objects will be recalled again in a later chapter.

Summary

In this chapter, we took a deep dive into setting up interactable objects within the VRTK. We began by looking at how a Primary Grab action differs from a Secondary Grab action. We went on to set up and test different property values for the Primary Grab action. We tested out the two settings available to the Grab Type property. We then examined the Follow Tracking property and learned how to set it up to disallow an Interactable object from passing through other objects in our scene. We then imported a free asset from the Unity Asset Store in order to set up an Interactable object that is more realistic than the white cube in the scene. We learned how to set up the new Drill Machine game object to become an Interactable object. We then explored the Grab Offset property and learned how to pick up a Drill Machine using a precise point. We went over how to set up orientation handles against an interactable object, which allowed us to grab an object and orient it differently for each of our hands. We learned how to add Secondary Grab actions, including swapping an object back and forth between our hands, performing a two-handed grab against an object, and scaling an object in size, to our Interactable object. Last, we set up a Unity layer that we could assign all interactable objects in our scene too.