We’re still not done connecting all the dots in setting up fully functional VR hands. We still need to configure the Interactors for both of our hands to provide the all-important ability to grab an Interactable object . Currently, we have our hands set up and the grab animation enabled to respond to a grip press. However, we haven’t yet told the Interactor components on both hands to engage their grab mechanism. In this chapter, you’ll learn how to engage the grabbing mechanism for both Interactors. You’ll also learn about the example avatar cuboid objects and their importance in helping your grab interactions work. We’ll set up a very basic cube Interactable to test whether our Interactor’s grabbing mechanism is working. Finally, we’ll set up velocity trackers on our Left and Right controllers.
Setting Up the Grab Action Property on Interactors to Enable Grabbing
Within the hierarchy, navigate to [VRTK_CAMERA_RIGS_SETUP] ➤ Camera Rigs, Tracked Alias ➤ Aliases ➤ Left Controller Alias ➤ Interactions Interactor Left. In the Inspector, ensure that the Interactor Facade component has been expanded. Within the Interactor Settings section, you’ll notice a Grab Action property that hasn’t been populated. You’ll also notice that this Grab Action property can only accept a Boolean Action component . Suppose this component of the Grab Action property happens to emit a True value. In that case, your Left-hand Interactors grab mechanism will be initiated, allowing you to grab an Interactable.
In the previous chapter, you directed all your Left devices, the Grip, Bumper, and Left mouse button inputs, into an intermediary game object: the Left-Hand Grab. This object contained a Boolean Action component. Now, this is precisely what the Grab Action property on your Left-hand Interactor desires. By connecting the Left-Hand Grab game object to the Grab Action property, you direct all left-hand device, grip (grab) inputs into this Grab Action property. The property can now listen for the Grip button press action of any left controller device. As soon as it receives this input, which if True—that is, if the Grip, Bumper, or Left Mouse button are pressed—will engage its grabbing mechanism for its Left Interactor. The same analogy applies for activating the grabbing mechanism against your Right Interactor, where you connect the Right-Hand Grab game object to its Grab Action property.
If you are building just for Oculus, you can drag and drop only the Oculus Touch Left Controllers, Left Grip Press button action object into the Grab Action property and be done with it. However, if you are building for other devices like the HTC Vive, Windows Mixed Reality devices, and the Spatial Simulator, you will need to keep changing the input for this Grab Action property depending on which device you want to use. Setting up an intermediary game object such as the Left-Hand Grab game object has provided you with a level of indirection where you can channel all possible ways a Left-Hand Grip press can occur into the intermediary game object, Left-Hand Grab. It has enabled you to decouple the grabbing action from other actions, allowing you to provide it to the Grab Action property.
Setting up the Grab Action property for the Left Interactor
Setting up the Grab Action property for the Right Interactor
The grab mechanism for both of your controllers is now functional.
Setting Up an Interactable Object
To test that our Interactors work, we need to set up a basic Interactable object so that our Interactors have something to work with. The Interactable object will be discussed in greater detail later in the book. For now, we’ll set up a very simple cube interactable game object, which will be placed on the table within our scene.
Start by selecting the Environment game object from within the hierarchy. Then, from within the main menu of the Unity Editor, select Game Object ➤ Tilia ➤ Prefabs ➤ Interactions ➤ Interactables ➤ Interactions Interactable. An Interactions Interactable (white cube) game object will be paced within your scene. Expand the Environment game object in the hierarchy and select its Interactions Interactable child game object. Double-click it to have it obtain the focus. You will notice that it got embedded within one of the containers in your scene and is halfway into the floor. Let’s reset its position and scale it so that we have it sitting on the table scaled down in size. Within the Inspector, set its Transform, position property values as follows: X = -12.831; Y = 0.785; and Z = -4.312. Set its Transform, scale property values for X, Y, and Z to 0.1. Now, you’ll see a much smaller-sized white Interactable cube sitting perfectly on your table.
Setting up properties for your Interactable game object (white cube)
Exploring the Example Avatar Cuboid Object
Within the hierarchy, navigate to [VRTK_CAMERA_RIGS_SETUP] ➤ Camera Rigs, Tracked Alias ➤ Aliases ➤ Left Controller Alias ➤ Interactions Interactor Left ➤ Avatar Container ➤ Example Avatar. In the Inspector, you’ll notice that your Example Avatar game object mesh has been set to a cube, and you could even set up a sphere mesh here. Most important, it has a Box collider on it that has been set up as a trigger, according to which its Is Trigger property has been checked. The Box collider on this cube is what enables your Grab interactions to work.
Testing Your Hand Proto Left and Hand Proto Right Game Objects
Setup for testing the grabbing mechanism on Hand Proto Left and Hand Proto Right
Ensure that within the [VRTK_CAMERA_RIGS_SETUP] game object, you have only the Camera Rigs, Spatial Simulator, and Camera Rigs, Tracked Alias game objects active. Then, hit the Play button and wait for your scene to load in Unity’s editor.
Maneuver yourself to move closer to the table by using the W, A, S, and D keys on your keyboard. Position yourself so that your Right hand slightly penetrates the white cube on the table. Now click the right mouse button. You’ll notice that you have grabbed the cube. Upon releasing the right mouse button, the cube will fall from your hand. Now try grabbing the cube with your Left hand by positioning yourself such that your Left hand is penetrating the white cube on the table slightly. Now click the left mouse button. You’ll notice that you have grabbed the cube. Upon releasing the left mouse button, the cube will fall from your hand. You can now grab the white cube using your custom-provided prototype hands.
While still in Playtest mode, select both Example Avatar game objects in the hierarchy and deactivate the Box Collider component within the Inspector. Now return to your Game window and try to grab the white cube with either your Left or Right hand. Whatever you do, you will no longer be able to grab the white cube. This indicates that the Box collider on your cuboid hands is working in conjunction with the Rigidbody on your Interactors to allow your grab mechanism to function. Essentially, you need some collider set up as a Trigger on your custom prototype hands, currently available to you via the Example Avatar game object. Finally, exit Play mode.
The Grab Mechanism with Oculus-Provided Hands
The Example Avatar game object is not only an integral part of your custom-provided prototype hands; it is also required by your Oculus-provided hands to have their grabbing mechanism function.
As shown in Figure 7-3, select one of the Oculus-provided hands, either the Custom Hand Left or Custom Hand Right, from within the hierarchy. Expand the hand until you see its child game objects Grab Volume Big and Grab Volume Small. Upon examining these game objects in the Inspector, you’ll see that both have been fitted with capsule colliders with their Is Trigger property checked. Logically this entails that you won’t require the Example Avatar with its Box collider to be set up as a Trigger to enable the grabbing mechanism when using the Oculus-provided hands, as they already have the required trigger colliders on them. However, it would be interesting to understand at a deeper level exactly how this grab mechanism is being triggered to enable it to function with your Oculus-provided hands.
To engage (trigger) the Interactors Grab mechanism, the collision Trigger event on the Box collider of the Example Avatar game object must reach the Rigidbody on the Interactions Interactor game object. For its grab action mechanism to function, the VRTK requires that a touching collider event be received at its Interactor game object level, where it has a Rigidbody component present. In Unity, Collision Trigger events are propagated only up to the closest Rigidbody available. In the setup where you’re using your Proto hands, you won’t face any issues. There is just one intermediate Trigger Collider in the hierarchy (on the Example Avatar) between your Hand Proto game object and its Interactions Interactor game object. However, this isn’t the case with your Oculus-provided hands, where trigger colliders exist on the game objects, Grab Volume Big, Grab Volume Small, and Example Avatar, thereby not allowing a touching collider event to be received at the Interactor game object level. Here, a Rigidbody component is present, thus not allowing the grabbing mechanism to trigger.
Looking at the Oculus-provided hand, Custom Hand Right, you’ll see that it has capsule colliders set up as a trigger within its child game objects Grab Volume Big and Grab Volume Small. Now, when a collision trigger event occurs, it will only reach as far as the Rigidbody on the Custom Hand Right game object. Its collision trigger event will never get to the Rigidbody on the Interactions Interactor game object. As stated in the previous paragraph, the VRTK requires that, for its grab action mechanism to be engaged, a touching collider event be received at its Interactor game object level, where it has a Rigidbody component present. As this is not the case with your Custom Hand Right game object, the Interactors grab mechanism will never be triggered. Unity’s rule specifying that Collision Trigger events are propagated only up to the closest Rigidbody available ensures that the touching collider event reaches only as far as the Rigidbody on the Custom Hand Right game object.
To work around this setback, you could remove the Rigidbody component on the Custom Hand Right game object. The Collision Trigger events would then propagate up to the Rigidbody on the Interactions Interactor game object. However, the moment you attempt to remove the Rigidbody component on the Custom Hand Right game object, you will be notified that the OVR Grabber component depends on it. Well, you have disabled the OVR Grabber component , so you may as well remove it. Attempting to remove the OVR Grabber component notifies you that the Hand component depends on it. You can’t remove the Hand component as required, and as all these components are dependent on one another, you have no choice but to live with the Rigidbody component on the Custom Hand Right game object.
Ideally, you don’t want to mess around with changing code in the OVR Grabber and Hand components, as you don’t want to break anything in the Oculus-provided hand prefab.
The simplest way is to ignore the capsule colliders and the Rigidbody in the Custom Hand Right game object. You have the Example Avatar game object with its Trigger Box collider available, so all you need is to allow it to be a part of your Oculus-provided hands. This way, you may use its trigger collider to communicate collision trigger events to the Rigidbody on the Interactions Interactor game object.
Setting Up the Example Avatar Object
As you will need to share the Example Avatar game object between the custom prototype hands and the Oculus-provided hands for each of your controllers, you should position this Example Avatar cuboid object appropriately to work well with both types of hands. To ensure this happens, you’ll first align your Hand Proto Left as closely as possible to your Oculus-provided Custom Hand Left. Once they are well aligned and overlapping, you’ll go about aligning the Example Avatar game object appropriately so that the grabbing functionality works well regardless of whether you’re using the custom prototype hands or the Oculus-provided hands. Getting these alignments right requires that you adjust the values and then test to see how it looks. You may want to tweak the provided values to suit your liking. These values have been obtained by physically moving the Hand Proto game object in the scene view and getting it to best align with its Oculus Custom Hand counterpart. This exercise needs to be performed for both Left and Right hands.
Within the hierarchy, ensure that you expand the Camera Rigs, Tracked Alias to replicate what’s shown in Figure 7-3. First, activate both of the Oculus-provided hands: Custom Hand Left and Custom Hand Right. Double-click on either of these hands to enable it to obtain focus. Using the Axis gizmo available in the right top corner within scene view, orient your hands to face forward and preferably with a top view. You’ll notice that your custom prototype hands and Oculus-provided hands are entwined. If you select your Oculus-provided hands, Custom Hand Left and Custom Hand Right, you’ll notice that their Transform Position values across X, Y, and Z are all zero.
Next, from within the hierarchy, select the Hand Proto Left game object. Set the Transform Position values in the Inspector as follows: X = -0.0312; Y = -0.0207; and Z = -0.022. Your left hands have now been aligned. You may want to further fine-tune these values to your liking.
Now let’s align your right hands. From within the hierarchy, select the Hand Proto Right game object. Set the Transform Position values in the Inspector as follows: X = 0.038; Y = -0.02; and Z = -0.015. Your right hands have now been aligned. You may want to further fine-tune these values to your liking.
Last, you need to align the Example Avatar game objects available within your left and right Interactions Interactor game objects. You should align these Example Avatar cuboid game objects to be positioned within the middle of your hands. When these game objects make contact with an Interactable object, and upon pressing the Grip button on your controller, its grabbing mechanism will be triggered. The Example Avatar cuboid game object controls the triggering of the grabbing mechanism, not the hands. The hands only play the appropriate animation to make the grab look realistic.
Select the Example Avatar game object nested in the Interactions Interactor Left game object from within the hierarchy. Set its Transform Position values as follows: X = -0.0076; Y = -0.0298; and Z = -0.0253. You may want to further fine-tune these values to your liking. With the Example Avatar game object still selected, deactivate its Mesh Renderer component within the Inspector. This ensures that you will no longer see the left cuboid hand.
Next, select the Example Avatar game object nested in the Interactions Interactor Right game object from within the hierarchy. Set its Transform Position values as follows: X = 0.0156; Y = -0.024; and Z = 0.008. You may want to further fine-tune these values to your liking. With the Example Avatar game object still selected, deactivate its Mesh Renderer component within the Inspector. This ensures that you will no longer see the right cuboid hand.
Lastly, ensure that you keep either the custom prototype hands or the Oculus-provided hands active. You don’t want both types of hands to be active in the hierarchy. The one you choose to keep active depends on which VR device you’re using.
If you have an Oculus headset, it would be good to test that your grabbing mechanism works with the Oculus-provided hands. Ensure that you activate your Oculus-provided hands and deactivate the custom prototype hands. Also, make sure that Camera Rigs, Oculus Integration is activated and Camera Rigs, Unity XR, Camera Rigs, Spatial Simulator are deactivated.
If you have an Xbox controller hooked to your computer, you can test the grabbing mechanism using the Bumper buttons. You’ll need to ensure that you have the Camera Rigs, Spatial Simulator active.
As locomotion hasn’t yet been set up against your controllers, you’ll still need to move using the W, A, S, and D keys on your keyboard. Locomotion will be dealt with in an upcoming chapter.
Congratulations! You now have your VR hands working and can grab objects, too.
Setting Up Velocity Trackers
When an Interactor grabs an Interactable object, you may require that the Interactor’s velocity be applied to the Interactable object. For example, if you pick up a ball and throw it, you may want the speed you’re moving your controller at to be applied to the ball so that you can control the power of your throw. In VR, it’s mostly your Headset, Left Controller and Right Controller that can move freely in the virtual world, so ideally, it would be one of these objects whose velocity you would track. Your VRTK Camera Rigs, Tracked Alias prefab, already has the relevant Velocity Tracker components on the appropriate aliases.
Velocity Tracker Processor component available on the Headset Alias, Left Controller Alias, and Right Controller Alias
Setting up the Velocity Tracker against the Left Interactor
Setting up the Velocity Tracker against the Right Interactor
Summary
In this chapter, we went over how to get your Interactor grab mechanism to work against your custom prototype hands and the Oculus-provided hands. Setting up the grab mechanism to work with your custom prototype hands was very straightforward, but setting it up for the Oculus-provided hands was not as clear. We had to delve deeper into understanding how collision trigger events are propagated in Unity, up to its closest Rigidbody. We saw how the Rigidbody and colliders provided as part of the Oculus hands are a sort of roadblock in getting your grab mechanism working. You learned how to circumvent this roadblock without the use of code. You then learned how to set up your Example Avatar game object to be shared between your custom prototype hands and the Oculus-provided hands. Finally, you learned how to set up velocity trackers against your VR controllers.