Adding Realistic Physical Hands

In this section, you’ll learn to set up realistic hand collisions for your avatar hands . Currently, the way your hands are set up, they pass through game objects without colliding with them, which is not very realistic. Whenever your hand encounters an Interactable game object, it should be allowed either to grab it or, if your intention isn’t to grab the object, to collide with it, allowing you to push the object around. You’ll learn to set this up by using the VRTK’s Trackers Collider Follower prefab.

In the hierarchy, select the VRTK SETUP game object, and create a new child game object and rename this new child game object “VRTK COLLIDER FOLLOWER.” Then, expand the “Packages” folder in the Project tab , locate the Tilia Trackers Collider Follower Unity package, and expand it until you reach its “Prefabs” folder. Drag and drop the Trackers Collider Follower prefab onto the VRTK COLLIDER FOLLOWER game object in the hierarchy. Rename this Trackers Collider Follower game object “Trackers Collider Follower Right,” as this collider follower will track the Interactor on your right hand.

With the Trackers Collider Follower Right game object selected in the hierarchy, ensure that the Inspector’s Collider Follower Facade component has also been expanded. Now, drag and drop the Right Controller Alias game object from the hierarchy into the Tracking Settings Source property parameter, as shown in Figure 21-1. This ensures that the Trackers Collider Follower Right game object will follow your Right Controller (your right hand) around.

Then, expand the Trackers Collider Follower Right game object in the hierarchy and select the Collider game object. You’ll notice that it contains a Sphere Collider component that is currently huge and not positioned well enough to encompass your right hand. You need to update the Sphere Collider property so that it maps your right hand well. In the Sphere Collider component, change the values for its Center property to X = 0.02; Y = -0.02; and Z = 0.015. Also, set the Radius property to 0.04, see Figure 21-2.

Now, playtest the Demo scene, and select the Scene tab in the Unity editor. Double-click the Collider game object in the hierarchy and you should be able to see your right hand in the Unity editor. Click the Edit Collider button in the Sphere component of the Collider game object and ensure that your Sphere Collider is encompassing your right hand well. Adjust the collider properties further if required.

You also need to ensure that as soon as your Right Controller’s Grab button is pressed, the Trackers Collider Follower Right game object will be deactivated. This will guarantee that there’s no conflict between grabbing an Interactable and interacting with it by pushing it around.

You’ll need to hook up your Right Controller’s Grip button so that as soon as it’s pressed, the Trackers Collider Follower Right game object will be deactivated. This will ensure that you can’t push your Interactable game objects around when you’re in Grab mode. Similarly, the moment you ungrab an Interactable game object, the Trackers Collider Follower Right game object will be activated. This ensures that you can push the Interactable game objects around again when you’re not in grabbing mode.

Let’s set this up now. Expand the Buttons Input Actions game object in the hierarchy, and select the Right-Hand Grab game object. In the Inspector, you’ll see that the Activated and Deactivated events for this Right-Hand Grab game object have an animator action setup.

Click the plus symbol located in the bottom right corner of the Activated event to add a new event listener. Then, drag and drop the Trackers Collider Follower Right game object into the event listener box for the Activated event. For the function, select Game Object, Set Active from the “Static Parameters” section, and leave the check box unchecked. When you press your Right Controller’s Grip/Grab button, your Trackers Collider Follower Right game object will be deactivated, see Figure 21-3.

Next, you need to set up the Deactivated event. Click the plus symbol located in the event’s bottom right corner twice to add two new event listeners to it. Drag and drop the Trackers Collider Follower Right game object into the first event listener box. For the function, select Game Object, Set Active from the “Static Parameters” section and ensure that you check its check box. Then, drag and drop the Trackers Collider Follower Right game object into the event listener box for the second Deactivated event listener. For the function, select Collider Follower Facade, Snap to Source from the “Static Parameters” section. This ensures that the moment you release the Grip button on your Right Controller, you’ll be able to push Interactable game objects around, see Figure 21-4.

Playtest the Demo scene using your VR headset. Approach the worktable and push the Hammer, Axe, and Drill Machine around with your open hand. You’ll find that these objects are easy to push around. Now, press the Grip button on your Right Controller so that your hand forms a clenched fist, and then approach any of the Interactable game objects on the worktable. Try pushing around any Interactable game objects using your right hand with your fist clenched and the Grip button on your Right Controller still pressed. You won’t be able to push these Interactable game objects around with your hand in this position. Last, try grabbing an Interactable game object, and note that you can grab it as before, as discussed in Chapter 8. Now, set up a Trackers Collider Follower Left game object for your Left Controller following these same steps. I will leave this as an exercise for you to complete on your own.

As the last step, you need to avoid any uncalled-for jittering behavior that may arise when the Trackers Collider Follower Right game object that is now following your right hand encounters your pseudo-body. In the hierarchy, select the Collider game object, nested within the Trackers Collider Follower Right game object, and set its Layer property to Transparent FX in the Inspector.

Now, from the main menu, select Edit ➤ Project Settings to open the Project Settings dialog. From within the left pane, select Physics. Then, scroll down within the right pane until you see the Layer Collision Matrix. Here, ensure that the check box at the intersection of the Transparent FX row and the Ignore Raycast column has been unchecked, as shown in Figure 21-5.

You can now be assured that the collider following your right hand won’t be able to interact with your pseudo-body but will be able to interact with Interactable game objects.

Set this up for your Trackers Collider Follower Left game object, too.

Obtaining Haptic Feedback

In this section, we’ll become familiar with how to obtain haptic feedback against our Left and Right Controllers whenever we grab an Interactable game object. Note that the haptic feedback provided against the Unity XR Rig is not great, so minimal feedback will be obtained when using the Unity XR Rig.

To get started, expand the “Packages” folder in the Project tab, and locate the Tilia Output Interactor Haptics Unity package and expand it until you reach its “Prefabs” folder. Now, drag and drop the Output Interactor Haptics prefab onto the VRTK SETUP game object in the hierarchy and select it. Ensure that the Interactor Haptics Facade component has been expanded in the Inspector.

We now need to set up the properties available in the “Linked Settings “section. Drag and drop the Camera Rigs, Tracked Alias game object from the hierarchy into the Tracked Alias property parameter. Then, drag and drop the Interactions Interactor Left game object from the hierarchy into the Left Interactor property parameter. Last, drag and drop the Interactions Interactor Right game object from the hierarchy into the Right Interactor property parameter. In the “Haptic Settings” section, set the Intensity property value to 1, as shown in Figure 21-6.

Now, select the Interactions Interactor Left game object in the hierarchy, and in the Inspector, expand the Grabbed event in the “Interactor Events” section. Click the plus symbol in the bottom right corner of this event to add an event listener. Drag and drop the Output Interactor Haptics game object from the hierarchy into this event listener box. For the function, select Interactor Haptics Facade, Perform Default Haptics (Interactor Facade). Then, drag and drop the Interactions Interactor Left game object from the hierarchy into the text box below the function drop-down, see Figure 21-7.

Then, select the Interactions Interactor Right game object in the hierarchy, and in the Inspector, expand the Grabbed event in the “Interactor Events” section. Click the plus symbol in the bottom right corner of this event to add an event listener. Drag and drop the Output Interactor Haptics game object from the hierarchy into this event listener box. For the function, select Interactor Haptics Facade, Perform Default Haptics (Interactor Facade). Last, drag and drop the Interactions Interactor Right game object from the hierarchy into the text box below the function drop-down, see Figure 21-8.

Now, whenever you grab an Interactable game object using your right hand, your Right Controller will emit a short haptic pulse. A similar haptic pulse will be emitted by your Left Controller when you grab an Interactable game object with your left hand.

Highlighting Interactable Game Objects

The VRTK provides you with some very basic shaders that you can use to highlight your Interactable game objects upon touching them. It lets you provide the player with a visual cue indicating that an Interactable game object has been touched. To achieve this, we’ll use one of the six shaders provided by the VRTK. Of course, if you have access to a better-quality shader, you could use that one instead. To achieve this highlighting, the VRTK provides you with a Tilia package called Tilia Visuals Interactable Highlighter Unity . In this section, we’ll have the Hammer game object get highlighted when either of your hands encounters (touches) it.

To set this up, expand the “Packages” folder in the Project tab, and locate the Tilia Visuals Interactable Highlighter Unity package and expand it until you reach its “Prefabs” folder. In the hierarchy, select the VRTK SETUP game object and expand it. Then, drag and drop the Visuals Interactable Highlighter prefab onto the VRTK SETUP game object in the hierarchy and select it.

In the Inspector, ensure that the Interactable Highlighter Facade component has been expanded. Then, in the hierarchy, expand the Environment game object and locate the Hammer game object. Drag and drop the Hammer game object from the hierarchy into the Interactable property parameter. This lets the Interactable Highlighter Facade know that the Hammer Interactable game object should be highlighted upon being touched. For the Highlight Material property , click the target icon in its text box to display the Select Material dialog. Double-click on a material of your choice to serve as the highlight for the Hammer. I’ve selected the material Ambisonic Object , see Figure 21-9.

Now, test the Demo scene using your VR headset. Approach the Hammer and touch it with either your left or right hand, and you’ll see that the chosen Ambisonic Object material immediately gets highlighted on the Hammer. When you actually grab the Hammer, the highlight disappears. Using the Visuals Interactable Highlighter prefab makes it extremely simple to highlight your Interactable game objects. You now need to set up the Axe game object to be highlighted upon being touched. I’ll leave this as an exercise for you to complete.

Summary

In this chapter, you learned a few tips and tricks that can be used within any of your VR games or apps. We started by going over how to set up realistic physical hand collisions against our avatar hands. We achieved this by using the VRTK’s Trackers Collider Follower prefab. We saw how to get the Sphere Collider to encompass our right hand well. We then set up the Activated and Deactivated events to activate and deactivate the Trackers Collider Follower. Finally, we set up the Layer Collision Matrix so that the game objects assigned to the Transparent FX and Ignore Raycast layers wouldn’t collide with each other. In the next section of the chapter, we became acquainted with how to obtain haptic feedback against our controllers upon grabbing an Interactable game object. This was made possible using the VRTK’s Output Interactor Haptics prefab. We learned to set up several properties against this prefab. We also learned to set up the Grabbed event on the Interactions Interactor Left and Right game objects, allowing your controllers (Interactors) to receive haptic feedback. Last, we found out how to have an Interactable game object get highlighted upon touching it using the VRTK’s Visuals Interactable Highlighter prefab. Here, we selected the Interactable object we wanted to highlight and provided it with a suitable highlighting material. We ended the chapter by testing this highlighting mechanic.