The positions and rotations of each skeleton component are available to access at any frame, therefore, if we create a data structure to store them, then a recording (and playback) of motions controlled via the Kinect can be implemented. This is demonstrated in the following recipe.
This recipe assumes you already have a project that uses Zigfu to control a character. You could either start with one of the scenes from the ZDK, or you could also build on the project from the previous recipe, where you created your own block character (that's what we did).
- Open your existing Zigfu project.
- Create a sphere named
GhostHandL, ensuring it is similar in size to the hand of the character in your scene (so this would be (0.2,0.2,0.2) if you are building on the previous recipe). - Add a red material to
GhostHandL. - Create a new
ObjectAtFrameC# script with the following code:// file: ObjectAtFrame.cs using UnityEngine; using System.Collections; public class ObjectAtFrame { public Vector3 position; public Quaternion rotation; public ObjectAtFrame(Vector3 newPosition, Quaternion newRotation) { position = newPosition; rotation = newRotation; } } - Add the following script class to the Main Camera:
// file: RecordMovements.cs using UnityEngine; using System.Collections; using System.Collections.Generic; public class RecordMovements : MonoBehaviour { public Transform leftHand; public Transform redSphere; private bool isRecording = false; private List<ObjectAtFrame> movementList = new List<ObjectAtFrame>(); private int currentFrame = 0; private void OnGUI() { if( !isRecording ) { bool startRecordingButtonClicked = GUILayout.Button ("START recording"); if( startRecordingButtonClicked ) { isRecording = true; movementList.Clear(); } } else { GUILayout.Label ( "RECORDING --------------------" ); GUILayout.Label("number of frames record = " + movementList.Count); bool stopRecordingButtonClicked = GUILayout.Button ("STOP recording"); if( stopRecordingButtonClicked ){ isRecording = false; } } } private void Update () { if( isRecording ) StoreFrame(); else if( movementList.Count > 0) PlayBackFrame(); } private void StoreFrame() { ObjectAtFrame leftHandAtFrame = new ObjectAtFrame(leftHand.position, leftHand.rotation); movementList.Add (leftHandAtFrame); } private void PlayBackFrame(){ currentFrame++; if( currentFrame > (movementList.Count -1)) currentFrame = 0; ObjectAtFrame objectNow = movementList[currentFrame] as ObjectAtFrame; redSphere.position = objectNow.position; redSphere.rotation = objectNow.rotation; } } - With the Main Camera selected in the Hierarchy view, associate the following objects in the Hierarchy view to their corresponding public variables in the Inspector:
- Drag redSphere into the Red Sphere variable
- Drag the left hand of your character into the Left Hand variable
- Run the project. Once the Kinect is controlling your character, click the START recording button, move your left hand in a big circle, and then click the STOP recording button.
- You should then see the red sphere following your recorded left hand movement of a big circle (it will cycle through this recorded movement repeatedly).
Important data structures for this recipe:
- The
ObjectAtFrameclass provides a data structure to store the position and rotation of one part of the character's body for a given frame. - The
RecordMovementsclass attached to the Main Camera, uses a genericListdata structure namedmovementList, to create a dynamic ordered collection ofObjectAtFrameobjects.
When the START recording button is clicked, the isRecording Boolean flag is set to true, and movementList is emptied with the Clear() method. The Update() method is executed each frame, and if isRecording is true, the StoreFrame() method is called.
The StoreFrame() method retrieves the position (Vector3) and rotation (Quaternion) of the Transform component of the left hand of the character (this was associated before running the project, by dragging that left hand object of the character to the public Left Hand variable (leftHand)). A new ObjectAtFrame object leftHandAtFrame is created with the position and rotation values, and added to the end of movementList.
When the STOP recording button is clicked, the isRecording Boolean flag is set to false. The Update() method is executed for each frame, and if isRecording is false, and we have recorded 1 or more frames (that is, the Count() method of movementList is greater than zero), the PlayBackFrame() method is called.
The PlayBackFrame() method increments the currentFame integer, and resets it back to zero if its value is greater than the index of the last element in movementList. The position and rotation of the element for currentFame in movementList is retrieved, and applied to the Transform component of our redSphere object. Thus, the red sphere is made to follow the movement and rotation we recorded for that frame.
Here are some details you don't want to miss.
This recipe has illustrated the "proof of concept" of how to record motion movements of a single skeleton object in Unity with the Kinect device. However, it is more likely that we would want to record the position and rotation of every object in the skeleton. This would require a slightly more sophisticated data structure. If performance was not an issue, perhaps a Dictionary data structure could be used, with identifiers for each part of the skeleton. If efficiency were important, then perhaps parallel arrays, two for each skeleton object, could be implemented (one for position Vector3 objects, and one for rotation Quaternion objects). Using a fixed number of array elements (that is, a maximum number of frames) would be the trade-off of speed against flexibility for this approach.