Chapter 20. Skeletal Animation
Skeletal animation gets its name from the hierarchical set of interconnected transforms (bones) associated with a model’s mesh (the skeleton’s skin). When these transforms are modified, over time, the mesh is animated. In this chapter, you explore skeletal animation and develop the systems to animate your models.
Hierarchical Transformations
In most of the demonstrations thus far, you have applied transformations—scale, rotation, and translation—to orient your objects in world space. These transformations modify the position of each vertex in an associated model. That’s the basis of animation: If you modify these transformations over time, your object will animate. However, most people wouldn’t consider an object moving or rotating as a whole to be true animation. Animation is typically defined as subelements of an object changing position with respect to the entire model.
Consider the example of human model with a running animation. The model itself could be constructed as just a single mesh, but to make the model run, the vertices in the arms, hips, thighs, lower legs, and feet each need to be transformed separately. However, just as with a real human body, the arms are connected to shoulders, thighs are connected to hips, lower legs are connected to thighs, and so on. And when one part of the body moves, its associated parts move as well. Thus, these transformations must be applied according to a hierarchy, with the final positions of the vertices associated with each node determined by the node’s local transformation—and the transformations of its entire ancestry. This hierarchy is commonly referred to as a skeleton, and each node in the hierarchy is a bone. A bone is just a transformation, and that transformation impacts the vertices associated with the bone and all the vertices down the hierarchy. Figure 20.1 shows an example of the skeleton of a model of a human soldier. At the root of the skeleton is a bone labeled Hips with child bones for LowerBack, RightThigh, and LeftThigh; these bones have children of their own, and on down the hierarchy. Figure 20.2 shows this skeleton surrounded by a mesh, with a few of the bones labeled. Building the skeleton is a process known as rigging and is performed by an artist.
Figure 20.1 The skeletal hierarchy of a human soldier. (Animated model provided by Brian Salisbury, Florida Interactive Entertainment Academy.)
Figure 20.2 The human solider skeleton surrounded by a mesh. (Animated model provided by Brian Salisbury, Florida Interactive Entertainment Academy.)
Skinning
A skeleton is mapped to a mesh in a process known as skinning. This task is performed by an artist and involves associating vertices to specific bones. A vertex can be associated with more than one bone (typically up to four), with each bone influencing the vertex by a given amount (for example, at a joint). Thus, the final position of the vertex is derived from a weighted average of the transformations of the associated bones.
The term skinning also applies to how the vertices are transformed at runtime. CPU skinning indicates that the vertices are transformed on the CPU, while GPU skinning pushes this work onto the GPU. GPU skinning is typically much faster than CPU skinning, although it generally limits the number of bones that can be used. Listing 20.1 presents a GPU skinning shader.
Listing 20.1 The SkinnedModel.fx Shader
#include "include\Common.fxh"
#define MaxBones 60
/************* Resources *************/
cbuffer CBufferPerFrame
{
float4 AmbientColor = { 1.0f, 1.0f, 1.0f, 0.0f };
float4 LightColor = { 1.0f, 1.0f, 1.0f, 1.0f };
float3 LightPosition = { 0.0f, 0.0f, 0.0f };
float LightRadius = 10.0f;
float3 CameraPosition;
}
cbuffer CBufferPerObject
{
float4x4 WorldViewProjection : WORLDVIEWPROJECTION;
float4x4 World : WORLD;
float4 SpecularColor : SPECULAR = { 1.0f, 1.0f, 1.0f, 1.0f };
float SpecularPower : SPECULARPOWER = 25.0f;
}
cbuffer CBufferSkinning
{
float4x4 BoneTransforms[MaxBones];
}
Texture2D ColorTexture;
SamplerState ColorSampler
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = WRAP;
AddressV = WRAP;
};
/************* Data Structures *************/
struct VS_INPUT
{
float4 ObjectPosition : POSITION;
float2 TextureCoordinate : TEXCOORD;
float3 Normal : NORMAL;
uint4 BoneIndices : BONEINDICES;
float4 BoneWeights : WEIGHTS;
};
struct VS_OUTPUT
{
float4 Position : SV_Position;
float3 Normal : NORMAL;
float2 TextureCoordinate : TEXCOORD0;
float3 WorldPosition : TEXCOORD1;
float Attenuation : TEXCOORD2;
};
/************* Vertex Shader *************/
VS_OUTPUT vertex_shader(VS_INPUT IN)
{
VS_OUTPUT OUT = (VS_OUTPUT)0;
float4x4 skinTransform = (float4x4)0;
skinTransform += BoneTransforms[IN.BoneIndices.x] *
IN.BoneWeights.x;
skinTransform += BoneTransforms[IN.BoneIndices.y] *
IN.BoneWeights.y;
skinTransform += BoneTransforms[IN.BoneIndices.z] *
IN.BoneWeights.z;
skinTransform += BoneTransforms[IN.BoneIndices.w] *
IN.BoneWeights.w;
float4 position = mul(IN.ObjectPosition, skinTransform);
OUT.Position = mul(position, WorldViewProjection);
OUT.WorldPosition = mul(position, World).xyz;
float4 normal = mul(float4(IN.Normal, 0), skinTransform);
OUT.Normal = normalize(mul(normal, World).xyz);
OUT.TextureCoordinate = IN.TextureCoordinate;
float3 lightDirection = LightPosition - OUT.WorldPosition;
OUT.Attenuation = saturate(1.0f - (length(lightDirection) /
LightRadius));
return OUT;
}
/************* Pixel Shaders *************/
float4 pixel_shader(VS_OUTPUT IN) : SV_Target
{
float4 OUT = (float4)0;
float3 lightDirection = LightPosition - IN.WorldPosition;
lightDirection = normalize(lightDirection);
float3 viewDirection = normalize(CameraPosition
- IN.WorldPosition);
float3 normal = normalize(IN.Normal);
float n_dot_l = dot(normal, lightDirection);
float3 halfVector = normalize(lightDirection + viewDirection);
float n_dot_h = dot(normal, halfVector);
float4 color = ColorTexture.Sample(ColorSampler,
IN.TextureCoordinate);
float4 lightCoefficients = lit(n_dot_l, n_dot_h, SpecularPower);
float3 ambient = get_vector_color_contribution(AmbientColor, color.
rgb);
float3 diffuse = get_vector_color_contribution(LightColor,
lightCoefficients.y * color.rgb) * IN.Attenuation;
float3 specular = get_scalar_color_contribution(SpecularColor,
min(lightCoefficients.z, color.w)) * IN.Attenuation;
OUT.rgb = ambient + diffuse + specular;
OUT.a = 1.0f;
return OUT;
}
/************* Techniques *************/
technique11 main11
{
pass p0
{
SetVertexShader(CompileShader(vs_5_0, vertex_shader()));
SetGeometryShader(NULL);
SetPixelShader(CompileShader(ps_5_0, pixel_shader()));
}
}
This is just a point light shader that supports skinned models. All the skinning-specific work is performed in the vertex shader, which makes use of the new BoneTransforms shader variable and the BoneIndices and BoneWeights shader inputs. The BoneTransforms variable is an array of matrices that contains the transformation of each bone in the model’s skeleton. The BoneIndices member of the VS_INPUT structure is an array of four unsigned integers that index into the BoneTransforms array. Each BoneWeight element applies the weighted average for vertices that are influenced by more than one bone.
The vertex shader first composes a skinTransform matrix with this code:
float4x4 skinTransform = (float4x4)0;
skinTransform += BoneTransforms[IN.BoneIndices.x] * IN.BoneWeights.x;
skinTransform += BoneTransforms[IN.BoneIndices.y] * IN.BoneWeights.y;
skinTransform += BoneTransforms[IN.BoneIndices.z] * IN.BoneWeights.z;
skinTransform += BoneTransforms[IN.BoneIndices.w] * IN.BoneWeights.w;
If a vertex isn’t influenced by a bone, its weight is 0 and this thereby eliminates the bone’s contribution to the skinTransform matrix. The CPU-side application should guarantee that the bone weights sum to 1.
Next, the vertex position is multiplied by the skinTransform matrix (likewise for the surface normal, tangent, and binormal, if applicable). The bone matrices are still within the object’s local coordinate system, thus the normal WorldViewProjection matrix is applied to move the vertex from animated local space to homogenous clip space before the next shader stage.
Note that the pixel shader is identical to the original point light pixel shader. Indeed, skinned models can be supported for any of the lighting techniques we have presented in this book. All that is required are the additional bone-specific shader variables/inputs we’ve just discussed.
Importing Animated Models
Before you can invoke the skinned model shader, you need to import the animation data for a particular model. This is easier said than done and requires additional plumbing. Thankfully, the Open Asset Import Library (see Chapter 15, “Models”) supports animated models, and you can employ it to at least get animation data from various file formats. However, you have more work to do to get the data into a format your rendering engine can use and to isolate the runtime code base from the Open Asset Import Library (with the idea that transforming data into runtime format is best done as part of a build-time process).
The first class to create is the SceneNode class (see Listing 20.2).
Listing 20.2 Declaration of the SceneNode Class
class SceneNode : public RTTI
{
RTTI_DECLARATIONS(SceneNode, RTTI)
public:
const std::string& Name() const;
SceneNode* Parent();
std::vector<SceneNode*>& Children();
const XMFLOAT4X4& Transform() const;
XMMATRIX TransformMatrix() const;
void SetParent(SceneNode* parent);
void SetTransform(XMFLOAT4X4& transform);
void SetTransform(CXMMATRIX transform);
SceneNode(const std::string& name);
SceneNode(const std::string& name, const XMFLOAT4X4& transform);
protected:
std::string mName;
SceneNode* mParent;
std::vector<SceneNode*> mChildren;
XMFLOAT4X4 mTransform;
private:
SceneNode();
SceneNode(const SceneNode& rhs);
SceneNode& operator=(const SceneNode& rhs);
};
The SceneNode class establishes the hierarchical transformation structure used in skeletal animation. Each SceneNode has a name, a transformation, a parent, and a collection of children. If the parent is NULL, the node represents the root of the hierarchy.
The Bone class derives from SceneNode; Listing 20.3 gives its declaration.
Listing 20.3 Declaration of the Bone Class
class Bone : public SceneNode
{
RTTI_DECLARATIONS(Bone, SceneNode)
public:
UINT Index() const;
void SetIndex(UINT index);
const XMFLOAT4X4& OffsetTransform() const;
XMMATRIX OffsetTransformMatrix() const;
Bone(const std::string& name, UINT index, const XMFLOAT4X4& offsetTransform);
private:
Bone();
Bone(const Bone& rhs);
Bone& operator=(const Bone& rhs);
UINT mIndex; // Index into the model's bone
container
XMFLOAT4X4 mOffsetTransform; // Transforms from mesh space to bone
space
};
A Bone object represents a particular element within the model’s skeleton. Because a Bone object is also a SceneNode object, it shares the same hierarchical structure. The Bone and SceneNode classes are separate to allow transformations within the hierarchy that impact the skeleton but have no vertices associated with them. This might seem odd at first, but it is actually very common. The Bone class extends SceneNode to include an index into the model’s skeleton and an offset transform. The index is used to associate vertices to the shader’s array of bone transformations (the BoneIndices shader input).
The offset transform moves an object from mesh space to bone space. This transformation matrix is necessary because the vertices associated with this bone are specified in mesh space (that is, local/model space), but the bone transformations are in bone space (the bone’s own coordinate system, relative to its parent’s bone). Therefore, to manipulate a vertex by a bone transformation, you must first move it from mesh space to bone space using the offset transform.
With the Bone class defined, you can augment the Model class to contain a set of Bones. More specifically, the Model class should include the following data members:
std::vector<Bone*> mBones;
std::map<std::string, UINT> mBoneIndexMapping;
SceneNode* mRootNode;
The mBones data member is the collection of bones without respect to their hierarchy. The mBoneIndexMapping associates a bone’s name with its index into the mBones collection. This same mapping identifies the Bone::mIndex member, which is the shader’s lookup into the bone transformations array. These separate constructs are a side effect of using the Open Asset Import Library and its approach to storing bone data. These members could potentially be removed if you processed the animation data at build time.
The mRootNode member represents the root node within the transformation hierarchy and is populated after all the bones within the model have been discovered. The Open Asset Import Library stores bone data per mesh, so the actual discovery of the bone data is done through an augmented Mesh class. In particular, the Mesh class is now responsible for finding two pieces of information: the bones themselves and the bone vertex weights. Listing 20.4 shows the declaration of the BoneVertexWeight class.
Listing 20.4 Declaration of the BoneVertexWeight Class
class BoneVertexWeights
{
public:
typedef struct _VertexWeight
{
float Weight;
UINT BoneIndex;
_VertexWeight(float weight, UINT boneIndex)
: Weight(weight), BoneIndex(boneIndex) { }
} VertexWeight;
const std::vector<VertexWeight>& Weights();
void AddWeight(float weight, UINT boneIndex);
static const UINT MaxBoneWeightsPerVertex = 4;
private:
std::vector<VertexWeight> mWeights;
};
An instance of the BoneVertexWeights class is associated with every vertex within the mesh through the member Mesh::mBoneWeights (of type std::vector<BoneVertexWeights>). The Mesh class constructor (where the processing of the Open Asset Import Library mesh structure is performed) should be modified to include the code in Listing 20.5.
Listing 20.5 Bone Processing with the Mesh Class Constructor
if (mesh.HasBones())
{
mBoneWeights.resize(mesh.mNumVertices);
for (UINT i = 0; i < mesh.mNumBones; i++)
{
aiBone* meshBone = mesh.mBones[i];
// Look up the bone in the model's hierarchy, or add it if not
found.
UINT boneIndex = 0U;
std::string boneName = meshBone->mName.C_Str();
auto boneMappingIterator = mModel.mBoneIndexMapping.
find(boneName);
if (boneMappingIterator != mModel.mBoneIndexMapping.end())
{
boneIndex = boneMappingIterator->second;
}
else
{
boneIndex = mModel.mBones.size();
XMMATRIX offsetMatrix = XMLoadFloat4x4(&(XMFLOAT4X4
(reinterpret_cast<const float*>(meshBone->mOffsetMatrix[0]))));
XMFLOAT4X4 offset;
XMStoreFloat4x4(&offset, XMMatrixTranspose(offsetMatrix));
Bone* modelBone = new Bone(boneName, boneIndex, offset);
mModel.mBones.push_back(modelBone);
mModel.mBoneIndexMapping[boneName] = boneIndex;
}
for (UINT i = 0; i < meshBone->mNumWeights; i++)
{
aiVertexWeight vertexWeight = meshBone->mWeights[i];
mBoneWeights[vertexWeight.mVertexId].
AddWeight(vertexWeight.mWeight, boneIndex);
}
}
}
The Open Asset Import Library stores its bone data within an array of aiBone objects. An aiBone object stores the name of the bone, its offset transform, and its list of vertex weights. The code in Listing 20.5 iterates through the list of aiBone objects and attempts to find each bone within the model’s mBoneIndexMapping container. If it doesn’t find a bone, it’s added to the model (after its offset matrix is transposed because the transform is stored in column-major order and is needed in row-major order). This step is necessary because the bones are stored in the model but are spread out across any number of aiMesh objects. Finally, the vertex weights are copied to the Mesh::mBoneWeights collection using the aiVertexWeight::mVertexId member.
Most of the bone data is found within the mesh, but the skeletal hierarchy is part of the aiScene object and is processed within the Model class constructor. More specifically, the aiScene object has an mRootNode member of type aiNode. The aiNode structure is analogous to our SceneNode class, and the aiScene::mRootNode member represents the root of the transformation hierarchy. Processing these nodes is done through the recursive Model::BuildSkeleton() method, presented in Listing 20.6.
Listing 20.6 Processing the Skeletal Hierarchy
SceneNode* Model::BuildSkeleton(aiNode& node, SceneNode*
parentSceneNode)
{
SceneNode* sceneNode = nullptr;
auto boneMapping = mBoneIndexMapping.find(node.mName.C_Str());
if (boneMapping == mBoneIndexMapping.end())
{
sceneNode = new SceneNode(node.mName.C_Str());
}
else
{
sceneNode = mBones[boneMapping->second];
}
XMMATRIX transform = XMLoadFloat4x4(&(XMFLOAT4X4(reinterpret_cast
<const float*>(node.mTransformation[0]))));
sceneNode->SetTransform(XMMatrixTranspose(transform));
sceneNode->SetParent(parentSceneNode);
for (UINT i = 0; i < node.mNumChildren; i++)
{
SceneNode* childSceneNode = BuildSkeleton(*(node.mChildren[i]),
sceneNode);
sceneNode->Children().push_back(childSceneNode);
}
return sceneNode;
}
This method is invoked with the aiSceneNode::mRootNode member and is recursively invoked for each of the node’s children. Note that the hierarchy doesn’t necessarily consist entirely of Bone objects. If the node name matches a Bone in the model, then the bone is used. Otherwise, the node represents a transformation that doesn’t have any associated vertices but nonetheless impacts the transformation hierarchy. In that case, a SceneNode class is added to the hierarchy. You must not skip these nodes.
At this point, you’ve imported all the data required to represent the skeletal structure of the model and its skinning information. The final step is to import the animations that dictate how the skeleton is transformed over time. Animations are composed as a set of keyframes. A keyframe represents a moment in time and records the transformation of a bone at that moment. An animation might be created to match the game’s expected frame rate. For example, a 1-second animation might contain 60 keyframes if the game is expected to run at 60 frames per second. But if an animation has fewer keyframes than your game’s frame rate, you can interpolate between keyframes to produce a smooth animation.
The Open Asset Import Library stores animations within the aiScene::mAnimations data member, an array of aiAnimation objects. Each aiAnimation instance contains the name of the animation, its duration (in ticks), and the number of ticks per second. It also stores sets of keyframes, one for each of the bones involved in the animation. The AnimationClip class is an analogous data structure (which removes the public dependency on the Open Asset Import Library). Listing 20.7 gives its declaration.
Listing 20.7 Declaration of the AnimationClip Class
class AnimationClip
{
friend class Model;
public:
~AnimationClip();
const std::string& Name() const;
float Duration() const;
float TicksPerSecond() const;
const std::vector<BoneAnimation*>& BoneAnimations() const;
const std::map<Bone*, BoneAnimation*>& BoneAnimationsByBone()
const;
const UINT KeyframeCount() const;
UINT GetTransform(float time, Bone& bone, XMFLOAT4X4&2- transform)
const;
void GetTransforms(float time, std::vector<XMFLOAT4X4>&
boneTransforms) const;
void GetTransformAtKeyframe(UINT keyframe, Bone& bone, XMFLOAT4X4&
transform) const;
void GetTransformsAtKeyframe(UINT keyframe,
std::vector<XMFLOAT4X4>& boneTransforms) const;
void GetInteropolatedTransform(float time, Bone& bone, XMFLOAT4X4&
transform) const;
void GetInteropolatedTransforms(float time,
std::vector<XMFLOAT4X4>& boneTransforms) const;
private:
AnimationClip(Model& model, aiAnimation& animation);
AnimationClip();
AnimationClip(const AnimationClip& rhs);
AnimationClip& operator=(const AnimationClip& rhs);
std::string mName;
float mDuration;
float mTicksPerSecond;
std::vector<BoneAnimation*> mBoneAnimations;
std::map<Bone*, BoneAnimation*> mBoneAnimationsByBone;
UINT mKeyframeCount;
};
The sets of keyframes, one for each bone in the animation, are stored in the mBoneAnimations container. The declaration of the BoneAnimation class is presented shortly and simply contains a collection of keyframes for a given bone. This data is processed within the AnimationClip's private constructor, through the implementation in Listing 20.8.
Listing 20.8 Processing Animation Data
AnimationClip::AnimationClip(Model& model, aiAnimation& animation)
: mName(animation.mName.C_Str()),
mDuration(static_cast<float>(animation.mDuration)),
mTicksPerSecond(static_cast<float>(animation.mTicksPerSecond)),
mBoneAnimations(), mBoneAnimationsByBone(), mKeyframeCount(0)
{
assert(animation.mNumChannels > 0);
if (mTicksPerSecond <= 0.0f)
{
mTicksPerSecond = 1.0f;
}
for (UINT i = 0; i < animation.mNumChannels; i++)
{
BoneAnimation* boneAnimation = new BoneAnimation(model,
*(animation.mChannels[i]));
mBoneAnimations.push_back(boneAnimation);
assert(mBoneAnimationsByBone.find(&(boneAnimation->GetBone()))
== mBoneAnimationsByBone.end());
mBoneAnimationsByBone[&(boneAnimation->GetBone())] =
boneAnimation;
}
for (BoneAnimation* boneAnimation : mBoneAnimations)
{
if (boneAnimation->Keyframes().size() > mKeyframeCount)
{
mKeyframeCount = boneAnimation->Keyframes().size();
}
}
}
The BoneAnimation objects are also referenced in the mBoneAnimationsByBone map, which facilitates quick lookup when retrieving a specific bone’s animation data. Also notice the calculation of the mKeyframeCount member. This data member is provided to allow the retrieval of bone transformations (by sequence position instead of time), up to a maximum keyframe. However, each BoneAnimation instance can contain a different number of keyframes. By choosing the largest keyframe count in the set of BoneAnimation objects, you allow retrieval of all available keyframes. You clamp at the last keyframe for BoneAnimation objects with fewer keyframes than the sequence number specified.
Retrieving bone transformations is the job of the AnimationClip::GetTransform*() and GetTransforms*() methods. The singular versions of these methods get the transformations of specific bones. The plural versions collect the transformations of all the bones, according to the retrieval parameters. These methods relay calls to the BoneAnimation class, in Listing 20.9.
Listing 20.9 Declaration of the BoneAnimation Class
class BoneAnimation
{
friend class AnimationClip;
public:
~BoneAnimation();
Bone& GetBone();
const std::vector<Keyframe*> Keyframes() const;
UINT GetTransform(float time, XMFLOAT4X4&transform) const;
void GetTransformAtKeyframe(UINT keyframeIndex, XMFLOAT4X4&
transform) const;
void GetInteropolatedTransform(float time, XMFLOAT4X4& transform)
const;
private:
BoneAnimation(Model& model, aiNodeAnim& nodeAnim);
BoneAnimation();
BoneAnimation(const BoneAnimation& rhs);
BoneAnimation& operator=(const BoneAnimation& rhs);
UINT FindKeyframeIndex(float time) const;
Model* mModel;
Bone* mBone;
std::vector<Keyframe*> mKeyframes;
};
The keyframe data is processed within the BoneAnimation constructor (see Listing 20.10).
Listing 20.10 Processing Keyframe Data
BoneAnimation::BoneAnimation(Model& model, aiNodeAnim& nodeAnim)
: mModel(&model), mBone(nullptr), mKeyframes()
{
UINT boneIndex = model.BoneIndexMapping().at(nodeAnim.
mNodeName.C_Str());
mBone = model.Bones().at(boneIndex);
assert(nodeAnim.mNumPositionKeys == nodeAnim.mNumRotationKeys);
assert(nodeAnim.mNumPositionKeys == nodeAnim.mNumScalingKeys);
for (UINT i = 0; i < nodeAnim.mNumPositionKeys; i++)
{
aiVectorKey positionKey = nodeAnim.mPositionKeys[i];
aiQuatKey rotationKey = nodeAnim.mRotationKeys[i];
aiVectorKey scaleKey = nodeAnim.mScalingKeys[i];
assert(positionKey.mTime == rotationKey.mTime);
assert(positionKey.mTime == scaleKey.mTime);
Keyframe* keyframe = new Keyframe(static
_cast<float>(positionKey.mTime), XMFLOAT3(positionKey.mValue.x,
positionKey.mValue.y, positionKey.mValue.z), XMFLOAT4(rotationKey.
mValue.x, rotationKey.mValue.y, rotationKey.mValue.z, rotationKey.
mValue.w), XMFLOAT3(scaleKey.mValue.x, scaleKey.mValue.y, scaleKey.
mValue.z));
mKeyframes.push_back(keyframe);
}
}
This code merely copies data from aiNodeAnim structures to Keyframe instances. Listing 20.11 gives the declaration of the Keyframe class.
Listing 20.11 Declaration of the Keyframe Class
class Keyframe
{
friend class BoneAnimation;
public:
float Time() const;
const XMFLOAT3& Translation() const;
const XMFLOAT4& RotationQuaternion() const;
const XMFLOAT3& Scale() const;
XMVECTOR TranslationVector() const;
XMVECTOR RotationQuaternionVector() const;
XMVECTOR ScaleVector() const;
XMMATRIX Transform() const;
private:
Keyframe(float time, const XMFLOAT3& translation, const XMFLOAT4& rotationQuaternion, const XMFLOAT3& scale);
Keyframe();
Keyframe(const Keyframe& rhs);
Keyframe& operator=(const Keyframe& rhs);
float mTime;
XMFLOAT3 mTranslation;
XMFLOAT4 mRotationQuaternion;
XMFLOAT3 mScale;
};
Each keyframe consists of a translation, rotation, and scale for a specific time within the animation. The rotation is represented as a quaternion, a four-dimensional extension of complex numbers that make three-dimensional rotation convenient. We’ve been using Euler angles throughout the book to describe rotation in 3D space. Such rotations are represented as a combination of a unit vector (the axis of rotation) and an angle (θ). Quaternions provide an easy way to encode these axis-angle representations in four numbers, which can be applied to a position vector. Quaternions also avoid gimbal lock, the loss of a degree of freedom that occurs when two Euler angles become parallel.
The implementation of the Keyframe class is very simple: It just holds data and exposes it through public accessors. However, the Keyframe::Transform() method deserves a look because it’s the method that combines the translation, rotation, and scale into a transformation matrix.
XMMATRIX Keyframe::Transform() const
{
XMVECTOR rotationOrigin = XMLoadFloat4(&Vector4Helper::Zero);
return XMMatrixAffineTransformation(ScaleVector(), rotationOrigin,
RotationQuaternionVector(),
TranslationVector());
}
This method is employed within the BoneAnimation class and its GetTransform*() methods. You can query uninterpolated bone transformations at a specific time through BoneAnimation::GetTransform(). This method works by finding the latest keyframe whose time position is less than the specified time. The BoneAnimation::GetInterpolatedTransform() method queries the two keyframes surrounding a time position and interpolates the translation, rotation, and scale of the frames. The BoneAnimation::GetTransformAtKeyframe() method returns the transformation at the specified keyframe, or clamps at the last keyframe if the index specified is larger than the bone’s keyframe set (as discussed previously, not all bones necessarily have the same number of keyframes for an animation). Listing 20.12 presents the implementation for each of these methods.
Listing 20.12 Bone Transformation Retrieval
UINT BoneAnimation::GetTransform(float time, XMFLOAT4X4& transform)
const
{
UINT keyframeIndex = FindKeyframeIndex(time);
Keyframe* keyframe = mKeyframes[keyframeIndex];
XMStoreFloat4x4(&transform, keyframe->Transform());
return keyframeIndex;
}
void BoneAnimation::GetTransformAtKeyframe(UINT keyframeIndex,
XMFLOAT4X4& transform) const
{
// Clamp the keyframe
if (keyframeIndex >= mKeyframes.size() )
{
keyframeIndex = mKeyframes.size() - 1;
}
Keyframe* keyframe = mKeyframes[keyframeIndex];
XMStoreFloat4x4(&transform, keyframe->Transform());
}
void BoneAnimation::GetInteropolatedTransform(float time, XMFLOAT4X4&
transform) const
{
Keyframe* firstKeyframe = mKeyframes.front();
Keyframe* lastKeyframe = mKeyframes.back();
if (time <= firstKeyframe->Time())
{
// Specified time is before the start time of the animation, so
return the first keyframe
XMStoreFloat4x4(&transform, firstKeyframe->Transform());
}
else if (time >= lastKeyframe->Time())
{
// Specified time is after the end time of the animation, so
return the last keyframe
XMStoreFloat4x4(&transform, lastKeyframe->Transform());
}
else
{
// Interpolate the transform between keyframes
UINT keyframeIndex = FindKeyframeIndex(time);
Keyframe* keyframeOne = mKeyframes[keyframeIndex];
Keyframe* keyframeTwo = mKeyframes[keyframeIndex + 1];
XMVECTOR translationOne = keyframeOne->TranslationVector();
XMVECTOR rotationQuaternionOne =
keyframeOne->RotationQuaternionVector();
XMVECTOR scaleOne = keyframeOne->ScaleVector();
XMVECTOR translationTwo = keyframeTwo->TranslationVector();
XMVECTOR rotationQuaternionTwo =
keyframeTwo->RotationQuaternionVector();
XMVECTOR scaleTwo = keyframeTwo->ScaleVector();
float lerpValue = ((time - keyframeOne-
>Time()) / (keyframeTwo->Time() - keyframeOne->Time()));
XMVECTOR translation = XMVectorLerp(translationOne,
translationTwo, lerpValue);
XMVECTOR rotationQuaternion = XMQuaternionSlerp
(rotationQuaternionOne, rotationQuaternionTwo, lerpValue);
XMVECTOR scale = XMVectorLerp(scaleOne, scaleTwo, lerpValue);
XMVECTOR rotationOrigin = XMLoadFloat4(&Vector4Helper::Zero);
XMStoreFloat4x4(&transform, XMMatrixAffineTransformation(scale,
rotationOrigin, rotationQuaternion, translation));
}
}
UINT BoneAnimation::FindKeyframeIndex(float time) const
{
Keyframe* firstKeyframe = mKeyframes.front();
if (time <= firstKeyframe->Time())
{
return 0;
}
Keyframe* lastKeyframe = mKeyframes.back();
if (time >= lastKeyframe->Time())
{
return mKeyframes.size() - 1;
}
UINT keyframeIndex = 1;
for (; keyframeIndex < mKeyframes.size() - 1 && time >=
mKeyframes[keyframeIndex]->Time(); keyframeIndex++);
return keyframeIndex - 1;
}
The BoneAnimation::GetInterpolatedTransform() method deserves the closest examination. Here, the two surrounding keyframes are selected and their data are retrieved. This equation determines the interpolation value:
lerpValue = (time - keyframe1.Time) / (keyframe2.Time - keyframe1.Time)
The following scenario demonstrates this calculation:
time = 0.1
keyframe1.Time = 0
keyframe2.Time= 0.5
lerpValue = (0.1 - 0) / (0.5 - 0) = .1 / .5 = 0.2
As we’ve discussed in previous chapters, linear interpolation is calculated as:
lerp (x,y,s) = x * (1 - s) + (y * s)
Thus, in the previous scenario, a lerp value of 0.2 implies that 80 percent of computed value comes from key frame 1, and 20 percent comes from keyframe 2.
The BoneAnimation::GetInterpolatedTransform() method calculates the interpolated translation, rotation, and scale, and uses these values to compute the bone transformation.
Animation Rendering
With all the data imported and the supporting infrastructure in place, you are now ready to render an animation. Rendering an animated model works based on the following steps:
1. Advance the current time.
2. Update the transformations of each bone in the skeleton.
3. Send the updated bone transformations to the skinned model shader.
4. Execute the draw call(s) for the skinned model.
Most of these steps are encapsulated within an AnimationPlayer component (see Listing 20.13).
Listing 20.13 Declaration of the AnimationPlayer Class
class AnimationPlayer : GameComponent
{
RTTI_DECLARATIONS(AnimationPlayer, GameComponent)
public:
AnimationPlayer(Game& game, Model& model, bool interpolationEnabled
= true);
const Model& GetModel() const;
const AnimationClip* CurrentClip() const;
float CurrentTime() const;
UINT CurrentKeyframe() const;
const std::vector<XMFLOAT4X4>& BoneTransforms() const;
bool InterpolationEnabled() const;
bool IsPlayingClip() const;
bool IsClipLooped() const;
void SetInterpolationEnabled(bool interpolationEnabled);
void StartClip(AnimationClip& clip);
void PauseClip();
void ResumeClip();
virtual void Update(const GameTime& gameTime) override;
void SetCurrentKeyFrame(UINT keyframe);
private:
AnimationPlayer();
AnimationPlayer(const AnimationPlayer& rhs);
AnimationPlayer& operator=(const AnimationPlayer& rhs);
void GetBindPose(SceneNode& sceneNode);
void GetPose(float time, SceneNode& sceneNode);
void GetPoseAtKeyframe(UINT keyframe, SceneNode& sceneNode);
void GetInterpolatedPose(float time, SceneNode& sceneNode);
Model* mModel;
AnimationClip* mCurrentClip;
float mCurrentTime;
UINT mCurrentKeyframe;
std::map<SceneNode*, XMFLOAT4X4> mToRootTransforms;
std::vector<XMFLOAT4X4> mFinalTransforms;
XMFLOAT4X4 mInverseRootTransform;
bool mInterpolationEnabled;
bool mIsPlayingClip;
bool mIsClipLooped;
};
The AnimationPlayer class is associated with a model and has an active AnimationClip specified with a call to StartClip(). This method also resets the mCurrentTime and mCurrentKeyFrame members and sets mIsPlayingClip to true. The full implementation of StartClip() is presented here:
void AnimationPlayer::StartClip(AnimationClip& clip)
{
mCurrentClip = &clip;
mCurrentTime = 0.0f;
mCurrentKeyframe = 0;
mIsPlayingClip = true;
XMMATRIX inverseRootTransform = XMMatrixInverse
(&XMMatrixDeterminant(mModel->RootNode()->TransformMatrix()),
mModel->RootNode()->TransformMatrix());
XMStoreFloat4x4(&mInverseRootTransform, inverseRootTransform);
GetBindPose(*(mModel->RootNode()));
}
The last few lines of StartClip() store the inverse root transformation and invoke the GetBindPose() method. The inverse of the model’s root node transformation is the last transform you apply to each bone, so it’s computed once and saved for subsequent use. The GetBindPose() method is a recursive call invoked against the model’s root node and initializes the model into its bind pose. The bind pose is the initial position of a model, before any animation is applied. More specifically, the bind pose is the result of the initial transformations for each bone in a skinned model. Listing 20.14 shows a bottom-up implementation of GetBindPose().
Listing 20.14 Retrieving a Skinned Model’s Bind Pose (Bottom-up)
void AnimationPlayer::GetBindPose(SceneNode& sceneNode)
{
XMMATRIX toRootTransform = sceneNode.TransformMatrix();
SceneNode* parentNode = sceneNode.Parent();
while (parentNode != nullptr)
{
toRootTransform = toRootTransform *
parentNode->TransformMatrix();
parentNode = parentNode->Parent();
}
Bone* bone = sceneNode.As<Bone>();
if (bone != nullptr)
{
XMStoreFloat4x4(&(mFinalTransforms[bone->Index()]),
bone->OffsetTransformMatrix() * toRootTransform * XMLoadFloat4x4
(&mInverseRootTransform));
}
for (SceneNode* childNode : sceneNode.Children())
{
GetBindPose(*childNode);
}
}
This implementation first retrieves the transformation matrix for the scene node. This is a relative transform (relative to the parent node) but is used to initialize the toRootTransform variable. The to-root transform changes the coordinate space of the node from its own bone space to the model space of the root node. If the incoming scene node is the root node (that is, it has no parent), the to-root transform is the relative to-parent transform. Otherwise, the to-root transform is combined with the transform for each ancestor in the hierarchy. The final transform (what’s sent to the skinned model shader) is composed from the offset transform, the to-root transform, and the inverse root transform. Recall that the vertices associated with the bone are originally in model space and that the offset transform moves them to bone space so that the skeleton can transform them.
This version of GetBindPose() is a bottom-up approach because you are traversing the hierarchy from the current node up for each node in the skeleton. Listing 20.15 presents the same method using a top-down approach, which sacrifices some memory to eliminate duplicate multiplications.
Listing 20.15 Retrieving a Skinned Model’s Bind Pose (Top-down)
void AnimationPlayer::GetBindPose(SceneNode& sceneNode)
{
XMMATRIX toParentTransform = sceneNode.TransformMatrix();
XMMATRIX toRootTransform = (sceneNode.Parent() != nullptr ?
toParentTransform * XMLoadFloat4x4(&(mToRootTransforms.at(sceneNode.
Parent()))) : toParentTransform);
XMStoreFloat4x4(&(mToRootTransforms[&sceneNode]), toRootTransform);
Bone* bone = sceneNode.As<Bone>();
if (bone != nullptr)
{
XMStoreFloat4x4(&(mFinalTransforms[bone->Index()]),
bone->OffsetTransformMatrix() * toRootTransform * XMLoadFloat4x4
(&mInverseRootTransform));
}
for (SceneNode* childNode : sceneNode.Children())
{
GetBindPose(*childNode);
}
}
After the animation clip is initialized, the animation can be automatically advanced as time passes. This is accomplished through the overridden Update() method, derived from the GameComponent class. Listing 20.16 presents this method.
Listing 20.16 Advancing an Animation Automatically
void AnimationPlayer::Update(const GameTime& gameTime)
{
if (mIsPlayingClip)
{
assert(mCurrentClip != nullptr);
mCurrentTime += static_cast<float>(gameTime.ElapsedGameTime())
* mCurrentClip->TicksPerSecond();
if (mCurrentTime >= mCurrentClip->Duration())
{
if (mIsClipLooped)
{
mCurrentTime = 0.0f;
}
else
{
mIsPlayingClip = false;
return;
}
}
if (mInterpolationEnabled)
{
GetInterpolatedPose(mCurrentTime, *(mModel->RootNode()));
}
else
{
GetPose(mCurrentTime, *(mModel->RootNode()));
}
}
}
The Update() method first advances time according to the elapsed frame time and the clip’s ticks-per-second property (recall that the animation clip’s duration is stored in ticks, not seconds). If the clip is looped, it will reset the time after the animation has finished; otherwise, the clip stops. If time is left within the playing clip, the final bone transformations are updated through the GetPose() or GetInterpolatedPose() methods. Listing 20.17 presents the GetInterpolatedPose() method. GetPose() is identical, except that it invokes AnimationClip::GetTransform() instead of AnimationClip::GetInterpolatedTransform() to update the toParentTransform matrix.
Listing 20.17 Retrieving the Current Pose
void AnimationPlayer::GetInterpolatedPose(float time, SceneNode&
sceneNode)
{
XMFLOAT4X4 toParentTransform;
Bone* bone = sceneNode.As<Bone>();
if (bone != nullptr)
{
mCurrentClip->GetInteropolatedTransform(time, *bone,
toParentTransform);
}
else
{
toParentTransform = sceneNode.Transform();
}
XMMATRIX toRootTransform = (sceneNode.Parent() != nullptr ?
XMLoadFloat4x4(&toParentTransform) * XMLoadFloat4x4(&
(mToRootTransforms.at(sceneNode.Parent()))) : XMLoadFloat4x4
(&toParentTransform));
XMStoreFloat4x4(&(mToRootTransforms[&sceneNode]), toRootTransform);
if (bone != nullptr)
{
XMStoreFloat4x4(&(mFinalTransforms[bone->Index()]),
bone->OffsetTransformMatrix() * toRootTransform * XMLoadFloat4x4
(&mInverseRootTransform));
}
for (SceneNode* childNode : sceneNode.Children())
{
GetInterpolatedPose(time, *childNode);
}
}
You can also manually advance from frame to frame with the GetPoseAtKeyframe() method, which is invoked through the SetCurrentKeyFrame() method:
void AnimationPlayer::SetCurrentKeyFrame(UINT keyframe)
{
mCurrentKeyframe = keyframe;
GetPoseAtKeyframe(mCurrentKeyframe, *(mModel->RootNode()));
}
The final transforms (what you pass to the skinned model shader) are exposed through the AnimationPlayer::BoneTransforms() accessor. The AnimationPlayer class also exposes accessors for the associated model, current animation clip, current time, and current keyframe, as well as methods for pausing and resuming the clip. Visit the book’s companion website for the complete implementation.
Using the AnimationPlayer is a matter of instantiating the class, starting an animation clip, and either manually advancing the keyframes or updating the AnimationPlayer component. This also implies that you have instantiated a Model object with a file containing a skeletal hierarchy and animations. The book’s companion website contains a sample model in COLLADA (.dae) format. You must also create a material class to interface with the skinned model shader. This material has just one animation-specific variable (BoneTransforms) and creates vertex buffers that include the bone weights and indices. The material’s input layout should have the following elements:
D3D11_INPUT_ELEMENT_DESC inputElementDescriptions[] =
{
{ "POSITION", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, 0, D3D11_INPUT_
PER_VERTEX_DATA, 0 },
{ "TEXCOORD", 0, DXGI_FORMAT_R32G32_FLOAT, 0, D3D11_APPEND_ALIGNED_
ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
{ "NORMAL", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, D3D11_APPEND_ALIGNED_
ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
{ "BONEINDICES", 0, DXGI_FORMAT_R32G32B32A32_UINT, 0, D3D11_APPEND_
ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
{ "WEIGHTS", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, D3D11_APPEND_
ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 }
};
Note the format of the BoneIndices and Weights elements—4×32-bit unsigned integers for the indices and 4×32-bit floating point values for the weights.
Drawing the skinned model is identical to drawing any other models, with the exception that you pass the bone transforms to the shader. For example:
mMaterial->BoneTransforms() << mAnimationPlayer->BoneTransforms();
Here, the mMaterial object refers to an instance of the SkinnedModelMaterial class. An interactive demo is available on the companion website. This demo enables you to vary the options of the animation player and manually step through keyframes. Figure 20.3 shows the animation demo rendering the running soldier depicted at the beginning of this chapter.
Figure 20.3 Output of the animation demo. (Animated model provided by Brian Salisbury, Florida Interactive Entertainment Academy.)
Summary
In this chapter, you explored skeletal animation. You developed the systems to import animation data from the Open Asset Import Library, as well as the components to retrieve and interpolate between keyframes. You also implemented a shader to transform the skinned model’s vertices on the GPU.
Exercises
1. From within the debugger, walk through the code used to import an animated model, to better understand all the moving parts (pun intended). Import a variety of animated models to compare the idiosyncrasies between file formats (which the Open Asset Import Library might not successfully abstract).
2. Explore the animation demo found on the book’s companion website. Vary the options provided to automatically and manually advance keyframes with and without interpolation, and observe the results.
3. Integrate the animation shader with additional lighting models (directional lighting and spotlighting, for example).