In the preceding chapter, we animated our Robot Drone with some Inverse Kinematics (IK). In this chapter, we will work with Motion Capture (MoCap) on the MetaHuman we created in Chapter 8, Using MetaHuman to Create a Photorealistic Human for UE5.
MoCap, as it is referred to by people in the industry, is just realistic motion captured from real human actors in special MoCap suits that can record all their movements, including the subtleties in 3D animation software. It's then converted to keyframes on a skeleton, like the ones we created for our Alien Plant and Robot. The only difference in this case is that the skeleton is human-like.
MoCap normally has a keyframe on every frame because it captures the full subtlety of organic human motion. If you have a realistic-looking human, like the one we created in MetaHuman, it's good to use something like MoCap.
It's hard and very time-consuming to animate as realistically as real captured motion from real actors, so MoCap has changed the amount of quality content in games and films. I started being a professional animator before MoCap was used. When MoCap started to become accessible to studios, animators thought that they would be replaced by this new technology.
This was not the case. Studios simply started creating more content. MoCap needs a lot of editing and tweaking by people with a good eye for motion, like animators. I spent a large part of my career working with MoCap, editing it, combining different MoCap takes together, and directing cutscenes shot with actors.
In the next two chapters, I will give you an introduction to how to use and edit MoCap in UE5.
In this chapter, we will cover the following:
In this chapter, we will need to work with UE5 and the Mixamo website. You will need the following to complete this chapter:
You can download the complete project from the GitHub repository of this book at https://github.com/PacktPublishing/Unreal-Engine-5-Character-Creation-Animation-and-Cinematics/tree/main/FullFinalUE5Project.
The files related to this chapter are placed at https://github.com/PacktPublishing/Unreal-Engine-5-Character-Creation-Animation-and-Cinematics/tree/main/Chapter18
Mixamo is a free-to-use online MoCap tool and MoCap library. The Mixamo library is subject to change, so I included the downloaded .fbx files from Mixamo in the file source for this chapter. Mixamo is incredibly easy to use. The aim here is to get some free-to-use MoCap and there're lots of different free resources out there if you search for them. Mixamo is just one of them. I hope to teach you more generally about how to work with MoCap. For UE, we preferably want our MoCap source file to be in the .fbx format.
Before we proceed to Mixamo, let's have a think about what MoCap we need for our final sequence. Our female character is going to be in a sitting idle animation, then stand up, and maybe stand there for a moment before walking a few steps forward, stopping, pressing a button on her arm (then the Robot Drone activates), and will then go back to a standing idle animation.
What follows is an example of how I found the MoCap we will use in this chapter on Mixamo. It's probably better to use the files I provided with the chapter since many MoCap files in Mixamo have similar names and later in the chapter, we want to make sure we're using the same source files as the ones I provided. The source FBX files can be found here: https://github.com/PacktPublishing/Unreal-Engine-5-Character-Creation-Animation-and-Cinematics/blob/main/Chapter18/MixamoMocapSource.zip.
You can execute the following steps – just discard the downloaded files afterward:
Once you sign up for a free account, you will have access to the https://www.mixamo.com/ website and its resources.
This is simply their standard female character skinned to a skeleton without any animation on it. To use MoCap in UE, it's better to have a skinned character without any animation on it. This is so that you can have one skinned character file that you can play multiple animations on. It's like our Robot Drone has one skinned skeleton and animation rig but we can play multiple animations on it.
This will download the xbot.fbx file to your computer. We will include this same file in the files accompanying this chapter.
Since we already downloaded the skinned skeleton in Step 6, for the animations, we don't need it again. By selecting, in the options in Step 9, Skin as Without Skin, it downloaded only the skeleton with the MoCap animation on it and not the skinned 3D model too.
For example, I changed the Seat slider from 50 to 40 since that lowered her sitting position, changed the Rate slider from 50 to 60 to make it a bit faster, changed Overdrive from 50 to 60 to exaggerate the movement a bit; and Character Arm-Space from 50 to 58 to make it a bit wider since they were clipping the legs. See the sliders in Mixamo as shown in Figure 18.5.
I hope this is a useful guide on how to get some free MoCap animations from Mixamo. As I said, all these files I downloaded are available in the chapter files at https://github.com/PacktPublishing/Unreal-Engine-5-Character-Creation-Animation-and-Cinematics/blob/main/Chapter18/MixamoMocapSource.zip.
Now that we know how to get free MoCap animations from an external source such as Mixamo, next, let's see how to get MoCap animations from the UE Marketplace itself.
The UE Marketplace is a vast resource of UE-compatible content. Some of it is free, but some of it is paid for since it was created by external content creators that then sell on the UE Marketplace.
Since you have already installed UE, it will have installed the Epic Games Launcher app with it. We can find the UE Marketplace in the Epic Games Launcher software application:
You have now learned how to add MoCap and animation content from the UE Marketplace to your project. Next, we will learn how to use this MoCap on the MetaHuman we created in Chapter 8, Use MetaHuman to Create a Photo-Realistic Human for UE5.
In UE, there's a way to retarget or share animations between character skeletons. Retargeting an animation is like converting it to work and be playable from one skeleton to a different skeleton. So, for example, you can use the MoCap we downloaded from Mixamo on their standard female skeleton and retarget it to play on our MetaHuman skeleton.
These two skeletons have different bones and bone names, and even proportions and sizes, but UE has a built-in way to cleverly map (retarget) and interpret animations between them. But for UE to do that, we need to set it up first.
However, first, let's import the Mixamo MoCap .fbx files into UE.
Here's the process to import Maximo MoCap FBX files into UE:
We want to make sure there's no animation on this import since this will be the clean Skinned Skeleton Mesh that the MoCap animations can play on.
If there is a Message Log window that appears after the import with some warnings, most of the time you can ignore this. It is normally minor things UE could not resolve in the FBX file from an external source. Most of the time, it has no bearing on the visual result of the import.
This is just to keep the animations organized in their own folder, so they're easier to find later.
After clicking Open, the FBX Import Options dialogue window will again appear, but this time, the settings look slightly different because UE can detect that there's no skinned mesh included with the skeleton. UE detects that these are probably animations on the skeleton only. We need to define what skinned mesh we want these animations to play on in UE. We need to select the Mixamo xbot skeleton we imported in Step 2 and Step 3.
The imported animations will appear in the MoCap folder, and they will be represented on the skinned xbot_Skeleton mesh as shown in Figure 18.19.
Now, with our Mixamo xbot Skinned Skeleton Mesh and MoCap imported, we can move on to setting up the retargeting setting for the Mixamo skeleton.
In the next steps, we will set up the retargeting settings on the Mixamo skeleton. The biggest part of this is to confirm to UE which skeleton bone is which body part. UE can then do its clever calculations in the background to map the motion from one skeleton to another.
In the full UE5 release, the way retargeting works has changed significantly from previous versions of UE. Two new Animation Node types are used to achieve retargeting and they are the IK Rig node and the IK Retargeter node.
Let's set up our first IK Rig node on our Mixamo xbot skeleton:
In the IK Rig interface, you can define different body parts by their bone chains. This is a very open system compared to the old way of doing retargeting in UE where it only really worked well on humanoid characters. With this new system, it is much more open and much more powerful, but the user has to use much more of their own common sense to get good results.
Since we're retargeting MoCap from one humanoid character to another, we need to define the main separate body parts that we want to retarget. On a humanoid character, this will typically be Head, Neck, Spine, LeftArm, RightArm, LeftLeg, RightLeg, and Hips.
Now, let's define these body parts by starting with Head.
The new chain will appear in the IK Retargeting list tab as seen in Figure 18.27.
The resulting IK Retargeting list should look as in Figure 18.34.
Congratulations! You've set up your first IK Rig node.
You have probably noticed that we only selected the main bones of every chain. This is roughly in line with the main universal IK or FK chains in these kinds of body parts.
For the arms, we left out things such as fingers, but we included what would be the clavicle joint. The clavicle joint is not normally included in an IK chain, but it has a big influence on the arm in general, so it is better to include it. For the legs, we left out the ball and toe joints since things such as toes and fingers might just complicate things for a simple retarget like this.
Next, we will set the IK Rig node up for the MetaHuman skeleton.
To set up the IK Rig node on the MetaHuman skeleton, the basic process is almost the same as the Mixamo xbot setup. However, because the MetaHuman rig is a far more complex rig with multiple skeletons linked via complex rigging, we just need to make sure we set the retargeting up on the correct asset in the first place.
If the MetaHumans and SciFiGirl assets you created in Chapter 8, Use MetaHuman to Create a Photo-Realistic Human for UE5, are not already in your project file, you can migrate them into your current project using the instructions in Chapter 20, Using Sequencer to Construct Your Final Scene, in the Migrating UE content between projects section. Alternatively, you can download the final example project and get it from within that at https://github.com/PacktPublishing/Unreal-Engine-5-Character-Creation-Animation-and-Cinematics/tree/main/FullFinalUE5Project.
To find the correct skeleton asset, do the following:
Once open, this may visually look weird and broken with just some polygons showing. Don't worry about this, we're still not at the skeleton asset itself. This is the f_med_nrw_body (female med) skinned model. This might be different depending on the style of MetaHuman we created.
Next, we want to make sure we apply Preview Mesh to our skeleton, so we can see our animations clearly when we retarget them.
She will unfortunately have no head. This is normal since, on the MetaHuman, the facial skeleton is separate.
Now, let us set up the IK Rig node on the MetaHuman skeleton.
Just like we did in the Setting up retargeting on the Mixamo skeleton section, we will define the body parts of this skeleton. We will use the same naming convention so it's easier for UE to match up the different limbs.
Note
You will notice that in this skeleton, the number of neck and spine bones is different from the NewIKRig_xbot IK Rig node. This doesn't matter; UE can still interpret this and retarget the animations.
You will notice that I ignore all the extra bones in the arm chain and only select the main arm bones.
The result should look like Figure 18.42:
Retargeting is now set up on both our Mixamo and MetaHuman skeletons. Now comes the fun part. Let's retarget some MoCap animation from the Mixamo source to our female MetaHuman
All the hard work in the retargeting setup has been done now. Compared to the setup, the retargeting itself is the easy part.
To retarget the animations, we need the second kind of node called IK Retargeter. Let's create one:
The new IK Retargeter is now created. Name it xbot_retargeter (see Figure 18.45).
The MetaHuman rig will appear next to the xbot rig. There's also an Asset Browser tab that lists all the animations associated with the xbot.
You will notice that the animation is being retargeted to the MetaHuman rig, but the arms look incorrect. The first thing to check is Chain Mapping. Next to the Asset Browser tab is the Chain Mapping tab.
In the Chain Mapping tab, you can see how UE matched up the body part IK chains we set up on the respective IK Rig assets. Sometimes, UE doesn't get it right, especially if the naming was not the same or close enough. If it is mismatched, you can fix it in the Chain Mapping tab by using the drop-down menus on the Source Chain side and selecting the correct match.
So, we know now that Chain Mapping is not the problem. The other thing it could be is the retarget pose. This is the initial pose UE uses to determine where the limbs are. However, if these poses are too different between the two rigs it is trying to retarget, it can cause problems.
To check and edit these poses, do the following:
Here is our problem. The base retarget poses are too different. The xbot is in a T-pose (looks like a T) and the MetaHuman is in an A-pose (looks like an A) when it comes to the arms. No wonder UE is having problems calculating the retarget. Let's fix that.
As you can see, it is already much better, but you can go back and forth between Edit Pose and testing until you get the best possible results.
Bear in mind that what you are seeing in this editor is just a preview of the retargeting. Once you're happy with the retargeting preview, you can finalize the retarget by exporting the animations to the MetaHuman rig.
You can follow the same process to set up retargeting on the UE Marketplace assets using the Mannequin skeleton or any other skeleton rig for that matter. You can even map a human arm motion to that of a spider leg if you get creative with this kind of retarget setup, so feel free to experiment.
Congratulations, you just retargeted a whole set of Mixamo MoCap animations to the MetaHuman skeleton.
In this chapter, we learned how to export free MoCap animations from the Mixamo library and import them into UE. We learned how to find some animation and MoCap assets on the Unreal Engine Marketplace and import them into our project. We then learned how to set up retargeting on both the Mixamo xbot skeleton and the MetaHuman skeleton. Finally, we retargeted a whole set of MoCap animations to the MetaHuman skeleton.
With this knowledge, you can find and share MoCap and other animation assets between different humanoid characters and you can let your creativity run free. Next, we will learn how to combine and edit this MoCap content on our female MetaHuman.
18.191.168.83