Chapter 14
Level Files and Binary Data
This chapter explores how to load and save JSON-based level files representing the game world. These level files store global properties as well as properties of all the actors and components in the game.
In addition, this chapter explores the trade-offs of using text-based file formats versus binary file formats. As an example, it discusses an implementation of a binary mesh file format.
Level File Loading
To this point, this book hasn’t used a data-driven approach to the placement of objects in the game world. Instead, the Game::LoadData function code dictates the actors and components in the game, as well as global properties, such as the ambient light. The current approach has several disadvantages, most notably that even small changes, such as placement of a cube in a level, requires recompilation of the source code. A designer who wants to change the placement of objects in the game shouldn’t have to change the C++ source code.
The solution is to create a separate data file for the level. This data file should be able to specify which actors the level contains and which properties and, optionally, adjust the components of these actors. This level file should also include any needed global properties.
For a 2D game, using a basic text file works perfectly well. You can simply define different ASCII characters for different objects in the world and create a text grid of these objects. This makes the level file look like ASCII art. Unfortunately, this approach doesn’t work very well for a 3D game because each object in the game world could be at some arbitrary 3D coordinate. Furthermore, in the game object model used in this book, actors can have components, so you may need to also save properties of each attached component.
For all the reasons just listed, you need a file format that’s more structured. As with the rest of the book, in this chapter you once again use a text-based JSON format for data. However, this chapter also explores the trade-offs that any text format makes, as well as techniques needed for binary file formats.
This section explores building up a JSON-level file format. You start with global properties and slowly add additional features to the file so that the Game::LoadData function has barely any code other than a function call that specifies the level file to load. Unlike earlier chapters, this chapter explores the usage of the RapidJSON library to parse in the JSON file.
Loading Global Properties
The only global properties the game world really has are the lighting properties—the ambient light and the global directional light. With such a limited number of properties, this is a good starting point for defining the JSON level file format. Listing 14.1 shows how you might specify the global lighting properties in the level file.
Listing 14.1 Level with Global Lighting Properties (Level0.gplevel)
{
"version": 1,
"globalProperties": {
"ambientLight": [0.2, 0.2, 0.2],
"directionalLight": {
"direction": [0.0, -0.707, -0.707],
"color": [0.78, 0.88, 1.0]
}
}
}
Listing 14.1 shows several constructs that you commonly encounter in a level file. First, at its core, a JSON document is a dictionary of key/value pairs (or properties) called a JSON object. The key name is in quotes, and then the value follows the colon. Values can be of several types. The basic types are strings, numbers, and Booleans. The complex types are arrays and JSON objects. For this file, the globalProperties key corresponds to a JSON object. This JSON object then has two keys: one for the ambient light and one for the directional light. The ambientLight key corresponds to an array of three numbers. Similarly, the directionalLight key corresponds to another JSON object, with two additional keys.
This nesting of JSON objects and properties drives the implementation of the parsing code. Specifically, you can see common operations where, given a JSON object and a key name, you want to read in a value. And in your C++ code, the types you have are far more varied than the JSON format, so you should add code to assist with parsing.
To parse these global properties in code, you begin by declaring a LevelLoader class. Because loading the level from a file affects the state of the game, but not the level loader itself, you declare the LoadLevel function as a static function, as follows:
class LevelLoader
{
public:
// Load the level -- returns true if successful
static bool LoadLevel(class Game* game, const std::string& fileName);
};
Note that in addition to the filename, the LoadLevel function takes in the pointer to the Game object. This is necessary because creating or modifying anything requires access to the game.
The first step in LoadLevel is to load and parse the level file into a rapidjson::Document. The most efficient approach is to first load the entire file into memory and then pass this buffer to the Parse member function of the Document. Because loading a JSON file into a Document is a common operation, it makes sense to create a helper function. This way, gpmesh, gpanim, and any other asset types that need to load in a JSON file can also reuse this function.
Listing 14.2 shows the implementation of LoadJSON. This function is also a static function. It takes in the filename and a reference to the output document. The first step loads the file into an ifstream. Note that you load the file in binary mode instead of text mode. This is for efficiency purposes because all you need to do is load the entire file into a character buffer (array) and pass that buffer directly to RapidJSON. You also use the std::ios::ate flag to specify that the stream should start at the end of the file.
If the file loads successfully, you use the tellg function to get the current position of the file stream. Because the stream is at the end of the file, this corresponds to the size of the entire file. Next, you have the seekg call set the stream back to the beginning of the file. You then create a vector with enough space to fit the entire file plus a null terminator and have the read function read the file into the vector. Finally, you call the Parse function on outDoc to parse the JSON file.
Listing 14.2 LevelLoader::LoadJSON Implementation
bool LevelLoader::LoadJSON(const std::string& fileName,
rapidjson::Document& outDoc)
{
// Load the file from disk into an ifstream in binary mode,
// loaded with stream buffer at the end (ate)
std::ifstream file(fileName, std::ios::in |
std::ios::binary | std::ios::ate);
if (!file.is_open())
{
SDL_Log("File %s not found", fileName.c_str());
return false;
}
// Get the size of the file
std::ifstream::pos_type fileSize = file.tellg();
// Seek back to start of file
file.seekg(0, std::ios::beg);
// Create a vector of size + 1 (for null terminator)
std::vector<char> bytes(static_cast<size_t>(fileSize) + 1);
// Read in bytes into vector
file.read(bytes.data(), static_cast<size_t>(fileSize));
// Load raw data into RapidJSON document
outDoc.Parse(bytes.data());
if (!outDoc.IsObject())
{
SDL_Log("File %s is not valid JSON", fileName.c_str());
return false;
}
return true;
}
You then call LoadJSON at the start of LoadLevel:
rapidjson::Document doc;
if (!LoadJSON(fileName, doc))
{
SDL_Log("Failed to load level %s", fileName.c_str());
return false;
}
Given a JSON object, you need to read in keys and extract their corresponding values. You shouldn’t assume that a given key will always be there, so you should first validate that the key exists and matches the expected type. If it does, you read in the value. You can implement this behavior in another class with a static function called JsonHelper. Listing 14.3 shows the JsonHelper::GetInt function. It tries to find the property, validates that it matches the expected type, and then returns true if successful.
Listing 14.3 JsonHelper::GetInt Implementation
bool JsonHelper::GetInt(const rapidjson::Value& inObject,
const char* inProperty, int& outInt)
{
// Check if this property exists
auto itr = inObject.FindMember(inProperty);
if (itr == inObject.MemberEnd())
{
return false;
}
// Get the value type, and check it's an integer
auto& property = itr->value;
if (!property.IsInt())
{
return false;
}
// We have the property
outInt = property.GetInt();
return true;
}
You can then use the GetInt function in LoadLevel to validate that the loaded file’s version matches the expected version:
int version = 0;
if (!JsonHelper::GetInt(doc, "version", version) ||
version != LevelVersion)
{
SDL_Log("Incorrect level file version for %s", fileName.c_str());
return false;
}
Here, the JSON object in question is the overall document (the root JSON object). You first make sure that GetInt returns a value and, if it does, you check that its value matches the expected value (a const called LevelVersion).
You also add similar functions to JsonHelper to extract other basic types: GetFloat, GetBool, and GetString. However, where this paradigm really becomes powerful is for non-basic types. Specifically, many properties in this game are of type Vector3 (such as ambientLight), so having a GetVector3 function is very useful. The overall construction of the function is still the same, except you need to validate that the property is an array with three members that are floats. You can similarly declare a GetQuaternion function.
Ambient and Directional Lights
With the helper functions in place, you can create a function to load in the global properties. Because the global properties are varied and may not necessarily need the same class types, you must manually query the specific properties you need. The LoadGlobalProperties function in Listing 14.4 demonstrates how to load the ambient light and directional light properties. Notice that for the most part, you call the helper functions you’ve created for these properties.
Note that you can access a property as a rapidjson::Value& directly through operator[]. The dirObj["directionalLight"] call gets the value with the directionalLight key name, and then the IsObject() call validates that the type of the value is a JSON object.
Another interesting pattern for the directional light is to have direct access to variables you want to set. In this case, you do not need to add any conditional checks on the GetVector3 calls. This is because if the property requested does not exist, the Get functions guarantee not to change the variable. If you have direct access to a variable and don’t care if the property is unset, then this reduces the amount of code.
Listing 14.4 LevelLoader::LoadGlobalProperties Implementation
void LevelLoader::LoadGlobalProperties(Game* game,
const rapidjson::Value& inObject)
{
// Get ambient light
Vector3 ambient;
if (JsonHelper::GetVector3(inObject, "ambientLight", ambient))
{
game->GetRenderer()->SetAmbientLight(ambient);
}
// Get directional light
const rapidjson::Value& dirObj = inObject["directionalLight"];
if (dirObj.IsObject())
{
DirectionalLight& light = game->GetRenderer()->GetDirectionalLight();
// Set direction/color, if they exist
JsonHelper::GetVector3(dirObj, "direction", light.mDirection);
JsonHelper::GetVector3(dirObj, "color", light.mDiffuseColor);
}
}
You then add a call to LoadGlobalProperties in LoadLevel, immediately after the validation code for the level file version:
// Handle any global properties
const rapidjson::Value& globals = doc["globalProperties"];
if (globals.IsObject())
{
LoadGlobalProperties(game, globals);
}
You can then add a call to LoadLevel in Game::LoadData, which loads in the Level0.gplevel file:
LevelLoader::LoadLevel(this, "Assets/Level0.gplevel");
Because you’re now loading in the light properties from the level file, you can also remove the code in LoadData that hard-coded the ambient light and directional light.
Loading Actors
Loading in the actors means the JSON file needs an array of actors, and each actor has property information for that actor. However, you need some way to specify which type of Actor you need (because there are subclasses). In addition, you want to avoid having a long set of conditional checks in the level loading code to determine which Actor subclass to allocate.
As before, it helps to first visualize what the data might look like. Listing 14.5 shows one method to specify the actors in the JSON file. This example only shows actors of type TargetActor, but the type can easily specify any other Actor subclass. Note that in addition to the type are any other properties to specify for that actor. Here, the only properties set are position and rotation, but these could conceivably be any property the actor has.
Listing 14.5 Level with Actors (Level1.gplevel)
{
// Version and global properties
// ...
"actors": [
{
"type": "TargetActor",
"properties": {
"position": [1450.0, 0.0, 100.0]
}
},
{
"type": "TargetActor",
"properties": {
"position": [0.0, -1450.0, 200.0],
"rotation": [0.0, 0.0, 0.7071, 0.7071]
}
},
{
"type": "TargetActor",
"properties": {
"position": [0.0, 1450.0, 200.0],
"rotation": [0.0, 0.0, -0.7071, 0.7071]
}
}
]
}
Assuming for a moment that you have a method to construct an actor of a specific type, you also need to be able to load properties for the actor. The simplest approach is to create a virtual LoadProperties function in the base Actor class, shown in Listing 14.6.
Listing 14.6 Actor::LoadProperties Function
void Actor::LoadProperties(const rapidjson::Value& inObj)
{
// Use strings for different states
std::string state;
if (JsonHelper::GetString(inObj, "state", state))
{
if (state == "active")
{
SetState(EActive);
}
else if (state == "paused")
{
SetState(EPaused);
}
else if (state == "dead")
{
SetState(EDead);
}
}
// Load position, rotation, and scale, and compute transform
JsonHelper::GetVector3(inObj, "position", mPosition);
JsonHelper::GetQuaternion(inObj, "rotation", mRotation);
JsonHelper::GetFloat(inObj, "scale", mScale);
ComputeWorldTransform();
}
Then, for some subclass of Actor, you can override the LoadProperties function to load any additional properties, as needed:
void SomeActor::LoadProperties(const rapidjson::Value& inObj)
{
// Load base actor properties
Actor::LoadProperties(inObj);
// Load any of my custom properties
// ...
}
Now that you have a way to load properties, the next step is to solve the issue of constructing an actor of the correct type. One approach is to create a map where the key is the string name of the actor type, and the value is a function that can dynamically allocate an actor of that type. The key is straightforward because it’s just a string. For the value, you can make a static function that dynamically allocates an actor of a specific type. To avoid having to declare a separate function in each subclass of Actor, you can instead create a template function like this in the base Actor class:
template <typename T>
static Actor* Create(class Game* game, const rapidjson::Value& inObj)
{
// Dynamically allocate actor of type T
T* t = new T(game);
// Call LoadProperties on new actor
t->LoadProperties(inObj);
return t;
}
Because it’s templated on a type, it can dynamically allocate an object of the specified type and then call LoadProperties to set any parameters of the actor type, as needed.
Then, back in LevelLoader, you need to create the map. The key type is std::string, but for the value, you need a function that matches the signature of the Actor::Create function. For this, you can once again use the std::function helper class to define the signature.
First, you use an alias declaration (which is like a typedef) to create an ActorFunc type specifier:
using ActorFunc = std::function<
class Actor*(class Game*, const rapidjson::Value&)
>;
The template parameters to std::function specify that the function returns an Actor* and takes in two parameters: Game* and rapidjson::Value&.
Next, you declare the map as a static variable in LevelLoader:
static std::unordered_map<std::string, ActorFunc> sActorFactoryMap;
Then in LevelLoader.cpp, you construct the sActorFactoryMap to fill in the different actors you can create:
std::unordered_map<std::string, ActorFunc> LevelLoader::sActorFactoryMap
{
{ "Actor", &Actor::Create<Actor> },
{ "BallActor", &Actor::Create<BallActor> },
{ "FollowActor", &Actor::Create<FollowActor> },
{ "PlaneActor", &Actor::Create<PlaneActor> },
{ "TargetActor", &Actor::Create<TargetActor> },
};
This initialization syntax sets up entries in the map with a key as the specified string name and the value as the address of an Actor::Create function, templated to create the specific type of Actor subclass. Note that you don’t call the various create functions here. Instead, you just get the memory address of a function and save it for later use.
With the map set up, you can now create a LoadActors function, as in Listing 14.7. Here, you loop over the actors array in the JSON file and get the type string for the actor. You use this type to then look up in sActorFactoryMap. If you find the type, you call the function stored as the value in the map (iter->second), which in turn calls the correct version of Actor::Create. If you don’t find the type, you have a helpful debug log message output.
Listing 14.7 LevelLoader::LoadActors Implementation
void LevelLoader::LoadActors(Game* game, const rapidjson::Value& inArray)
{
// Loop through array of actors
for (rapidjson::SizeType i = 0; i < inArray.Size(); i++)
{
const rapidjson::Value& actorObj = inArray[i];
if (actorObj.IsObject())
{
// Get the type
std::string type;
if (JsonHelper::GetString(actorObj, "type", type))
{
// Is this type in the map?
auto iter = sActorFactoryMap.find(type);
if (iter != sActorFactoryMap.end())
{
// Construct with function stored in map
Actor* actor = iter->second(game, actorObj["properties"]);
}
else
{
SDL_Log("Unknown actor type %s", type.c_str());
}
}
}
}
}
You then add a call to LoadActors inside LoadLevel, immediately after loading in the global properties:
const rapidjson::Value& actors = doc["actors"];
if (actors.IsArray())
{
LoadActors(game, actors);
}
With this code, you’re now loading in actors and setting their properties. However, you are not yet able to adjust properties of components nor add additional components in the level file.
Loading Components
Loading data for components involves many of the same patterns as for actors. However, there is one key difference. Listing 14.8 shows a snippet of the declaration of two different actors with their components property set. The base Actor type does not have any existing components attached to it. So in this case, the MeshComponent type means that you must construct a new MeshComponent for the actor. However, the TargetActor type already has a MeshComponent, as one is created in the constructor for TargetActor. In this case, the properties specified should update the existing component rather than create a new one. This means the code for loading components needs to handle both cases.
Listing 14.8 Actors with Components in JSON (Excerpt from the Full File)
"actors": [
{
"type": "Actor",
"properties": {
"position": [0.0, 0.0, 0.0],
"scale": 5.0
},
"components": [
{
"type": "MeshComponent",
"properties": { "meshFile": "Assets/Sphere.gpmesh" }
}
]
},
{
"type": "TargetActor",
"properties": { "position": [1450.0, 0.0, 100.0] },
"components": [
{
"type": "MeshComponent",
"properties": { "meshFile": "Assets/Sphere.gpmesh" }
}
]
}
]
To determine whether Actor already has a component of a specific type, you need a way to search through an actor’s component vector by type. While you might be able to use the built-in type information in C++, it’s more common for game programmers to use their own type information (and disable the built-in functionality). This is mainly because of the well-documented downsides of the built-in C++ Run-type type information (RTTI) not obeying the “you only pay for what you use” rule.
There are many ways to implement your own type information; this chapter shows a simple approach. First, you declare a TypeID enum in the Component class, like so:
enum TypeID
{
TComponent = 0,
TAudioComponent,
TBallMove,
// Other types omitted
// ...
NUM_COMPONENT_TYPES
};
Then, you add a virtual function called GetType that simply returns the correct TypeID based on the component. For example, the implementation of MeshComponent::GetType is as follows:
TypeID GetType() const override { return TMeshComponent; }
Next, you add a GetComponentOfType function to Actor that loops through the mComponents vector and returns the first component that matches the type:
Component* GetComponentOfType(Component::TypeID type)
{
Component* comp = nullptr;
for (Component* c : mComponents)
{
if (c->GetType() == type)
{
comp = c;
break;
}
}
return comp;
}
The disadvantage of this approach is that every time you create a new Component subclass, you must remember to add an entry to the TypeID enum and implement the GetType function. You could automate this somewhat by using macros or templates, but the code here does not do so for the sake of readability and understanding.
Note that this system also assumes that you won’t have multiple components of the same type attached to one actor. If you wanted to have multiple components of the same type, then GetComponentOfType would potentially have to return a collection of components rather than just a single pointer.
Also, the type information does not give inheritance information; you can’t figure out SkeletalMeshComponent is a subclass of MeshComponent, as GetType for SkeletalMeshComponent just returns TSkeletalMeshComponent. To support inheritance information, you would need an approach that saves some hierarchy information as well.
With the basic type system in place, you can move on to more familiar steps. As with Actor, you need to create a virtual LoadProperties function in the base Component class and then override it for any subclasses, as needed. The implementations in the various subclasses are not necessarily straightforward. Listing 14.9 shows the implementation of LoadProperties for MeshComponent. Recall that MeshComponent has an mMesh member variable that’s a pointer to the vertex data to draw. You don’t want to specify the vertex directly in the JSON file; instead, you want to reference the gpmesh file. The code first checks for the meshFile property and then gets the corresponding mesh from the renderer.
Listing 14.9 MeshComponent::LoadProperties Implementation
void MeshComponent::LoadProperties(const rapidjson::Value& inObj)
{
Component::LoadProperties(inObj);
std::string meshFile;
if (JsonHelper::GetString(inObj, "meshFile", meshFile))
{
SetMesh(mOwner->GetGame()->GetRenderer()->GetMesh(meshFile));
}
int idx;
if (JsonHelper::GetInt(inObj, "textureIndex", idx))
{
mTextureIndex = static_cast<size_t>(idx);
}
JsonHelper::GetBool(inObj, "visible", mVisible);
JsonHelper::GetBool(inObj, "isSkeletal", mIsSkeletal);
}
The next step is to add a static templated Create function for Component, which is very similar to the one in Actor except that the parameters are different. (It takes in Actor* as the first parameter instead of Game*.)
You then need a map in LevelLoader. You use std::function again to create a helper type called ComponentFunc:
using ComponentFunc = std::function<
class Component*(class Actor*, const rapidjson::Value&)
>;
Then, you declare the map. However, unlike with the sActorFactoryMap, which has only a single value, in this case, you need a pair of values. The first element in the pair is an integer corresponding to the TypeID of the component, and the second element is the ComponentFunc:
static std::unordered_map<std::string,
std::pair<int, ComponentFunc>> sComponentFactoryMap;
Then, in LevelLoader.cpp, you instantiate the sComponentFactoryMap:
std::unordered_map<std::string, std::pair<int, ComponentFunc>>
LevelLoader::sComponentFactoryMap
{
{ "AudioComponent",
{ Component::TAudioComponent, &Component::Create<AudioComponent>}
},
{ "BallMove",
{ Component::TBallMove, &Component::Create<BallMove> }
},
// Other components omitted
// ...
};
You then implement a LoadComponents helper function in LevelLoader, as shown in Listing 14.10. As with LoadActors, it takes in an array of the components to load and loops through this array. You then use the sComponentFactoryMap to find the component type. If it is found, you then check if the actor already has a component of the type. The iter-> second.first accesses the first element of the value pair, which corresponds to the type ID. If the actor doesn’t already have a component of the requested type, then you create one by using the function stored in the second element of the value pair (iter->second.second). If the component already exists, you can then just directly call LoadProperties on it.
Listing 14.10 LevelLoader::LoadComponents Implementation
void LevelLoader::LoadComponents(Actor* actor,
const rapidjson::Value& inArray)
{
// Loop through array of components
for (rapidjson::SizeType i = 0; i < inArray.Size(); i++)
{
const rapidjson::Value& compObj = inArray[i];
if (compObj.IsObject())
{
// Get the type
std::string type;
if (JsonHelper::GetString(compObj, "type", type))
{
auto iter = sComponentFactoryMap.find(type);
if (iter != sComponentFactoryMap.end())
{
// Get the typeid of component
Component::TypeID tid = static_cast<Component::TypeID>
(iter->second.first);
// Does the actor already have a component of this type?
Component* comp = actor->GetComponentOfType(tid);
if (comp == nullptr)
{
// It's a new component, call function from map
comp = iter->second.second(actor, compObj["properties"]);
}
else
{
// It already exists, just load properties
comp->LoadProperties(compObj["properties"]);
}
}
else
{
SDL_Log("Unknown component type %s", type.c_str());
}
}
}
}
}
Finally, you add code in LoadActors that accesses the components property, if it exists, and calls LoadComponents on it:
// Construct with function stored in map
Actor* actor = iter->second(game, actorObj["properties"]);
// Get the actor's components
if (actorObj.HasMember("components"))
{
const rapidjson::Value& components = actorObj["components"];
if (components.IsArray())
{
LoadComponents(actor, components);
}
}
With all this code in place, you can now load the entire level from a file, including the global properties, actors, and any components associated with each actor.
Saving Level Files
Saving to a level file is conceptually simpler than loading from a file. First, you write the global properties for the level. Then, you loop through every actor in the game and every component attached to every actor. For each of these, you need to write out the relevant properties.
The implementation details are bit involved because the RapidJSON interface is slightly more complicated for creating JSON files than for reading in files. However, overall you can use techniques like those used for loading the level file.
First, you create helper Add functions in JsonHelper so that you can quickly add additional properties to an existing JSON object. For example, the AddInt function has the following syntax:
void JsonHelper::AddInt(rapidjson::Document::AllocatorType& alloc,
rapidjson::Value& inObject, const char* name, int value)
{
rapidjson::Value v(value);
inObject.AddMember(rapidjson::StringRef(name), v, alloc);
}
The last three parameters are identical to the parameters of the GetInt function, except the Value is now not const. The first parameter is an allocator that RapidJSON uses when needing to allocate memory. Every call to AddMember requires an allocator, so you must pass one in. You can get the default allocator just from a Document object, but you could conceivably use a different allocator if desired. You then create a Value object to encapsulate the integer and use the AddMember function to add a value with the specified name to inObject.
The rest of the Add functions are similar, except for AddVector3 and AddQuaternion, for which you must first create an array and then add float values to that array. (You’ll see this array syntax when looking at the global properties.)
You then create a skeleton for the LevelLoader::SaveLevel function, as shown in Listing 14.11. First, you create the RapidJSON document and make an object for its root via SetObject. Next, you add the version integer. Then, you use the StringBuffer and PrettyWriter to create a pretty-printed output string of the JSON file. Finally, you use a standard std::ofstream to write out the string to a file.
Listing 14.11 LevelLoader::SaveLevel Implementation
void LevelLoader::SaveLevel(Game* game,
const std::string& fileName)
{
// Create the document and root object
rapidjson::Document doc;
doc.SetObject();
// Write the version
JsonHelper::AddInt(doc.GetAllocator(), doc, "version", LevelVersion);
// Create the rest of the file (TODO)
// ...
// Save JSON to string buffer
rapidjson::StringBuffer buffer;
// Use PrettyWriter for pretty output (otherwise use Writer)
rapidjson::PrettyWriter<rapidjson::StringBuffer> writer(buffer);
doc.Accept(writer);
const char* output = buffer.GetString();
// Write output to file
std::ofstream outFile(fileName);
if (outFile.is_open())
{
outFile << output;
}
}
For now, this function only writes out the version to the output file. But with this skeleton code, you can start adding the remaining output.
Saving Global Properties
Next, you need to add a SaveGlobalProperties function to LevelLoader. We omit the implementation here, as it’s very similar to the other functions written thus far. You simply need to add the properties for the ambient light and the directional light object.
Once this function is complete, you integrate it into your SaveLevel function as follows:
rapidjson::Value globals(rapidjson::kObjectType);
SaveGlobalProperties(doc.GetAllocator(), game, globals);
doc.AddMember("globalProperties", globals, doc.GetAllocator());
Saving Actors and Components
To be able to save actors and components, you need a way to get a string name of the type, given an Actor or Component pointer. You already have a TypeID for components, so to get a corresponding string, you need only declare a constant array of the different names in Component. You declare this array in Component.h as follows:
static const char* TypeNames[NUM_COMPONENT_TYPES];
And then in Component.cpp, you fill in the array. It’s important that you maintain the same ordering as the TypeID enum:
const char* Component::TypeNames[NUM_COMPONENT_TYPES] = {
"Component",
"AudioComponent",
"BallMove",
// Rest omitted
// ...
};
By maintaining the ordering, you make it easy to get the name of a component, given the type, using a snippet like this:
Component* comp = /* points to something */;
const char* name = Component::TypeNames[comp->GetType()];
To do the same thing for the Actor and its subclasses, you need to add a TypeID enum to Actor as well. This is essentially the same as the code for TypeIDs in components earlier in this chapter, so we omit it here.
You then need to create a virtual SaveProperties function in both Actor and Component and then override it in every subclass that needs to do so. This ends up playing out very similarly to the LoadProperties functions written when loading in the level files. As an example, Listing 14.12 shows the implementation of Actor::SaveProperties. Note that you liberally use the Add functions in LevelLoader, and you need to pass in the allocator because all the Add functions need it.
Listing 14.12 Actor::SaveProperties Implementation
void Actor::SaveProperties(rapidjson::Document::AllocatorType& alloc,
rapidjson::Value& inObj) const
{
std::string state = "active";
if (mState == EPaused)
{
state = "paused";
}
else if (mState == EDead)
{
state = "dead";
}
JsonHelper::AddString(alloc, inObj, "state", state);
JsonHelper::AddVector3(alloc, inObj, "position", mPosition);
JsonHelper::AddQuaternion(alloc, inObj, "rotation", mRotation);
JsonHelper::AddFloat(alloc, inObj, "scale", mScale);
}
With all these pieces in place, you can then add SaveActors and SaveComponents functions to LevelLoader. Listing 14.13 shows the SaveActors function. First, you get the vector of actors from the game by const reference. Then, you loop through every actor and create a new JSON object for it. You then add the string for the type by using the TypeID and TypeNames functionality. Next, you create a JSON object for the properties and call the actor’s SaveProperties function. You then create an array for the components before calling SaveComponents. Finally, you add the actor’s JSON object into the JSON array of actors.
Listing 14.13 LevelLoader::SaveActors Implementation
void LevelLoader::SaveActors(rapidjson::Document::AllocatorType& alloc,
Game* game, rapidjson::Value& inArray)
{
const auto& actors = game->GetActors();
for (const Actor* actor : actors)
{
// Make a JSON object
rapidjson::Value obj(rapidjson::kObjectType);
// Add type
AddString(alloc, obj, "type", Actor::TypeNames[actor->GetType()]);
// Make object for properties
rapidjson::Value props(rapidjson::kObjectType);
// Save properties
actor->SaveProperties(alloc, props);
// Add the properties to the JSON object
obj.AddMember("properties", props, alloc);
// Save components
rapidjson::Value components(rapidjson::kArrayType);
SaveComponents(alloc, actor, components);
obj.AddMember("components", components, alloc);
// Add actor to inArray
inArray.PushBack(obj, alloc);
}
}
You similarly implement a SaveComponents function. With all this code implemented, you can now save all the actors and components to the file. For testing purposes, pressing the R key in this chapter’s game project saves to the Assets/Save.gplevel level file.
note
With some work, you could create a single serialize function that both load and saves properties. This way, you could avoid having to update two different functions every time you add a new property to an actor or a component.
While this code will save almost everything in the game, it doesn’t quite fully capture the current state of the game at a specific point in time. For example, it does not save the state of any active FMOD sound events. To implement this, you would need to ask FMOD for the current timestamp of the sound events, and then when loading the game from the file, you would need to restart the sound events with those timestamps. It takes some additional work to go from saving a level file to being usable as a save file for the player.
Binary Data
You’ve used JSON file formats throughout this book: for meshes, animations, skeletons, text localization, and now for level loading. The advantages of using a text-based file format are numerous. Text files are easy for humans to look at, find errors in, and (if needed) manually edit. Text files also play very nicely with source control systems such as Git because it’s very easy to see what changed in a file between two revisions. During development, it’s also easier to debug loading of assets if they are text files.
However, the disadvantage of using text-based file formats is that they are inefficient, both in terms of disk and memory usage as well as in terms of performance at runtime. Formats such as JSON or XML take up a lot of space on disk simply because of the formatting characters they use, such as braces and quotation marks. On top of this, parsing text-based files at runtime is slow, even with high-performance libraries such as RapidJSON. For example, on my computer, it takes about three seconds to load in the CatWarrior.gpmesh file in a debug build. Clearly, this would lead to slow load times for a larger game.
For the best of both worlds, you may want to use text files during development (at least for some members of the team) and then binary files in optimized builds. This section explores how to create a binary mesh file format. To keep things simple, in the code that loads in the gpmesh JSON format, you will first check if a corresponding gpmesh.bin file exists. If it does, you’ll load that in instead of the JSON file. If it doesn’t exist, the game will create the binary version file so that next time you run the game, you can load the binary version instead of the text version.
Note that one potential downside of this approach is that it may lead to bugs that occur only with the binary format but not the text one. To avoid this, it’s important that you continue to use both formats throughout development. If one of the two formats becomes stale, then there’s a greater chance that format will stop working.
Saving a Binary Mesh File
With any binary file format, an important step is to decide on a layout for the file. Most binary files begin with some sort of header that defines the contents of the file as well as any specific size information that’s needed to read in the rest of the file. In the case of a mesh file format, you want the header to store information about the version, the number of vertices and indices, and so on. Listing 14.14 shows the MeshBinHeader struct that defines the layout of the header. In this example, the header is not packed (reduced in size as much as possible), but it gives the general idea of what you might want to store in a header.
Listing 14.14 MeshBinHeader Struct
struct MeshBinHeader
{
// Signature for file type
char mSignature[4] = { 'G', 'M', 'S', 'H' };
// Version
uint32_t mVersion = BinaryVersion;
// Vertex layout type
VertexArray::Layout mLayout = VertexArray::PosNormTex;
// Info about how many of each you have
uint32_t mNumTextures = 0;
uint32_t mNumVerts = 0;
uint32_t mNumIndices = 0;
// Box/radius of mesh, used for collision
AABB mBox{ Vector3::Zero, Vector3::Zero };
float mRadius = 0.0f;
};
The mSignature field is a special 4-byte magic number that specifies the file type. Most popular binary file types have some sort of signature. The signature helps you figure out what a file type is from its first few bytes without knowing anything other than the signature to look for. The rest of the data is information you need to reconstruct the mesh data from the file.
After the header is the main data section of the file. In this case, there are three main things to store: the filenames for associated textures, the vertex buffer data, and the index buffer data.
With the file format decided on, you can then create the SaveBinary function, as shown in Listing 14.15. This function takes in a lot of parameters because there’s a lot of information needed to create the binary file. In total, you need the filename, a pointer to the vertex buffer, the number of vertices, the layout of these vertices, a pointer to the index buffer, the number of indices, a vector of the texture names, the bounding box of the mesh, and the radius of the mesh. With all these parameters, you can save the file.
Listing 14.15 Mesh::SaveBinary Implementation
void Mesh::SaveBinary(const std::string& fileName, const void* verts,
uint32_t numVerts, VertexArray::Layout,
const uint32_t* indices, uint32_t numIndices,
const std::vector<std::string>& textureNames,
const AABB& box, float radius)
{
// Create header struct
MeshBinHeader header;
header.mLayout = layout;
header.mNumTextures =
static_cast<unsigned>(textureNames.size());
header.mNumVerts = numVerts;
header.mNumIndices = numIndices;
header.mBox = box;
header.mRadius = radius;
// Open binary file for writing
std::ofstream outFile(fileName, std::ios::out
| std::ios::binary);
if (outFile.is_open())
{
// Write the header
outFile.write(reinterpret_cast<char*>(&header), sizeof(header));
// For each texture, we need to write the size of the name,
// followed by the string, followed by a null terminator
for (const auto& tex : textureNames)
{
uint16_t nameSize = static_cast<uint16_t>(tex.length()) + 1;
outFile.write(reinterpret_cast<char*>(&nameSize),
sizeof(nameSize));
outFile.write(tex.c_str(), nameSize - 1);
outFile.write(" ", 1);
}
// Figure out number of bytes for each vertex, based on layout
unsigned vertexSize = VertexArray::GetVertexSize(layout);
// Write vertices
outFile.write(reinterpret_cast<const char*>(verts),
numVerts * vertexSize);
// Write indices
outFile.write(reinterpret_cast<const char*>(indices),
numIndices * sizeof(uint32_t));
}
}
The code in Listing 14.15 does quite a lot. First, you create an instance of the MeshBinHeader struct and fill in all its members. Next, you create a file for output and open it in binary mode. If this file successfully opens, you can write to it.
Then you write the header of the file with the write function call. The first parameter write expects is a char pointer, so in many cases it’s necessary to cast a different pointer to a char*. This requires a reinterpret_cast because a MeshBinHeader* cannot directly convert to a char*. The second parameter to write is the number of bytes to write to the file. Here, you use sizeof to specify the number of bytes corresponding to the size of MeshBinHeader. In other words, you are writing sizeof(header) bytes starting at the address of header. This is a quick way to just write the entire struct in one fell swoop.
warning
WATCH OUT FOR ENDIANNESS: The order in which a CPU platform saves values larger than 1 byte is called endianness. The method used here to read and write MeshBinHeader will not work if the endianness of the platform that writes out the gpmesh.bin file is different from the endianness of the platform that reads the gpmesh.bin file.
Although most platforms today are little endian, endianness can still be a potential issue with code of this style.
Next, you loop through all the texture names and write each of them to the file. For each filename, you first write the number of characters in the filename (plus one for the null terminator) and then write the string itself. Note that this code assumes that a filename can’t be larger than 64 KB, which should be a safe assumption. The reason you write the number of characters and the name is for loading. The header only stores the number of textures and not the size of each string. Without storing the number of characters, at load time you would have no way of knowing how many bytes to read for the filename.
After writing all the filenames, you then write all the vertex and index buffer data directly to the file. You don’t need to include the sizes here because they already appear in the header. For the vertex data, the number of bytes is the number of vertices times the size of each vertex. Luckily, you can use a VertexArray helper function to get the size of each vertex based on layout. For the index data, you have a fixed size (32-bit indices), so the total number of bytes is easier to calculate.
Then in Mesh::Load, if the binary file doesn’t exist, the code loads the JSON file and creates the corresponding binary file.
Loading a Binary Mesh File
Loading a binary mesh file is like writing to it but in reverse. The steps are to load in the header, check the validity of the header, load in the textures, load in the vertex and index data, and finally create the actual VertexArray (which will upload the data to the GPU via OpenGL). Listing 14.16 shows the outline of the code for Mesh::LoadBinary.
Listing 14.16 Mesh::LoadBinary Outline
void Mesh::LoadBinary(const std::string& filename,
Renderer* renderer)
{
std::ifstream inFile(fileName, /* in/binary flags ... */);
if (inFile.is_open())
{
MeshBinHeader header;
inFile.read(reinterpret_cast<char*>(&header), sizeof(header));
// Validate the header signature and version
char* sig = header.mSignature;
if (sig[0] != 'G' || sig[1] != 'M' || sig[2] != 'S' ||
sig[3] != 'H' || header.mVersion != BinaryVersion)
{
return false;
}
// Read in the texture file names (omitted)
// ...
// Read in vertices/indices
unsigned vertexSize = VertexArray::GetVertexSize(header.mLayout);
char* verts = new char[header.mNumVerts * vertexSize];
uint32_t* indices = new uint32_t[header.mNumIndices];
inFile.read(verts, header.mNumVerts * vertexSize);
inFile.read(reinterpret_cast<char*>(indices),
header.mNumIndices * sizeof(uint32_t));
// Now create the vertex array
mVertexArray = new VertexArray(verts, header.mNumVerts,
header.mLayout, indices, header.mNumIndices);
// Delete verts/indices
delete[] verts;
delete[] indices;
mBox = header.mBox;
mRadius = header.mRadius;
return true;
}
return false;
}
First, you open the file for reading in binary mode. Next, you read in the header via the read function. Much as with write, read takes in a char* for where to write and the number of bytes to read from the file. Next, you verify that the signature and version in the header match what is expected; if they don’t, you can’t load the file.
After this, you read in all the texture filenames and load them, though we omit that code from Listing 14.16 to save space. Next, you allocate memory to store the vertex and index buffers, and you use read to grab the data from the file. Once you have the vertex and index data, you can construct the VertexArray object and pass in all the information it needs. You need to make sure to clean up the memory and set the mBox and mRadius members before returning.
Note that LoadBinary returns false if the file fails to load. This way, the Mesh::Load code first tries to load the binary file. If it succeeds, that’s it. Otherwise, it can proceed using the JSON parsing code from before:
bool Mesh::Load(const std::string& fileName, Renderer* renderer)
{
mFileName = fileName;
// Try loading the binary file first
if (LoadBinary(fileName + ".bin", renderer))
{
return true;
}
// ...
With the switch to binary mesh file loading, the performance improves significantly in debug mode. The CatWarrior.gpmesh.bin file now loads in one second as opposed to three—meaning a 3x performance gain over the JSON version! This is great because you’ll spend most of your development time running in debug mode.
Unfortunately, in an optimized build, the performance of both the JSON and binary path is almost identical. This could be due to several factors, including the RapidJSON library being very optimized or other aspects being the primary overhead, such as transferring the data to the GPU or loading in the textures.
On the disk space side of things, you save space. While the JSON version of the Feline Swordsman is around 6.5 MB on disk, the binary version is only 2.5 MB.
Game Project
This chapter’s game project implements the systems discussed in this chapter. Everything loads from a gplevel file, and pressing the R key saves the current state of the world into Assets/Saved.gplevel. The project also implements the binary saving and loading of mesh files in the .gpmesh.bin format. The code is available in the book’s GitHub repository, in the Chapter14 directory. Open Chapter14-windows.sln in Windows and Chapter14-mac.xcodeproj on Mac.
Figure 14.1 shows the game project in action. Notice that it looks identical to the game project from Chapter 13, “Intermediate Graphics.” However, the entire contents of the game world now load directly from the Assets/Level3.gplevel file, which was in turn created by saving the level file. The first time the game runs, it creates a binary mesh file for every mesh loaded. Subsequent runs load meshes from the binary files instead of JSON.
Figure 14.1 Chapter 14 game project
Summary
This chapter explores how to create level files in JSON. Loading from a file requires several systems. First, you create helper functions that wrap the functionality of the RapidJSON library to easily be able to write the game’s types to JSON. You then add code to set global properties, load in actors, and load in components associated with the actors. To do this, you need to add some type information to components, as well as maps that associate names of types to a function that can dynamically allocate that type. You also need to create virtual LoadProperties functions in both Component and Actor.
You also need to create code to save the game world to JSON, and you create helper functions to assist with this process. At a high level, saving the file requires saving all the global properties first and then looping through all the actors and components to write their properties. As with file loading, you have to create virtual SaveProperties functions in both Component and Actor.
Finally, this chapter discusses the trade-offs involved in using a text-based file format instead of a binary one. While a text format is often more convenient to use in development, it comes at a cost of inefficiency—both in performance and disk usage. This chapter explores how to design a binary file format for mesh files, which involves writing and reading from files in binary mode.
Additional Reading
There are no books devoted specifically to level files or binary data. However, the classic Game Programming Gems series has some articles on the topic. Bruno Sousa’s article discusses how to use resource files, which are files that combine several files into one. Martin Brownlow’s article discusses how to create a save-anywhere system. Finally, David Koenig’s article looks at how to improve the performance of loading files.
Brownlow, Martin. “Save Me Now!” Game Programming Gems 3. Ed. Dante Treglia. Hingham: Charles River Media, 2002.
Koenig, David L. “Faster File Loading with Access Based File Reordering.” Game Programming Gems 6. Ed. Mike Dickheiser. Rockland: Charles River Media, 2006.
Sousa, Bruno. “File Management Using Resource Files.” Game Programming Gems 2. Ed. Mark DeLoura. Hingham: Charles River Media, 2001.
Exercises
In this chapter’s first exercise, you need to reduce the size of the JSON files created by SaveLevel. In the second exercise you convert the Animation file format to binary.
Exercise 14.1
One issue with the SaveLevel code is that you write every property for every actor and all its components. However, for a specific subclass like TargetActor, few if any of the properties or components change after construction.
To solve this problem, when it’s time to save the level, you can create a temporary TargetActor and write out the JSON object for that actor by using the normal writing techniques. This JSON object serves as the template, as it’s the state of TargetActor when it’s originally spawned. Then, for each TargetActor to save in the level, compare its JSON object to the template one and write only the properties and components that are different.
You can then use this process for all the different types of actors. To assist with this, RapidJSON provides overloaded comparison operators. Two rapidjson::Values are equal only if they have the same type and contents. This way, you can eliminate setting at least most of the components (because they won’t change). It will require a bit more work to do this on a granular (per-property) level.
Exercise 14.2
Applying the same binary file techniques you used for the mesh files, create a binary file format for the animation files. Because all the tracks of bone transforms are the same size, you can use a format where, after writing the header, you write the ID for each track followed by the entire track information. For a refresher on the animation file format, refer to Chapter 12, “Skeletal Animation.”