348 20.PointerPatchingAssets
asset. This can lead to a powerful, optimized tools pipeline with minimal
load times and great flexibility.
■ Symbolic pointers with delayed bindings. Rather than all pointers having
physical memory addresses, some studios use named addresses that can be
patched at run time after loading is complete. This way you can have pointers
that point to the player’s object or some other level-specific data without
needing to write custom support for each case.
■ Generic run-time asset caches. Once loading is handled largely through
pointer-patched blocks with named assets, it is fairly simple to build a
generic asset caching system on top of this, even allowing dynamic reloading
of assets at run time with minimal effort.
■ Simple tools interface that handles byte swapping and recursion. Writing out
data should be painless and natural, allowing for recursive traversals of live
data structures and minimal intrusion.
■ Special pointer patching consideration for virtual tables. A method for
virtual table patching may be necessary to refresh class instances that have
virtual functions. Or choose ways to represent your data without using virtual
functions.
■ Offline introspection tools. Not only is it very useful for debugging the asset
pipeline, but a generic set of introspection tools can help perform vital
analyses about the game’s memory consumption based on asset type,
globally, and without even loading the game on the final platform!
■ Propagate memory alignment requirements. Careful attention to data
alignment allows hardware to receive data in the fastest possible way. Design
your writing interface and linking tools to preserve and propagate alignments
so that all the thinking is done in tools, even if it means inserting some
wasted space to keep structures starting on the right address. All the run-time
code should need to know is the alignment of the block as a whole.
While all the above properties have their merits, a complete and full-featured
system is an investment for studios to undertake on their own. A very basic
system is provided on the website. It supports byte swapping of atomic types,
pointer patching, and a clean and simple interface for recursion.
20.3ABriefExample
Here is a small example of a simple tree structure. Trees are traditionally very
slow because they require many memory allocations and are somewhat
challenging to serialize due to the amount of pointer work required to reconstruct
20.3ABriefExample 349
them at run time. While this example only contains a few nodes, it is nonetheless
a nontrivial example given the comparative complexity of any reasonable and
flexible alternative.
ToolsSide
The first order of business is to declare the data structure we want in the run-time
code. Note that there are no requirements placed on the declarations—minimal
intrusion makes for easier adoption. You should be able to use the actual run-
time structure declaration in those cases where there are limited external
dependencies.
Listing 20.1 shows a simple example of how to dump out a live tree data
structure into a pointer-patched block in just a few lines of code. As you can see,
the
WriteTree() function just iterates over the entire structure—any order is
actually fine—and submits the contents of each node to the writer object. Each
call to
ttw.Write*() is copying some data from the Node into the writer’s
memory layout, which matches exactly what the run-time code will use. As
written,
WriteTree() simply starts writing a tree recursively down both the left
and right branches until it exhausts the data. The writer interface is designed to
handle data in random order and has an explicit finalization stage where
addresses of structures are hooked to pointers that were written out. This
dramatically improves the flexibility of the tools code.
struct Node
{
Node(float v) : mLeft(NULL), mRight(NULL), mValue(v) {}
Node *mLeft;
Node *mRight;
float mValue;
};
void WriteTree(const Node *n, ToolsTimeWriter &ttw)
{
ttw.StartStruct(n);
ttw.WritePtr();
if (n->mLeft)
WriteTree(n->mLeft, ttw);
ttw.WritePtr();
350 20.PointerPatchingAssets
if (n->mRight)
WriteTree(n->mRight, ttw);
ttw.Write4();
ttw.EndStruct();
}
// First, we construct a handful of nodes into a tree.
Node *root = new Node(3.14F);
root->mLeft = new Node(5.0F);
root->mRight = new Node(777.0F);
root->mLeft->mRight = new Node(1.0F);
root->mLeft->mRight->mLeft = new Node(0.01F);
ToolsTimeWriter ttw(false);
ttw.StartAsset("TreeOfNodes", root);
WriteTree(root, ttw);
std::vector<unsigned char> packedData = ttw.Finalize();
Listing 20.1. This is the structure declaration for the sample tree.
Figure 20.2. This is the in-memory layout of our sample tree. Notice it requires five
memory allocations and various pointer traversals to configure.
3.14
mLeft mRight mValue
5.0
mLeft mRight mValue
777.0
mLeft mRight mValue
1.0
mLeft mRight mValue
0.01
mLeft mRight mValue
NULL NULL NULL
NULL
NULL NULL
20.3ABriefExample 351
Figure 20.3. This is the finalized data from the writer, minus the header and format
details. Notice that all offsets are relative to the pointer’s address, rather than an absolute
index. This vastly simplifies bookkeeping when merging multiple blocks together.
Next, there is a block of code that creates a live tree structure using typical
allocation methods you would use in tools code, graphically depicted in
Figure 20.2. Finally, we write out the live tree structure and finalize it down to a
single block of opaque data. This is done by declaring a writer object, marking
the start of the specific structure that the run-time code will later find by name,
calling our data writing function above, and then retrieving the baked-down
block of bytes that should be written to disk. Amazingly, Figure 20.3 shows the
entire structure in just a handful of bytes.
The
ToolsTimeWriter class shown in Listing 20.2 is only a toy
implementation. Some basic limitations in this implementation are that it only
supports 32-bit pointers, doesn’t handle alignment requirements per structure,
etc. Still, it is educational to see the approach taken by this one of many possible
interfaces.
class ToolsTimeWriter
{
public:
ToolsTimeWriter(bool byteSwap);
// Add an entry to the asset table in the header.
void StartAsset(char const *name, void *s);
// Starting a struct allows for a recursion context stack.
void StartStruct(void *s);
00 04 08 0C 10 14 18 1C 20 24 28 2C 30 34 38
000C 002C 3.14 0000 0008 5.0 000C 0000 1.0 0000 0000 0.01 0000 0000 777.0
root root->
mLeft
root->
mRight
root->
mLeft->
mRight
root->
mLeft->
mRight->
mLeft
Addr
Data
Node
352 20.PointerPatchingAssets
// Once a struct has been started, you call these to pump out
// data. These are needed to handle byte swapping and to
// measure struct sizes.
void Write1(void);
void Write2(void);
void Write4(void);
void Write8(void);
void WritePtr(void);
void WriteRaw(int numBytes);
// This pops the recursion context off the stack.
void EndStruct(void);
std::vector<unsigned char> Finalize(void);
};
Listing 20.2. This basic interface is close to the minimum requirements for a pointer patching
asset interface.
TheRun‐TimePointerPatchingProcess
The primary feature of a pointer patching system is that almost all of the work is
done once offline, which means there isn’t a great deal to discuss about the run-
time code. The basic set of operations is to load a coherent chunk of data into
memory, then fix up all of the pointers once.
An exceedingly clever implementation might segregate all the ancillary data,
such as the pointer table, the asset table, and the format header, into a metadata
block that can be thrown away once the pointers are patched and the assets are
registered with their respective caches. However, with the metadata gone, this
data can no longer be easily relocated in memory. The metadata is generally quite
small, so we recommend keeping it. An example file format can be seen in
Figure 20.4.
Listings 20.3 and 20.4 show how incredibly simple it is to set up pointer-
patched blocks at run time. By adding more metadata, you can set up the loader
so that it dispatches types to asset handlers in the engine rather than making your
code search for each asset one by one. Then the job of figuring out what content
should be loaded can be completely handled in tools.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.