Building the WARoS Match Engine

The WARoS match engine will be responsible for the following:

• Provide a host runtime for each WebAssembly module
• Provide a game loop for moving the match forward
• Manage game state
• Complete a match after a given number of cycles

Building a game engine is no small feat. The goal of what we’re building in this chapter isn’t to produce the best game engine. It’s to illustrate how we can host multiple WebAssembly modules simultaneously within a virtual environment and allow their logic to safely interact and manipulate shared state. In the following sections, you’ll see how to accomplish all of this with Rust.

Threading, Time Slicing, and the Game Loop

Building games and game engines is loads of fun for certain types of people, like myself, who have often been referred to as masochists due to the depth and complexity of some of the problems that need to be solved in this domain. One of the problems we encounter when building game engines, the game loop, also shows up in a surprising number of other types of software, including big enterprise apps.

In Poindexter’s original implementation, he created his own virtual CPU. This meant that for each robot loaded into memory, he could control the allocation of time. Specifically, he could dole out processing cycles to each robot in turn. This allowed the robot code to be written in a way that felt real-time. The robots all operate within while loops that control their behavior throughout the match.

With WebAssembly, we don’t have that luxury. We aren’t in control of the time-slicing mechanism inside the module, and the WebAssembly standard doesn’t even have any formal definition for internal threading or atomic locking (at the time this book was written). I really wanted to maintain the “robot loop” feel of the original robot code, and so this presented a huge problem—how do I let each WebAssembly module run its own tight, infinite loop without blocking everything, gumming up the works, or worse, running afoul of Rust’s ever-vigilant borrow checker?

As I looked for solutions, I discovered that this type of problem of time sharing among multiple threads shows up in a lot of places. If you spend most of your time working in high-level web or data access frameworks, you’re probably insulated from the solutions to this concurrency problem, but they’re there if you want to look.

My initial solution was to spin up a single Rust thread for each WebAssembly module, in which the module would be allowed to run its infinite loop (triggered by the bot_init exported function). Then, I’d run a traditional game loop^[44] that ran its own tight loop. Each iteration of my game loop would deal with the state changes and interactions coming from each robot, synchronizing and managing it all. The following diagram shows what this architecture looks like:

On the surface, it looks like this will do the job, but the devil is in the details, as they say. All we need is some kind of mutex that lets us block out other threads long enough for us to make our changes. Seems simple enough, right? Not quite. My first implementation of this literally ground to a halt. Even after Rust’s compiler blessed my code, my core game engine just continued to die in spectacular fashion, over and over again. In the next section, I’ll explain why.

Entities, Components, and Systems

If you’re not familiar with the concept of a game loop, it’s fairly simple. This is a loop that’s always running from the beginning of the match until the end. During each iteration of the loop, it takes care of maintenance of state and other actions.

The game loop in most games will do things like collision detection, inflicting damage, casting spells, depleting fuel from spaceships—all the housekeeping to keep all the individual parts of the game up to date. When I started working on this, I created a single struct called Robot that I figured could hold all that state that I need managed by the game loop. It looked something like this:

I took most of those fields from the original robot struct in Crobots.^[45] If my game loop was the only thing that needed to manipulate that list of robots and their data, that would probably be fine. But the robots all need query access for their own data, and they have functions as part of the API that write values like desired heading and speed, and they perform actions like firing cannons.

This is where things went horribly wrong. Each iteration of my game loop would acquire a write lock on this list of robots because you can’t acquire a write lock on just one element of a vector. It would then do all of the normal things and move on. Each thread from a running WebAssembly module would then use a read lock on the entire vector and occasionally try and grab a write lock to modify a robot’s data.

In this situation, the first thread that managed to get a write lock would ring the death bell for the rest of the app. If that thread was the game loop, it would only let go of it for such a short period of time (basically between the end and start of the loop block) and the loop would appear stalled, or the robots would stall. In some variants of this plan, everything stopped, and I felt like I had bitten off more than I could chew. Threading is hard. Shared mutable state is super hard, especially in Rust where you can’t cheat. You can just “hope” you never have a data race.

The solution was to separate out the read concerns from the write concerns, and to narrow the focus of the write concerns down to as small a piece of state as possible. This is where the concept of Entity—Component—System (ECS)^[46] comes in.

First and foremost, this pattern emphasizes composition over inheritance. Since we don’t have classes with inheritance in Rust, this works out nicely. The real goal of ECS architecture is to separate concerns into small tiny pieces. Think of it as applying the microservices approach to a single piece of shared state. In my use of the ECS pattern in the game engine, I diverge from the core definition a little bit to keep the focus on the book and not on game design, but my heart is in the right place.

Entity

Entities are just arbitrary things. In most ECS implementations, an entity is just some form of unique identifier and that’s all. In our case, the entity will be the WebAssembly module name, which doubles as the player name.

Component

A component is raw state for some aspect of an entity. A component labels an entity as “having a particular aspect.” In our case, entities will have motion, damage, projectile, and scanner components.

System

Systems perform logic and take action globally against components under their purview. In our engine, there’s a system responsible for each component. In traditional ECS implementations, systems are often running in their own background threads, but I’m invoking mine sequentially to keep things as simple as I can while still building a functional game engine.

With the concerns cut apart into components, and systems being the only things allowed to write to a component, we now have an architecture that can handle large amounts of concurrency without ever creating a situation where anyone is waiting on locks. Read locks are “free,” and we can have as many of them active at a time as we want, but write locks are the ones that “stop the world,” so to speak.

Now, in each iteration of the game loop, for example, the motion system can write to each of the motion components, advancing that component based on its speed, heading, acceleration, etc. The motion system doesn’t care whether that component belongs to a player, a chair, or a doorknob. This narrow focus and division of responsibility is an elegant solution to concurrency that extends far beyond creating game engines with ECS, which is why I wanted to illustrate it in this chapter.

One potentially confusing aspect of a setup like this is that the robot API doesn’t really do much. It makes requests by setting some desired state and then releasing the write lock, allowing the next pass of the relevant system to do the real work. As you’ll see when you dig into the code, when a player launches a missile, the runtime just sets a missile status to ReadyToLaunch, and then the ProjectileSystem performs the actual launching of the missile the next time through the game loop.

Creating the Runtime Host

The first thing we need to do in order to pit WebAssembly robots against each other is load the modules and expose a Runtime for them. In previous chapters you saw how this works with the wasmi crate. Those implementations were pretty simple with only a function or two. Since we’ve got an entire API to support, we’ll need to do a little bit of encapsulation and abstraction to keep the runtime clean.

Create a new library project beneath your workspace root called botengine (remember to also add this to your workspace Cargo.toml). Edit botengine’s Cargo.toml to look like the following:

Before we get into the runtime host, let’s take a look at botengine/src/lib.rs. This is the root of the engine library, and it’s where we’re going to put the Combatant struct. The combatant is a wrapper around the loading, parsing, and interpreting of the WebAssembly module, as well as a start:

​①​

The Arc (Atomically Reference Counted) is what lets us share pointers to the GameState struct.

​②​

This function returns a JoinHandle but since the WebAssembly module is running an infinite loop, calling join is likely to never return.

​③​

Creates a new Runtime to host the WebAssembly module and passes it a reference to the game state.

​④​

Invokes the bot_init function in the WebAssembly module, starting the robot’s infinite loop.

There are a couple of subtle things happening in this code. First, the Runtime is being instantiated inside the combatant’s thread. This means the runtime doesn’t need to cross thread boundaries, which makes the Rust compiler happy. There’s also a clone happening of the player’s name converting the &str into a String, letting us get away without having to use a lifetime specifier.

The rest of the lib.rs file contains implementations of various error handling traits, as is considered best practice when you’re building a library you expect to expose to the rest of the world as a crate:

Next we need to implement the Runtime struct. We will put this in the botengine/src/runtime.rs file. This won’t compile yet because the functions in the runtime all defer to code that’s part of game state, components, or systems—all of which we’ll discuss shortly. There’s a lot of code here, and while I do enjoy typing, I wouldn’t recommend trying to type this all in by hand. Save some time and just grab the whole source for this sample:

In this file, the gen_funcref function is one of the most important. This function takes as input the name of a function (one of the imports the WebAssembly modules expect the host to provide) and returns a function reference in the form of an Option<FuncRef>.

With the mapping between import name and import signature created, next we need to be able to call functions at the request of the modules. As you’ve seen before, we do this through the Externals trait. The invoke_index function takes the index of a function (returned from our gen_funcref function) and invokes it.

Finally, you get to the real meat of the Runtime—implementing the functions imported by WebAssembly modules. There’s a lot of code here, but it all pretty much follows two basic patterns—reading from or writing to components. Occasionally we’ll defer a call to a system to do a complex query, but that’s an exception. Remember that the systems are the things tasked with processing, so these functions should do their work and get out of the way of the systems as quickly as possible. Let’s take a look at the code to make a change to a component, like the code from drive:

The writelock function (defined in the botengine/src/game/mod.rs file) creates a write lock on the self.game_state.motion_components HashMap. Then, the entry^[47] function is used to grab a reference to a single entry within the hash map. The Entry API is easily one of my favorites within all of Rust, and you owe it to yourself to learn it and exploit all of its power.

Anything inside the closure passed to and_modify has safe, mutable access to the value within that entry. In this case, it’s a single motion component and we modify it to set a new heading, reset its distance along that heading to 0, and set the desired speed.

The code to read a value from a component looks similar:

Here the readlock function (also a utility defined elsewhere) grabs a read lock on the damage_components HashMap. Instead of calling entry, we use get here and match on the result. As long as no other thread is writing to damage_components at that moment, acquiring a read lock is free.

Implementing the Game Loop

The game loop is the core gear that makes everything else in the game work. Without it, nothing happens. The job of this loop is to tell each of the active systems to apply its logic. The loop also keeps track of the number of loops or “cycles” that have occurred. When we discuss replay, you’ll see why this is important.

In more formal ECS frameworks, each system might be operating in its own loop, so you have a bunch of miniature game loops rather than a single all-encompassing one. I opted for the single loop here to keep the sample easier to read and to help with playback:

The GameLoop struct’s new function takes an Arc of the GameState, number of combatants, and a Sender as initial arguments. The Arc is used to allow code to pass multiple safe references to the same source object.

The core loop is very simple—iterate through each system and invoke its apply function. You’ll see a few of those in the next section. Let’s take a look at how the game state is defined:

First I use a couple of type aliases to keep the “bracket noise” down when nesting generic type parameters in the upcoming function signatures and structs. Each system has a corresponding ComponentHash, which is a HashMap wrapped by a RwLock and an Arc. In other words, the game state holds an atomically reference counted read-write lock on a hash map for each component.

In the combatant_entered function, each of the HashMaps is updated to contain a new default entry for the new player. Remember when I said the Entry API was incredibly powerful? Here it is again:

We can skip the usual ugliness with if statements or pattern matches to check for the existence of an item and just use the or_insert function to insert a new entry if one doesn’t already exist.

Next, we have what might be considered the most complex Rust syntax that I’ve used in the book so far:

All I’ve done is create a shortcut (I could also have used a macro) for calling .write().unwrap(). Every single access to every component for every system required the use of the RwLock methods, and I got tired of typing them all of the time.

This function signature includes a lifetime specifier (’a) and a generic type parameter (T). Without getting into too many gory details, this code indicates that the returned RwLockWriteGuard must last as long as the component passed in as a parameter. We use generics here so we can get a RwLockWriteGuard on any of the component HashMaps, even though each map contains values of different types.

Building the Components and Systems

As we went over earlier, a component is a small, discrete piece of state. A system is some set of logic that operates on that state. The separation of components and systems makes code easier to test, and the smaller surface area over which write locks need to be acquired help reduce the risk of “wait blocks” in the game loop.

I won’t show the code for each system in the interest of saving a few trees (and bytes). You can download all of the engine code and check out each system on your own. With a few exceptions, most of the system implementations try and mimic the behavior and spirit of Poindexter’s original Crobots code. Let’s take a look at the core of the damage system:

The apply function will mutate a damage component for each player in the game by calling the damage system’s advance function on that mutable reference, acquired from a write lock. Since the damage system is ideally the only part of the game that mutates damage components, we don’t have to worry about blocking other threads to perform these quick mutations.

There is a “breadcrumb” pattern used by multiple systems in the game. Each system sets some value at the end of its apply loop, leaving it behind to be used by other systems that process next. For example, the collision system leaves values in the collision components indicating detected collisions during that game loop cycle.

The damage system just reads any active collisions and applies damage accordingly. It’s responsible for ensuring that detected collisions only last as long as they should. For example, explosion damage from projectiles actually last a few cycles (also called “ticks”), which means the damage system may apply damage multiple times per explosion.

Let’s take a look at the rest of the code for the damage system to see how it applies collision damage, projectile damage, and then checks to see if players are dead:

The logger you’ve seen in the game loop and the damage system helps support playback, which we’ll discuss next.

Supporting Match Playback

The game engine’s loop runs as fast as it possibly can. This is fine for when we just want to determine who will win the match. But what if we want to watch the match live, or watch a replay of the match? To facilitate this, I’m using a Sender^[48] to send important game events bound to the cycle/frame during which those events occurred. Anything listening on the other end of that channel can store the events in a database, display them on the console, or render them on a website. To show them at a more human-friendly speed, just apply a frame rate delay between events such that each cycle represents some fixed fraction of a second.