This chapter outlines the basic patterns and principles for using MongoDB as a persistent storage engine for an online game, particularly one that contains role-playing characteristics.
In designing an online game, there is a need to store various data about the player’s character. Some of the attributes might include:
In addition, we need to store all this data for large numbers of players who might be playing the game simultaneously, and this data needs to be both readable and writable with minimal latency in order to ensure responsiveness during gameplay.
In addition to the preceding data, we also need to store data for:
Another consideration when designing the persistence backend for an online game is its flexibility. Particularly in early releases of a game, we might wish to change gameplay mechanics significantly as players provide feedback. When implementing these changes, being able to migrate persistent data from one format to another with minimal (or no) downtime is essential.
The solution presented by this use case assumes that the read and write performance is equally important and must be accessible with minimal latency.
Ultimately, the particulars of the schema depend on the
design of the game. When designing our schema, we’ll try to
encapsulate all the commonly used data into a small number of objects in
order to minimize the number of queries to the database and the number
of seeks in a query. Encapsulating all player state into a character
collection, item data into an item
collection, and location data into
a location
collection satisfies both these criteria.
In a role-playing game, then, a typical character state document might look like the following:
{
_id
:
ObjectId
(
'...'
),
name
:
'Tim'
,
character
:
{
intrinsics
:
{
strength
:
10
,
dexterity
:
16
,
intelligence
:
17
,
charisma
:
8
},
'class'
:
'mage'
,
health
:
212
,
mana
:
152
},
location
:
{
id
:
'maze-1'
,
description
:
'a maze of twisty little passages...'
,
exits
:
{
n
:
'maze-2'
,
s
:
'maze-1'
,
e
:
'maze-3'
},
players
:
[
{
id
:
ObjectId
(
'...'
),
name
:
'grue'
},
{
id
:
ObjectId
(
'...'
),
name
:
'Tim'
}
],
inventory
:
[
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
'scroll of cause fear'
}]
},
gold
:
523
,
armor
:
[
{
id
:
ObjectId
(
'...'
),
region
:
'head'
},
{
id
:
ObjectId
(
'...'
),
region
:
'body'
},
{
id
:
ObjectId
(
'...'
),
region
:
'feet'
}],
weapons
:
[
{
id
:
ObjectId
(
'...'
),
hand
:
'both'
}
],
inventory
:
[
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
'backpack'
,
inventory
:
[
{
qty
:
4
,
id
:
ObjectId
(
'...'
),
name
:
'potion of healing'
},
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
'scroll of magic mapping'
},
{
qty
:
2
,
id
:
ObjectId
(
'...'
),
name
:
'c-rations'
}
]},
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
"wizard's hat"
,
bonus
:
3
},
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
"wizard's robe"
,
bonus
:
0
},
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
"old boots"
,
bonus
:
0
},
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
"quarterstaff"
,
bonus
:
2
}
]
}
There are a few things to note about this document:
location
attribute. Note in particular that all
of the information necessary to describe the room is encapsulated within
the character state document. This allows the game system to render the
room without making a second query to the database to get room
information.
armor
and weapons
attributes contain little information
about the actual items being worn or carried. This information is
actually stored under the inventory
property. Since the inventory
information is stored in the same document, there is no need to
replicate the detailed information about each item into the armor
and
weapons
properties.
inventory
contains the item details necessary for rendering each
item in the character’s possession, including any enchantments (bonus
)
and quantity
. Once again, embedding this data into the character
record means we don’t have to perform a separate query to fetch item
details necessary for display.
Likewise, the item schema should include all details about all items globally in the game:
{
_id
:
ObjectId
(
'...'
),
name
:
'backpack'
,
bonus
:
null
,
inventory
:
[
{
qty
:
4
,
id
:
ObjectId
(
'...'
),
name
:
'potion of healing'
},
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
'scroll of magic mapping'
},
{
qty
:
2
,
id
:
ObjectId
(
'...'
),
name
:
'c-rations'
}
]},
weight
:
12
,
price
:
160
,
...
}
Note that this document contains more or less the same information as
stored in the inventory
attribute of character
documents, as well as
additional data that may only be needed sporadically in the case of
gameplay such as weight
and price
.
Finally, the location
schema specifies the state of the world in the
game:
{
id
:
'maze-1'
,
description
:
'a maze of twisty little passages...'
,
exits
:
{
n
:
'maze-2'
,
s
:
'maze-1'
,
e
:
'maze-3'
},
players
:
[
{
id
:
ObjectId
(
'...'
),
name
:
'grue'
},
{
id
:
ObjectId
(
'...'
),
name
:
'Tim'
}
],
inventory
:
[
{
qty
:
1
,
id
:
ObjectId
(
'...'
),
name
:
'scroll of cause fear'
}
],
}
Here, note that location
stores exactly the same information as is
stored in the location
attribute of the character
document. We’ll
use location
as the system of record when the game requires
interaction between multiple characters or between characters and
noninventory items.
In an online gaming system, with the state embedded in a single document
for character
, item
, and location
, the primary operations we’ll
be performing are as follows:
_id
This section describes procedures for performing these
queries, extractions, and updates. In particular, we will avoid loading
the location
or item
documents except when absolutely necessary.
The most basic operation in this system is loading the character state:
>>>
character
=
db
.
characters
.
find_one
({
'_id'
:
character_id
})
In this case, the default index that MongoDB supplies on the _id
field
is sufficient for good performance of this query.
In order to save space, the character
schema just described stores
item details only in the inventory
attribute, storing ObjectId
s in other locations. To display these item details, as on a character
summary window, we need to merge the information from the armor
and
weapons
attributes with information from the inventory
attribute.
Suppose, for instance, that our code is displaying the armor data using the following Jinja2 template:
<div>
<h2>
Armor</h2>
<dl>
{% if value.head %}<dt>
Helmet</dt>
<dd>
{{value.head[0].description}}</dd>
{% endif %} {% if value.hands %}<dt>
Gloves</dt>
<dd>
{{value.hands[0].description}}</dd>
{% endif %} {% if value.feet %}<dt>
Boots</dt>
<dd>
{{value.feet[0].description}}</dd>
{% endif %} {% if value.body %}<dt>
Body Armor</dt>
<dd><ul>
{% for piece in value.body %}<li>
piece.description</li>
{% endfor %}</ul></dd>
{% endif %}</dl>
</dd>
In this case, we want the various description
fields to be text
similar to “+3 wizard’s hat.” The context passed to this template,
then, would be of the following form:
{
"head"
:
[
{
"id"
:
...
,
"description"
:
"+3 wizard's hat"
}
],
"hands"
:
[],
"feet"
:
[
{
"id"
:
...
,
"description"
:
"old boots"
}
],
"body"
:
[
{
"id"
:
...
,
"description"
:
"wizard's robe"
}
],
}
In order to build up this structure, we’ll use the following helper functions:
def
get_item_index
(
inventory
):
'''Given an inventory attribute, recursively build up an item
index (including all items contained within other items)
'''
result
=
{}
for
item
in
inventory
:
result
[
item
[
'_id'
]]
=
item
if
'inventory'
in
item
:
result
.
update
(
get_item_index
(
item
[
'inventory]))
return
result
def
describe_item
(
item
):
'''Add a 'description' field to the given item'''
result
=
dict
(
item
)
if
item
[
'bonus'
]:
description
=
'
%+d
%s
'
%
(
item
[
'bonus'
],
item
[
'name'
])
else
:
description
=
item
[
'name'
]
result
[
'description'
]
=
description
return
result
def
get_armor_for_display
(
character
,
item_index
):
'''Given a character document, return an 'armor' value
suitable for display'''
result
=
dict
(
head
=
[],
hands
=
[],
feet
=
[],
body
=
[])
for
piece
in
character
[
'armor'
]:
item
=
describe_item
(
item_index
[
piece
[
'id'
]])
result
[
piece
[
'region'
]]
.
append
(
item
)
return
result
In order to actually display the armor, then, we’d use the following code:
>>>
item_index
=
get_item_index
(
...
character
[
'inventory'
]
+
character
[
'location'
][
'inventory'
])
>>>
armor
=
get_armor_for_dislay
(
character
,
item_index
)
Note in particular that we’re building an index not only for the items the character is actually carrying in inventory, but also for the items that the player might interact with in the room.
Similarly, in order to display the weapon information, we need to build a structure such as the following:
{
"left"
:
None
,
"right"
:
None
,
"both"
:
{
"description"
:
"+2 quarterstaff"
}
}
The helper function is similar to that for get_armor_for_display
:
def
get_weapons_for_display
(
character
,
item_index
):
'''Given a character document, return a 'weapons' value
suitable for display'''
result
=
dict
(
left
=
None
,
right
=
None
,
both
=
None
)
for
piece
in
character
[
'weapons'
]:
item
=
describe_item
(
item_index
[
piece
[
'id'
]])
result
[
piece
[
'hand'
]]
=
item
return
result
In order to actually display the weapons, then, we’d use the following code:
>>>
armor
=
get_weapons_for_display
(
character
,
item_index
)
In order to display information about the character’s attributes, inventory, and surroundings, we also need to extract fields from the character state. In this case, however, the schema just defined keeps all the relevant information for display embedded in those sections of the document. The code for extracting this data, then, is the following:
>>>
attributes
=
character
[
'character'
]
>>>
inventory
=
character
[
'inventory'
]
>>>
room_data
=
character
[
'location'
]
In our game, suppose the player decides to pick up an item from the room and add it to their inventory. In this case, we need to update both the character state and the global location state:
def
pick_up_item
(
character
,
item_index
,
item_id
):
'''Transfer an item from the current room to the character's inventory'''
item
=
item_index
[
item_id
]
character
[
'inventory'
]
.
append
(
item
)
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
]
},
{
'$push'
:
{
'inventory'
:
item
},
'$pull'
:
{
'location.inventory'
:
{
'_id'
:
item
[
'id'
]
}
}
})
db
.
location
.
update
(
{
'_id'
:
character
[
'location'
][
'id'
]
},
{
'$pull'
:
{
'inventory'
:
{
'id'
:
item_id
}
}
})
While the preceding code may be for a single-player game, if we allow
multiple players or nonplayer characters to pick up items, that
introduces a problem where two characters may try to
pick up an item simultaneously. To guard against that, we can use the
location
collection to decide between ties. In this case, the code is
now the following:
def
pick_up_item
(
character
,
item_index
,
item_id
):
'''Transfer an item from the current room to the character's inventory'''
item
=
item_index
[
item_id
]
character
[
'inventory'
]
.
append
(
item
)
result
=
db
.
location
.
update
(
{
'_id'
:
character
[
'location'
][
'id'
],
'inventory.id'
:
item_id
},
{
'$pull'
:
{
'inventory'
:
{
'id'
:
item_id
}
}
},
safe
=
True
)
if
not
result
[
'updatedExisting'
]:
raise
Conflict
()
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
]
},
{
'$push'
:
{
'inventory'
:
item
},
'$pull'
:
{
'location'
:
{
'_id'
:
item
[
'id'
]
}
}
})
By ensuring that the item is present before removing it from the room in
the update
call, we guarantee that only one player/nonplayer
character/monster can pick up the item.
In the game described here, the backpack
item can contain other items.
We might further suppose that some other items may be similarly
hierarchical (e.g., a chest in a room). Suppose that the player wishes to
move an item from one of these “containers” into their active
inventory
as a prelude to using it. In this case, we need to update
both the character state and the item state:
def
move_to_active_inventory
(
character
,
item_index
,
container_id
,
item_id
):
'''Transfer an item from the given container to the character's active
inventory
'''
result
=
db
.
item
.
update
(
{
'_id'
:
container_id
,
'inventory.id'
:
item_id
},
{
'$pull'
:
{
'inventory'
:
{
'id'
:
item_id
}
}
},
safe
=
True
)
if
not
result
[
'updatedExisting'
]:
raise
Conflict
()
item
=
item_index
[
item_id
]
container
=
item_index
[
item_id
]
character
[
'inventory'
]
.
append
(
item
)
container
[
'inventory'
]
=
[
item
for
item
in
container
[
'inventory'
]
if
item
[
'_id'
]
!=
item_id
]
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
]
},
{
'$push'
:
{
'inventory'
:
item
}
}
)
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
],
'inventory.id'
:
container_id
},
{
'$pull'
:
{
'inventory.$.inventory'
:
{
'id'
:
item_id
}
}
}
)
Note in this code that we:
Ensure that the item’s state makes this update reasonable (the item is actually contained within the container). Abort with an error if this is not true.
Update the in-memory character
document’s inventory, adding the
item.
Update the in-memory container
document’s inventory, removing the
item.
Update the character
document in MongoDB.
In the case that the character is moving an item from a container in his own inventory, update the character’s inventory representation of the container.
In our game, suppose the player decides to move north. In this case, we need to update the character state to match the new location:
def
move
(
character
,
direction
):
'''Move the character to a new location'''
# Remove character from current location
db
.
location
.
update
(
{
'_id'
:
character
[
'location'
][
'id'
]
},
{
'$pull'
:
{
'players'
:
{
'id'
:
character
[
'_id'
]
}
}
})
# Add character to new location, retrieve new location data
new_location
=
db
.
location
.
find_and_modify
(
{
'_id'
:
character
[
'location'
][
'exits'
][
direction
]
},
{
'$push'
:
{
'players'
:
{
'id'
:
character
[
'_id'
],
'name'
:
character
[
'name'
]
}
}
},
new
=
True
)
character
[
'location'
]
=
new_location
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
]
},
{
'$set'
:
{
'location'
:
new_location
}
})
Here, note that the code updates the old room, the new room, and the
character document. Since we’re using $push
and $pull
operations to update
the location
collection, we don’t need to worry about race conditions.
If the character wants to buy an item, we need to do the following:
The following code does just that:
def
buy
(
character
,
shopkeeper
,
item_id
):
'''Pick up an item, add to the character's inventory, and transfer
payment to the shopkeeper
'''
price
=
db
.
item
.
find_one
({
'_id'
:
item_id
},
{
'price'
:
1
})[
'price'
]
result
=
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
],
'gold'
:
{
'$gte'
:
price
}
},
{
'$inc'
:
{
'gold'
:
-
price
}
},
safe
=
True
)
if
not
result
[
'updatedExisting'
]:
raise
InsufficientFunds
()
try
:
pick_up_item
(
character
,
item_id
)
except
:
# Add the gold back to the character
result
=
db
.
character
.
update
(
{
'_id'
:
character
[
'_id'
]
},
{
'$inc'
:
{
'gold'
:
price
}
}
)
raise
character
[
'gold'
]
-=
price
db
.
character
.
update
(
{
'_id'
:
shopkeeper
[
'_id'
]
},
{
'$inc'
:
{
'gold'
:
price
}
}
)
Note that the buy()
function ensures that the character has sufficient gold
to pay for the item using the updatedExisting
trick used for picking
up items. The race condition for item pickup is handled as well,
“rolling back” the removal of gold from the character’s wallet if the
item cannot be picked up.
If the system needs to scale beyond a single MongoDB node, we’ll
want to use a sharded cluster. Sharding in this use case is fairly straightforward,
since all our items are always retrieved by _id
. To shard the character
and
location
collections, the commands would be the following:
>>>
db
.
command
(
'shardcollection'
,
'dbname.character'
,
{
...
'key'
:
{
'_id'
:
1
}
})
{ "collectionsharded" : "dbname.character", "ok" : 1 }
>>>
db
.
command
(
'shardcollection'
,
'dbname.location'
,
{
...
'key'
:
{
'_id'
:
1
}
})
{ "collectionsharded" : "dbname.location", "ok" : 1 }
3.14.246.148