MongoDB is very powerful, but it is still easy to get started with. In this chapter we’ll introduce some of the basic concepts of MongoDB:
A document is the basic unit of data for MongoDB, roughly equivalent to a row in a relational database management system (but much more expressive).
Similarly, a collection can be thought of as the schema-free equivalent of a table.
A single instance of MongoDB can host multiple independent databases, each of which can have its own collections and permissions.
MongoDB comes with a simple but powerful JavaScript shell, which is useful for the administration of MongoDB instances and data manipulation.
Every document has a special key, "_id"
, that
is unique across the document’s collection.
At the heart of MongoDB is the concept of a document: an ordered set of keys with associated values. The representation of a document differs by programming language, but most languages have a data structure that is a natural fit, such as a map, hash, or dictionary. In JavaScript, for example, documents are represented as objects:
{"greeting" : "Hello, world!"}
This simple document contains a single key,
"greeting"
, with a value of "Hello,
world!"
. Most documents will be more complex than this simple
one and often will contain multiple key/value pairs:
{"greeting" : "Hello, world!", "foo" : 3}
This example is a good illustration of several important concepts:
Key/value pairs in documents are ordered—the earlier document is distinct from the following document:
{"foo" : 3, "greeting" : "Hello, world!"}
In most cases the ordering of keys in documents is not important. In fact, in some programming languages the default representation of a document does not even maintain ordering (e.g., dictionaries in Python and hashes in Perl or Ruby 1.8). Drivers for those languages usually have some mechanism for specifying documents with ordering for the rare cases when it is necessary. (Those cases will be noted throughout the text.)
Values in documents are not just “blobs.” They can be one of
several different data types (or even an entire embedded document—see
Embedded Documents). In this example the value for
"greeting"
is a string, whereas the value for
"foo"
is an integer.
The keys in a document are strings. Any UTF-8 character is allowed in a key, with a few notable exceptions:
Keys must not contain the character (the
null
character). This character is used to signify
the end of a key.
The . and $ characters have some special properties and should be used only in certain circumstances, as described in later chapters. In general, they should be considered reserved, and drivers will complain if they are used inappropriately.
Keys starting with _ should be considered reserved; although this is not strictly enforced.
MongoDB is type-sensitive and case-sensitive. For example, these documents are distinct:
{"foo" : 3} {"foo" : "3"}
As are as these:
{"foo" : 3} {"Foo" : 3}
A final important thing to note is that documents in MongoDB cannot contain duplicate keys. For example, the following is not a legal document:
{"greeting" : "Hello, world!", "greeting" : "Hello, MongoDB!"}
A collection is a group of documents. If a document is the MongoDB analog of a row in a relational database, then a collection can be thought of as the analog to a table.
Collections are schema-free. This means that the documents within a single collection can have any number of different “shapes.” For example, both of the following documents could be stored in a single collection:
{"greeting" : "Hello, world!"} {"foo" : 5}
Note that the previous documents not only have different types for their values (string versus integer) but also have entirely different keys. Because any document can be put into any collection, the question often arises: “Why do we need separate collections at all?” It’s a good question—with no need for separate schemas for different kinds of documents, why should we use more than one collection? There are several good reasons:
Keeping different kinds of documents in the same collection can be a nightmare for developers and admins. Developers need to make sure that each query is only returning documents of a certain kind or that the application code performing a query can handle documents of different shapes. If we’re querying for blog posts, it’s a hassle to weed out documents containing author data.
It is much faster to get a list of collections than to extract
a list of the types in a collection. For example, if we had a
type
key in the collection that said whether each
document was a “skim,” “whole,” or “chunky monkey” document, it
would be much slower to find those three values in a single
collection than to have three separate collections and query for
their names (see Subcollections).
Grouping documents of the same kind together in the same collection allows for data locality. Getting several blog posts from a collection containing only posts will likely require fewer disk seeks than getting the same posts from a collection containing posts and author data.
We begin to impose some structure on our documents when we create indexes. (This is especially true in the case of unique indexes.) These indexes are defined per collection. By putting only documents of a single type into the same collection, we can index our collections more efficiently.
As you can see, there are sound reasons for creating a schema and for grouping related types of documents together. MongoDB just relaxes this requirement and allows developers more flexibility.
A collection is identified by its name. Collection names can be any UTF-8 string, with a few restrictions:
The empty string (“”) is not a valid collection name.
Collection names may not contain the character (the
null
character) because this delineates the end
of a collection name.
You should not create any collections that start with system., a prefix reserved for system collections. For example, the system.users collection contains the database’s users, and the system.namespaces collection contains information about all of the database’s collections.
User-created collections should not contain the reserved character $ in the name. The various drivers available for the database do support using $ in collection names because some system-generated collections contain it. You should not use $ in a name unless you are accessing one of these collections.
One convention for organizing collections is to use namespaced subcollections separated by the . character. For example, an application containing a blog might have a collection named blog.posts and a separate collection named blog.authors. This is for organizational purposes only—there is no relationship between the blog collection (it doesn’t even have to exist) and its “children.”
Although subcollections do not have any special properties, they are useful and incorporated into many MongoDB tools:
GridFS, a protocol for storing large files, uses subcollections to store file metadata separately from content chunks (see Chapter 7 for more information about GridFS).
The MongoDB web console organizes the data in its DBTOP section by subcollection (see Chapter 8 for more information on administration).
Most drivers provide some syntactic sugar for accessing a
subcollection of a given collection. For example, in the database
shell, db.blog
will give you the
blog collection, and
db.blog.posts
will give you the
blog.posts collection.
Subcollections are a great way to organize data in MongoDB, and their use is highly recommended.
In addition to grouping documents by collection, MongoDB groups collections into databases. A single instance of MongoDB can host several databases, each of which can be thought of as completely independent. A database has its own permissions, and each database is stored in separate files on disk. A good rule of thumb is to store all data for a single application in the same database. Separate databases are useful when storing data for several application or users on the same MongoDB server.
Like collections, databases are identified by name. Database names can be any UTF-8 string, with the following restrictions:
The empty string (“”) is not a valid database name.
A database name cannot contain any of these characters: ' ' (a single space), ., $, /, , or (the null character).
Database names should be all lowercase.
Database names are limited to a maximum of 64 bytes.
One thing to remember about database names is that they will actually end up as files on your filesystem. This explains why many of the previous restrictions exist in the first place.
There are also several reserved database names, which you can access directly but have special semantics. These are as follows:
This is the “root” database, in terms of authentication. If a user is added to the admin database, the user automatically inherits permissions for all databases. There are also certain server-wide commands that can be run only from the admin database, such as listing all of the databases or shutting down the server.
This database will never be replicated and can be used to store any collections that should be local to a single server (see Chapter 9 for more information about replication and the local database).
When Mongo is being used in a sharded setup (see Chapter 10), the config database is used internally to store information about the shards.
By prepending a collection’s name with its containing database, you
can get a fully qualified collection name called a
namespace. For instance, if you are using the
blog.posts collection in the cms
database, the namespace of that collection would be
cms.blog.posts
. Namespaces are limited to 121 bytes in
length and, in practice, should be less than 100 bytes long. For more on
namespaces and the internal representation of collections in MongoDB, see
Appendix C.
MongoDB is almost always run as a network server that clients can
connect to and perform operations on. To start the server, run the
mongod
executable:
$ ./mongod ./mongod --help for help and startup options Sun Mar 28 12:31:20 Mongo DB : starting : pid = 44978 port = 27017 dbpath = /data/db/ master = 0 slave = 0 64-bit Sun Mar 28 12:31:20 db version v1.5.0-pre-, pdfile version 4.5 Sun Mar 28 12:31:20 git version: ... Sun Mar 28 12:31:20 sys info: ... Sun Mar 28 12:31:20 waiting for connections on port 27017 Sun Mar 28 12:31:20 web admin interface listening on port 28017
Or if you’re on Windows, run this:
$ mongod.exe
For detailed information on installing MongoDB on your system, see Appendix A.
When run with no arguments, mongod
will use the
default data directory, /data/db/ (or
C:datadb on Windows), and port 27017. If the data
directory does not already exist or is not writable, the server will fail
to start. It is important to create the data directory (e.g.,
mkdir -p /data/db/), and to make sure your user has
permission to write to the directory, before starting MongoDB. The server
will also fail to start if the port is not available—this is often caused
by another instance of MongoDB that is already running.
The server will print some version and system information and then begin waiting for connections. By default, MongoDB listens for socket connections on port 27017.
mongod
also sets up a very basic HTTP server that
listens on a port 1,000 higher than the main port, in this case 28017.
This means that you can get some administrative information about your
database by opening a web browser and going to http://localhost:28017.
You can safely stop mongod
by typing Ctrl-c in
the shell that is running the server.
For more information on starting or stopping MongoDB, see Starting and Stopping MongoDB, and for more on the administrative interface, see Using the Admin Interface.
MongoDB comes with a JavaScript shell that allows interaction with a
MongoDB instance from the command line. The shell is very useful for
performing administrative functions, inspecting a running instance, or
just playing around. The mongo
shell is a crucial tool
for using MongoDB and is used extensively throughout the rest of the
text.
To start the shell, run the mongo
executable:
$ ./mongo MongoDB shell version: 1.6.0 url: test connecting to: test type "help" for help >
The shell automatically attempts to connect to a MongoDB server on
startup, so make sure you start mongod
before
starting the shell.
The shell is a full-featured JavaScript interpreter, capable of running arbitrary JavaScript programs. To illustrate this, let’s perform some basic math:
> x = 200 200 > x / 5; 40
We can also leverage all of the standard JavaScript libraries:
> Math.sin(Math.PI / 2); 1 > new Date("2010/1/1"); "Fri Jan 01 2010 00:00:00 GMT-0500 (EST)" > "Hello, World!".replace("World", "MongoDB"); Hello, MongoDB!
We can even define and call JavaScript functions:
> function factorial (n) { ... if (n <= 1) return 1; ... return n * factorial(n - 1); ... } > factorial(5); 120
Note that you can create multiline commands. The shell will detect whether the JavaScript statement is complete when you press Enter and, if it is not, will allow you to continue writing it on the next line.
Although the ability to execute arbitrary JavaScript is cool, the
real power of the shell lies in the fact that it is also a stand-alone
MongoDB client. On startup, the shell connects to the
test database on a MongoDB server and assigns this
database connection to the global variable db
. This
variable is the primary access point to MongoDB through the
shell.
The shell contains some add-ons that are not valid JavaScript syntax but were implemented because of their familiarity to users of SQL shells. The add-ons do not provide any extra functionality, but they are nice syntactic sugar. For instance, one of the most important operations is selecting which database to use:
> use foobar switched to db foobar
Now if you look at the db
variable, you can see
that it refers to the foobar database:
> db foobar
Because this is a JavaScript shell, typing a variable will convert the variable to a string (in this case, the database name) and print it.
Collections can be accessed from the db
variable. For example, db.baz
returns the
baz collection in the current database. Now that we
can access a collection in the shell, we can perform almost any database
operation.
We can use the four basic operations, create, read, update, and delete (CRUD), to manipulate and view data in the shell.
The insert
function adds a document to a
collection. For example, suppose we want to store a blog post. First,
we’ll create a local variable called post
that is a
JavaScript object representing our document. It will have the keys
"title"
, "content"
, and
"date"
(the date that it was published):
> post = {"title" : "My Blog Post", ... "content" : "Here's my blog post.", ... "date" : new Date()} { "title" : "My Blog Post", "content" : "Here's my blog post.", "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)" }
This object is a valid MongoDB document, so we can save it to
the blog collection using the
insert
method:
> db.blog.insert(post)
The blog post has been saved to the database. We can see it by
calling find
on the collection:
> db.blog.find() { "_id" : ObjectId("4b23c3ca7525f35f94b60a2d"), "title" : "My Blog Post", "content" : "Here's my blog post.", "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)" }
You can see that an "_id"
key was added and
that the other key/value pairs were saved as we entered them. The
reason for "_id"
’s sudden appearance is explained
at the end of this chapter.
find
returns all of the documents in a
collection. If we just want to see one document from a collection, we
can use findOne
:
> db.blog.findOne() { "_id" : ObjectId("4b23c3ca7525f35f94b60a2d"), "title" : "My Blog Post", "content" : "Here's my blog post.", "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)" }
find
and findOne
can
also be passed criteria in the form of a query document. This will
restrict the documents matched by the query. The shell will
automatically display up to 20 documents matching a
find
, but more can be fetched. See Chapter 4 for more information on querying.
If we would like to modify our post, we can use
update
. update
takes (at
least) two parameters: the first is the criteria to find which
document to update, and the second is the new document. Suppose we
decide to enable comments on the blog post we created earlier. We’ll
need to add an array of comments as the value for a new key in our
document.
The first step is to modify the variable post
and add a "comments"
key:
> post.comments = [] [ ]
Then we perform the update, replacing the post titled “My Blog Post” with our new version of the document:
> db.blog.update({title : "My Blog Post"}, post)
Now the document has a
"comments"
key. If we call
find
again, we can see the new key:
> db.blog.find() { "_id" : ObjectId("4b23c3ca7525f35f94b60a2d"), "title" : "My Blog Post", "content" : "Here's my blog post.", "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)" "comments" : [ ] }
remove
deletes documents permanently from
the database. Called with no parameters, it removes all documents from
a collection. It can also take a document specifying criteria for
removal. For example, this would remove the post we just
created:
> db.blog.remove({title : "My Blog Post"})
Now the collection will be empty again.
Because mongo
is simply a JavaScript shell, you
can get a great deal of help for it by simply looking up JavaScript
documentation online. The shell also includes built-in help that can be
accessed by typing help
:
> help HELP show dbs show database names show collections show collections in current database show users show users in current database show profile show recent system.profile entries w. time >= 1ms use <db name> set current database to <db name> db.help() help on DB methods db.foo.help() help on collection methods db.foo.find() list objects in collection foo db.foo.find( { a : 1 } ) list objects in foo where a == 1 it result of the last line evaluated
Help for database-level commands is provided by
db.help();
, and help at the collections can be
accessed with db.foo.help();
.
A good way of figuring out what a function is doing is to type it
without the parentheses. This will print the JavaScript source code for
the function. For example, if we are curious about how the
update
function works or cannot remember the order
of parameters, we can do the following:
> db.foo.update function (query, obj, upsert, multi) { assert(query, "need a query"); assert(obj, "need an object"); this._validateObject(obj); this._mongo.update(this._fullName, query, obj, upsert ? true : false, multi ? true : false); }
There is also an autogenerated API of all the JavaScript functions provided by the shell at http://api.mongodb.org/js.
Fetching a collection with
db.
almost
always works, unless the collection name actually is a property of the
database class. For instance, if we are trying to access the
version collection, we cannot say
collectionName
db.version
because db.version
is a database function. (It returns the version of the running MongoDB
server.)
> db.version function () { return this.serverBuildInfo().version; }
db
’s collection-returning behavior is only a
fallback for when JavaScript cannot find a matching property. When
there is a property with the same name as the desired collection, we
can use the getCollection
function:
> db.getCollection("version"); test.version
This can also be handy for collections with invalid JavaScript
in their names. For example, foo-bar is a valid
collection name, but it’s variable subtraction in JavaScript. You can
get the foo-bar collection with
db.getCollection("foo-bar")
.
In JavaScript, x.y
is identical to
x['y']
. This means that subcollections can be
accessed using variables, not just literal names. That is, if you
needed to perform some operation on every blog
subcollection, you could iterate through them with something like
this:
var collections = ["posts", "comments", "authors"]; for (i in collections) { doStuff(db.blog[collections[i]]); }
Instead of this:
doStuff(db.blog.posts); doStuff(db.blog.comments); doStuff(db.blog.authors);
The beginning of this chapter covered the basics of what a document is. Now that you are up and running with MongoDB and can try things on the shell, this section will dive a little deeper. MongoDB supports a wide range of data types as values in documents. In this section, we’ll outline all of the supported types.
Documents in MongoDB can be thought of as “JSON-like” in that they are conceptually similar to objects in JavaScript. JSON is a simple representation of data: the specification can be described in about one paragraph (http://www.json.org proves it) and lists only six data types. This is a good thing in many ways: it’s easy to understand, parse, and remember. On the other hand, JSON’s expressive capabilities are limited, because the only types are null, boolean, numeric, string, array, and object.
Although these types allow for an impressive amount of expressivity, there are a couple of additional types that are crucial for most applications, especially when working with a database. For example, JSON has no date type, which makes working with dates even more annoying than it usually is. There is a number type, but only one—there is no way to differentiate floats and integers, never mind any distinction between 32-bit and 64-bit numbers. There is no way to represent other commonly used types, either, such as regular expressions or functions.
MongoDB adds support for a number of additional data types while keeping JSON’s essential key/value pair nature. Exactly how values of each type are represented varies by language, but this is a list of the commonly supported types and how they are represented as part of a document in the shell:
Null can be used to represent both a null value and a nonexistent field:
{"x" : null}
There is a boolean type, which will be used for the values
'true'
and
'false'
:
{"x" : true}
This cannot be represented on the shell. As mentioned earlier, JavaScript supports only 64-bit floating point numbers, so 32-bit integers will be converted into those.
Again, the shell cannot represent these. The shell will display them using a special embedded document; see the section Numbers for details.
All numbers in the shell will be of this type. Thus, this will be a floating-point number:
{"x" : 3.14}
As will this:
{"x" : 3}
Any string of UTF-8 characters can be represented using the string type:
{"x" : "foobar"}
This type is not supported by the shell. If the shell gets a symbol from the database, it will convert it into a string.
An object id is a unique 12-byte ID for documents. See the section _id and ObjectIds for details:
{"x" : ObjectId()}
Dates are stored as milliseconds since the epoch. The time zone is not stored:
{"x" : new Date()}
Documents can contain regular expressions, using JavaScript’s regular expression syntax:
{"x" : /foobar/i}
Documents can also contain JavaScript code:
{"x" : function() { /* ... */ }}
Binary data is a string of arbitrary bytes. It cannot be manipulated from the shell.
BSON contains a special type representing the largest possible value. The shell does not have a type for this.
BSON contains a special type representing the smallest possible value. The shell does not have a type for this.
Undefined can be used in documents as well (JavaScript has distinct types for null and undefined):
{"x" : undefined}
Sets or lists of values can be represented as arrays:
{"x" : ["a", "b", "c"]}
Documents can contain entire documents, embedded as values in a parent document:
{"x" : {"foo" : "bar"}}
JavaScript has one “number” type. Because MongoDB has three number types (4-byte integer, 8-byte integer, and 8-byte float), the shell has to hack around JavaScript’s limitations a bit. By default, any number in the shell is treated as a double by MongoDB. This means that if you retrieve a 4-byte integer from the database, manipulate its document, and save it back to the database even without changing the integer, the integer will be resaved as a floating-point number. Thus, it is generally a good idea not to overwrite entire documents from the shell (see Chapter 3 for information on making changes to the values of individual keys).
Another problem with every number being represented by a double is
that there are some 8-byte integers that cannot be accurately
represented by 8-byte floats. Therefore, if you save an 8-byte integer
and look at it in the shell, the shell will display it as an embedded
document indicating that it might not be exact. For example, if we save
a document with a "myInteger"
key whose value is the
64-bit integer, 3
, and then look at it in the shell,
it will look like this:
> doc = db.nums.findOne() { "_id" : ObjectId("4c0beecfd096a2580fe6fa08"), "myInteger" : { "floatApprox" : 3 } }
The number is not changed in the database (unless you modify and resave the object from the shell, in which case it will turn into a float); the embedded document just indicates that the shell is displaying a floating-point approximation of an 8-byte integer. If this embedded document has only one key, it is, in fact, exact.
If you insert an 8-byte integer that cannot be accurately
displayed as a double, the shell will add two keys,
"top"
and "bottom"
, containing the
32-bit integers representing the 4 high-order bytes and 4 low-order
bytes of the integer, respectively. For instance, if we insert
9223372036854775807
, the shell will show us the
following:
> db.nums.findOne() { "_id" : ObjectId("4c0beecfd096a2580fe6fa09"), "myInteger" : { "floatApprox" : 9223372036854776000, "top" : 2147483647, "bottom" : 4294967295 } }
The "floatApprox"
embedded documents are
special and can be manipulated as numbers as well as documents:
> doc.myInteger + 1 4 > doc.myInteger.floatApprox 3
All 4-byte integers can be represented exactly by an 8-byte floating-point number, so they are displayed normally.
In JavaScript, the Date
object is used for
MongoDB’s date type. When creating a new Date
object,
always call new Date(...)
, not just
Date(...)
. Calling the constructor as a function
(that is, not including new
) returns a string
representation of the date, not an actual Date
object. This is not MongoDB’s choice; it is how JavaScript works. If you
are not careful to always use the Date
constructor,
you can end up with a mishmash of strings and dates. Strings do not
match dates, and vice versa, so this can cause problems with removing,
updating, querying…pretty much everything.
For a full explanation of JavaScript’s Date
class and acceptable formats for the constructor, see ECMAScript
specification section 15.9 (available for download at http://www.ecmascript.org).
Dates in the shell are displayed using local time zone settings. However, dates in the database are just stored as milliseconds since the epoch, so they have no time zone information associated with them. (Time zone information could, of course, be stored as the value for another key.)
Arrays are values that can be interchangeably used for both ordered operations (as though they were lists, stacks, or queues) and unordered operations (as though they were sets).
In the following document, the key "things"
has
an array value:
{"things" : ["pie", 3.14]}
As we can see from the example, arrays can contain different data types as values (in this case, a string and a floating-point number). In fact, array values can be any of the supported values for normal key/value pairs, even nested arrays.
One of the great things about arrays in documents is that MongoDB
“understands” their structure and knows how to “reach inside” of arrays
to perform operations on their contents. This allows us to query on
arrays and build indexes using their contents. For instance, in the
previous example, MongoDB can query for all documents where 3.14 is an
element of the "things"
array. If this is a common
query, you can even create an index on the "things"
key to improve the query’s speed.
MongoDB also allows atomic updates that modify the contents of arrays, such as reaching into the array and changing the value pie to pi. We’ll see more examples of these types of operations throughout the text.
Embedded documents are entire MongoDB documents that are used as the value for a key in another document. They can be used to organize data in a more natural way than just a flat structure.
For example, if we have a document representing a person and want
to store his address, we can nest this information in an embedded
"address"
document:
{ "name" : "John Doe", "address" : { "street" : "123 Park Street", "city" : "Anytown", "state" : "NY" } }
The value for the "address"
key in the previous
example is another document with its own values for
"street"
, "city"
, and
"state"
.
As with arrays, MongoDB “understands” the structure of embedded documents and is able to “reach inside” of them to build indexes, perform queries, or make updates.
We’ll discuss schema design in depth later, but even from this basic example, we can begin to see how embedded documents can change the way we work with data. In a relational database, the previous document would probably be modeled as two separate rows in two different tables (one for “people” and one for “addresses”). With MongoDB we can embed the address document directly within the person document. When used properly, embedded documents can provide a more natural (and often more efficient) representation of information.
The flip side of this is that we are basically denormalizing, so there can be more data repetition with MongoDB. Suppose “addresses” were a separate table in a relational database and we needed to fix a typo in an address. When we did a join with “people” and “addresses,” we’d get the updated address for everyone who shares it. With MongoDB, we’d need to fix the typo in each person’s document.
Every document stored in MongoDB must have an
"_id"
key. The "_id"
key’s value
can be any type, but it defaults to an ObjectId
.
In a single collection, every document must have a unique value for
"_id"
, which ensures that every document in a
collection can be uniquely identified. That is, if you had two
collections, each one could have a document where the value for
"_id"
was 123. However, neither collection could
contain more than one document where "_id"
was
123.
ObjectId
is the default type for
"_id"
. It is designed to be lightweight, while
still being easy to generate in a globally unique way across disparate
machines. This is the main reason why MongoDB uses
ObjectId
s as opposed to something more
traditional, like an autoincrementing primary key: it is difficult and
time-consuming to synchronize autoincrementing primary keys across
multiple servers. Because MongoDB was designed from the beginning to
be a distributed database, dealing with many nodes is an important
consideration. The ObjectId
type, as we’ll see,
is easy to generate in a sharded environment.
ObjectId
s use 12 bytes of storage, which
gives them a string representation that is 24 hexadecimal digits: 2
digits for each byte. This causes them to appear larger than they are,
which makes some people nervous. It’s important to note that even
though an ObjectId
is often represented as a
giant hexadecimal string, the string is actually twice as long as the
data being stored.
If you create multiple new ObjectId
s in
rapid succession, you can see that only the last few digits change
each time. In addition, a couple of digits in the middle of the
ObjectId
will change (if
you space the creations out by a couple of seconds). This is because
of the manner in which ObjectId
s are created.
The 12 bytes of an ObjectId
are generated as
follows:
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 |
Timestamp | Machine | PID | Increment |
The first four bytes of an ObjectId
are a
timestamp in seconds since the epoch. This provides a couple of useful
properties:
The timestamp, when combined with the next five bytes (which will be described in a moment), provides uniqueness at the granularity of a second.
Because the timestamp comes first, it means that
ObjectId
s will sort in
roughly insertion order. This is not a strong
guarantee but does have some nice properties, such as making
ObjectId
s efficient to index.
In these four bytes exists an implicit timestamp of when
each document was created. Most drivers expose a method for
extracting this information from an ObjectId
.
Because the current time is used in
ObjectId
s, some users worry that their servers
will need to have synchronized clocks. This is not necessary because
the actual value of the timestamp doesn’t matter, only that it is
often new (once per second) and increasing.
The next three bytes of an ObjectId
are a
unique identifier of the machine on which it was generated. This is
usually a hash of the machine’s hostname. By including these bytes, we
guarantee that different machines will not generate colliding
ObjectId
s.
To provide uniqueness among different processes generating
ObjectId
s concurrently on a single machine, the
next two bytes are taken from the process identifier (PID) of the
ObjectId
-generating process.
These first nine bytes of an ObjectId
guarantee its uniqueness across machines and processes for a single
second. The last three bytes are simply an incrementing counter that
is responsible for uniqueness within a second in a single process.
This allows for up to 2563 (16,777,216)
unique ObjectId
s to be generated per
process in a single second.
As stated previously, if there is no "_id"
key present when a document is inserted, one will be automatically
added to the inserted document. This can be handled by the MongoDB
server but will generally be done by the driver on the client side.
There are a couple of reasons for that:
Although ObjectId
s are designed to be
lightweight and easy to generate, there is still some overhead
involved in their generation. The decision to generate them on the
client side reflects an overall philosophy of MongoDB: work should
be pushed out of the server and to the drivers whenever possible.
This philosophy reflects the fact that, even with scalable
databases like MongoDB, it is easier to scale out at the
application layer than at the database layer. Moving work to the
client side reduces the burden requiring the database to
scale.
By generating ObjectId
s on the client
side, drivers are capable of providing richer APIs than would be
otherwise possible. For example, a driver might have its
insert
method either return the generated
ObjectId
or inject it directly into the
document that was inserted. If the driver allowed the server to
generate ObjectId
s,
then a separate query would be required to determine the value of
"_id"
for an inserted document.
18.227.111.208