In Chapter 3, you learned how the database works on the backend, what indexes are, how to use a database to quickly find the data you are looking for, and what the structure of a document looks like. You also saw a brief example that illustrated how to add data and find it again using the MongoDB shell. In this chapter, we will focus more on working with data from your shell.

We will use one database (named library) throughout this chapter, and we will perform actions such as adding data, searching data, modifying data, deleting data, and creating indexes. We’ll also look at how to navigate the database using various commands, as well as what DBRef is and what it does. If you have followed the instructions in the previous chapters to set up the MongoDB software, you can follow the examples in this chapter to get used to the interface. Along the way, you will also attain a solid understanding of which commands can be used for what kind of operations.

Navigating Your Databases

The first thing you need to know is how to navigate your databases and collections. With traditional SQL databases, the first thing you would need to do is create an actual database; however, as you probably remember from previous chapters, this is not required with MongoDB because the program creates the database and underlying collection for you automatically the moment you store data in it.

To switch to an existing database or create a new one, you can use the use function in the shell, followed by the name of the database you would like to use, whether or not it exists. This snippet shows how to use the library database:

> use library
Switched to db library

The mere act of invoking the use function, followed by the database’s name, sets your db (database) global variable to library. Doing this means that all the commands you pass down into the shell will automatically assume they need to be executed on the library database until you reset this variable to another database.

Viewing Available Databases and Collections

MongoDB automatically assumes a database needs to be created the moment you save data to it. It is also case sensitive. For these reasons, it can be quite tricky to ensure that you’re working in the correct database. Therefore, it’s best to view a list of all current databases available to MongoDB prior to switching to one, in case you forgot the database’s name or its exact spelling. You can do this using the show dbs function:

> show dbs
local  0.000GB

Note that this function will only show a database that already exists. At this stage, the database does not contain any data yet, so nothing else will be listed. If you want to view all available collections for your current database, you can use the show collections function:

> show collections
>

Inserting Data into Collections

One of the most frequently used pieces of functionality you will want to learn about is how to insert data into your collection. All data are stored in BSON format (which is both compact and reasonably fast to scan), so you will need to insert the data in BSON format as well. You can do this in several ways. For example, you can define it first and then save it in the collection using the insertOne function, or you can type the document while using the insert function on the fly:

> document = ({"Type": "Book", "Title" : "Definitive Guide to MongoDB 3rd ed., The", "ISBN" : "978-1-4842-1183-0", "Publisher" : "Apress", "Author" : ["Hows, David", "Plugge, Eelco", "Membrey, Peter", "Hawkins, Tim"] } )

> db.media.insertOne(document)
WriteResult({ "nInserted" : 1 })

Notice the WriteResult() output returned after inserting a document into the collection. WriteResult() will carry the status of the operation, as well as the action performed. When inserting a document, the nInserted property is returned, together with the number of documents inserted.

Line breaks can also be used while typing in the shell. This can be convenient if you are writing a rather lengthy document, as in this example:

> document = ( { "Type" : "Book",
..."Title" : "Definitive Guide to MongoDB 3rd ed., The",
..."ISBN" : " 978-1-4842-1183-0",
..."Publisher" : "Apress",
..."Author" : ["Hows, David", Plugge, Eelco", "Membrey, Peter"," "Hawkins, Tim"]
...} )

> db.media.insertOne(document)
WriteResult({ "nInserted" : 1 })

As mentioned previously, the other option is to insert your data directly through the shell, without defining the document first. You can do this by invoking the insert function immediately, followed by the document’s contents:

> db.media.insertOne( { "Type" : "CD", "Artist" : "Nirvana", "Title" : "Nevermind" })
WriteResult({ "nInserted" : 1 })

Or you can insert the data while using line breaks, as before. For example, you can expand the preceding example by adding an array of tracks to it. Pay close attention to how the commas and brackets are used in the following example:

> db.media.insertOne( { "Type" : "CD",
..."Artist" : "Nirvana",
..."Title" : "Nevermind",
... "Tracklist" : [
... {
... "Track" : "1",
... "Title" : "Smells Like Teen Spirit",
... "Length" : "5:02"
... },
... {
... "Track" : "2",
... "Title" : "In Bloom",
... "Length" : "4:15"
... }
... ]
...}
... )
WriteResult({ "nInserted" : 1 })

As you can see, inserting data through the Mongo shell is straightforward.

The process of inserting data is extremely flexible, but you must adhere to some rules when doing so. For example, the names of the keys while inserting documents have the following limitations:

• The $ character must not be the first character in the key name. Example: $tags
• The period [.] character must not appear anywhere in the key name. Example: ta.gs
• The name _id is reserved for use as a primary key ID; although it is not recommended, it can store anything unique as a value, such as a string or an integer.

Similarly, some restrictions apply when  creating a collection. For example, the name of a collection must adhere to the following rules:

• The collection’s namespace (including the database name and a “.” separator) cannot exceed 120 characters.
• An empty string (“ ”) cannot be used as a collection name.
• The collection’s name must start with either a letter or an underscore.
• The collection name system is reserved for MongoDB and cannot be used.
• The collection’s name cannot contain the “” null character.

Querying for Data

You’ve seen how to switch to your database and how to insert data; next, you will learn how to query for data in your collection. Let’s build on the preceding example and look at all the possible ways to get a good clear view of your data in a given collection.

The find() function provides the easiest way to retrieve data from multiple documents within one of your collections. This function is one that you will be using often.

Let’s assume that you have inserted the preceding two examples into a collection called media in the library database. If you were to use a simple find() function on this collection, you would get all of the documents you’ve added so far printed out for you:

> db.media.find()
{ "_id" : "ObjectId("4c1a8a56c603000000007ecb"), "Type" : "Book", "Title" :
"Definitive Guide to MongoDB 3rd ed., The", "ISBN" : "978-1-4842-1183-0", "Publisher" :
"Apress", "Author" : ["Hows, David ", "Plugge, Eelco", "Membrey, Peter", "Hawkins, Tim"]}

{ "_id" : "ObjectId("4c1a86bb2955000000004076"), "Type" : "CD", "Artist" :
"Nirvana", "Title" : "Nevermind", "Tracklist" : [
    {
        "Track" : "1",
            "Title" : "Smells Like Teen Spirit",
            "Length" : "5:02"
    },
    {
        "Track" : "2",
        "Title" : "In Bloom",
        "Length" : "4:15"
    }
] }

This is simple stuff, but typically you would not want to retrieve all the information from all the documents in your collection. Instead, you probably want to retrieve a certain type of document. For example, you might want to return all the CDs from Nirvana. If so, you can specify that only the desired information is requested and returned:

> db.media.find ( { Artist : "Nirvana" } )
{ "_id" : "ObjectId("4c1a86bb2955000000004076"), "Type" : "CD", "Artist" :
 "Nirvana", "Title" : "Nevermind", "Tracklist" : [
    {
        "Track" : "1",
        "Title" : "Smells Like Teen Spirit",
        "Length" : "5:02"
    },
    {
        "Track" : "2",
        "Title" : "In Bloom",
        "Length" : "4:15"
    }
] }

Okay, so this looks much better! You don’t have to see all the information from all the other items you’ve added to your collection, only the information that interests you. However, what if you’re still not satisfied with the results returned? For example, assume you want to get a list back that shows only the titles of the CDs you have by Nirvana, ignoring any other information, such as track lists. You can do this by inserting an additional parameter into your query that specifies the name of the keys you want to return, followed by a 1:

> db.media.find ( {Artist : "Nirvana"}, {Title: 1} )
{ "_id" : ObjectId("4c1a86bb2955000000004076"), "Title" : "Nevermind" }

Inserting the { Title : 1 } information specifies that only the information from the title field should be returned. The _id field is always returned, unless you specifically exclude it using { _id: 0 }.

You can also accomplish the opposite: inserting { Type : 0 } retrieves a list of all items you have stored from Nirvana, showing all information except for the Type field.

Take a moment to run the revised query with the { Title : 1 } insertion; no unnecessary information is returned at all. This saves you time because you see only the information you want. It also spares your database the time required to return unnecessary information.

Using the Dot Notation

When you start working with more complex document structures such as documents containing arrays or embedded objects, you can begin using other methods for querying information from those objects as well. For example, assume you want to find all CDs that contain a specific song you like. The following code executes a more detailed query:

> db.media.find( { "Tracklist.Title" : "In Bloom" } )
{ "_id" : "ObjectId("4c1a86bb2955000000004076"), "Type" : "CD", "Artist" :
"Nirvana", "Title" : "Nevermind", "Tracklist" : [
    {
        "Track" : "1",
        "Title" : "Smells Like Teen Spirit",
        "Length" : "5:02"
    },
    {
        "Track" : "2",
        "Title" : "In Bloom",
        "Length" : "4:15"
    }
] }

Using a period [.] after the key’s name tells your find function to look for information embedded in your documents. Things are a little simpler when working with arrays. For example, you can execute the following query if you want to find a list of books written by Peter Membrey:

> db.media.find( { "Author" : "Membrey, Peter" } )
{ "_id" : "ObjectId("4c1a8a56c603000000007ecb"), "Type" : "Book", "Title" :
"Definitive Guide to MongoDB 3rd ed., The", "ISBN" : "978-1-4842-1183-0", "Publisher" :
"Apress", "Author" : ["Hows, David ", "Plugge, Eelco", "Membrey, Peter", "Hawkins, Tim"] }

However, the following command will not match any documents, even though it might appear identical to the earlier track list query:

> db.media.find ( { "Tracklist" : {"Track" : "1" }} )

Subobjects must match exactly; therefore, the preceding query would only match a document that contains no other information, such as Track.Title:

{"Type" : "CD",
"Artist" : "Nirvana"
"Title" : "Nevermind",
"Tracklist" : [
    {
        "Track" : "1",
    },
    {
        "Track" : "2",
        "Title" : "In Bloom",
        "Length" : "4:15"
    }
]
}

Using the Sort, Limit, and Skip Functions

MongoDB includes several functions that you can use for more precise control over your queries. We’ll cover how to use the sort, limit, and skip functions in this section.

You can use the sort function to sort the results returned from a query. You can sort the results in ascending or descending order using 1 or -1, respectively. The function itself is analogous to the ORDER BY statement in SQL, and it uses the key’s name and sorting method as criteria, as in this example:

> db.media.find().sort( { Title: 1 })

This example sorts the results based on the Title key’s value in ascending order. This is the default sorting order when no parameters are specified. You would add the -1 flag to sort in descending order.

You can use the limit() function to specify the maximum number of results returned. This function requires only one parameter: the number of the desired results returned. When you specify 0, all results will be returned. The following example returns only ten items in your media collection:

> db.media.find().limit( 10 )

Another thing you might want to do is skip the first n documents in a collection. The following example skips 20 documents in your media collection:

> db.media.find().skip( 20 )

As you probably surmised, this command returns all documents within your collection, except for the first 20 it finds.

MongoDB wouldn’t be particularly powerful if it weren’t able to combine these commands. However, practically any function can be combined and used in conjunction with any other function. The following example limits the results by skipping a few and then sorts the results in descending order:

> db.media.find().sort ( { Title : -1 } ).limit ( 10 ).skip ( 20 )

You might use this example if you want to implement paging in your application. As you might have guessed, this command wouldn’t return any results in the media collection created so far, because the collection contains fewer documents than were skipped in this example.

Working with Capped Collections, Natural Order, and $natural

There are some additional concepts and features you should be aware of when sorting queries with MongoDB, including capped collections, natural order, and $natural. We’ll explain in this section what all of these terms mean and how you can leverage them in your sorts.

The natural order is the database’s native ordering method for objects within a (normal) collection. When you query for items in a collection without specifying an explicit sort order, the items are returned by default in forward natural order. This may initially appear identical to the order in which items were inserted; however, the natural order for a normal collection is not defined and may vary depending on document growth patterns, indexes used for a query, and the storage engine used.

A capped collection is a collection in your database where the natural order is guaranteed to be the order in which the documents were inserted. Guaranteeing that the natural order will always match the insertion order can be particularly useful when you’re querying data and need to be absolutely certain that the results returned are already sorted based on their order of insertion.

Capped collections have another great benefit: they are a fixed size. Once a capped collection is full, the oldest data will be purged and newer data will be added at the end, ensuring that the natural order follows the order in which the records were inserted. This type of collection can be used for logging and autoarchiving data.

Unlike a standard collection, a capped collection must be created explicitly, using the createCollection function. You must also supply parameters that specify the size (in bytes) of the collection you want to add. For example, imagine you want to create a capped collection named audit with a maximum size of 20480 bytes:

> db.createCollection("audit", {capped:true, size:20480})
{ "ok" : 1 }

Given that a capped collection guarantees that the natural order matches the insertion order, you don’t need to include any special parameters or any other special commands or functions when querying the data either, except of course when you want to reverse the default results. This is where the $natural parameter comes in. For example, assume you want to find the ten most recent entries from your capped collection that lists failed login attempts. You could use the $natural parameter to find this information:

> db.audit.find().sort( { $natural: -1 } ).limit ( 10 )

You can also limit the number of items added into a capped collection using the max: parameter when you create the collection. However, you must ensure that there is enough space in the collection for the number of items you want to add. If the collection becomes full before the number of items has been reached, the oldest item in the collection will be removed. The MongoDB shell includes a utility that lets you see the amount of space used by an existing collection, whether it’s capped or uncapped. You invoke this utility using the validate() function. This can be particularly useful if you want to estimate how large a collection might become.

As stated previously, you can use the max: parameter to cap the number of items that can be inserted into a collection, as in this example:

> db.createCollection("audit100", { capped:true, size:20480, max: 100})
{ "ok" : 1 }

Next, use the stats() function to check the size of the collection:

> db.audit100.stats()
{
   "ns" : "library.audit100",
   "count" : 0,
   "size" : 0,
   "storageSize" : 4096,
   "capped" : true,
   "max" : 100,
   "maxSize" : 20480,
   "sleepCount" : 0,
   "sleepMS" : 0,
   "wiredTiger" : {
      [..]
   },
   "nindexes" : 1,
   "totalIndexSize" : 4096,
   "indexSizes" : {
      "_id_" : 4096
   },
   "ok" : 1
}

The resulting output shows that the table (named audit100) is a capped collection with a maximum of 100 items to be added, and it currently contains zero items.

Retrieving a Single Document

So far we’ve only looked at examples that show how to retrieve multiple documents. If you want to receive only one result, however, querying for all documents—which is what you generally do when executing a find() function—would be a waste of CPU time and memory. For this case, you can use the findOne() function to retrieve a single item from your collection. Overall, the result and execution methods are identical to what occurs when you append the limit(1) function, but why make it harder on yourself than you should?

The syntax of the findOne() function is identical to the syntax of the find() function:

> db.media.findOne()

It’s generally advised to use the findOne() function if you expect only one result.

Using the Aggregation Commands

MongoDB comes with a nice set of aggregation commands. You might not see their significance at first, but once you get the hang of using them, you will see that the aggregation commands form an extremely powerful set of tools. For instance, you might use them to get an overview of some basic statistics about your database. In this section, we will take a closer look at how to use three of the functions from the available aggregate commands: count, distinct, and group.

In addition to these three basic aggregation commands, MongoDB also includes an aggregation framework. This powerful feature will allow you to calculate aggregated values without needing to use the map/reduce framework. The aggregation framework will be discussed in Chapter 5.

Returning the Number of Documents with count( )

The count() function returns the number of documents in the specified collection. So far you’ve added a number of documents in the media collection. The count() function can tell you exactly how many:

> db.media.count()
2

You can also perform additional filtering by combining count() with conditional operators, as shown here:

> db.media.find( { Publisher : "Apress", Type: "Book" } ).count()
1

This example returns only the number of documents added in the collection that are published by Apress and of the type Book. Note that the count() function ignores a skip() or limit() parameter by default. To ensure that your query doesn’t skip these parameters and that your count results will match the limit and/or skip parameters, use count(true):

> db.media.find( { Publisher: "Apress", Type: "Book" }).skip ( 2 ) .count (true)
0

Retrieving Unique Values with distinct( )

The preceding example shows a great way to retrieve the total number of documents from a specific publisher. However, this approach is definitely not precise. After all, if you own more than one book with the same title (for instance, the hardcopy and the e-book), then you would technically have just one book. This is where distinct() can help you: it will only return unique values.

For the sake of completeness, you can add an additional item to the collection. This item carries the same title, but has a different ISBN number:

> document = ( { "Type" : "Book", "Title" : "Definitive Guide to MongoDB 3rd ed., The", ISBN:
" 978-1-4842-1183-1", "Publisher" : "Apress", "Author" :
["Hows, David", "Membrey, Peter", "Plugge, Eelco", "Hawkins, Tim"] } )
> db.media.insert (document)
WriteResult({ "nInserted" : 1 })

At this point, you should have two books in the database with identical titles. When using the distinct() function on the titles in this collection, you will get a total of two unique items. However, the titles of the two books are unique, so they will be grouped into one item. The other result will be the title of the album “Nevermind”:

> db.media.distinct( "Title")
[ "Definitive Guide to MongoDB 3rd ed., The", "Nevermind" ]

Similarly, you will get two results if you query for a list of unique ISBN numbers:

> db.media.distinct ("ISBN")
[ "978-1-4842-1183-0", " 978-1-4842-1183-1" ]

The distinct() function also takes nested keys when querying; for instance, this command will give you a list of unique titles of your CDs:

> db.media.distinct ("Tracklist.Title")
[ "In Bloom", "Smells Like Teen Spirit" ]

Grouping Your Results

Last but not least, you can group your results. MongoDB’s group() function is similar to SQL’s GROUP BY function, although the syntax is a little different. The purpose of the command is to return an array of grouped items. The group() function takes three parameters: key, initial, and reduce.

The key parameter specifies which results you want to group. For example, assume you want to group results by Title. The initial parameter lets you provide a base for each grouped result (that is, the base number of items to start off with). By default, you want to leave this parameter at zero if you want an exact number returned. The reduce parameter groups all similar items together. Reduce takes two arguments: the current document being iterated over and the aggregation counter object. These arguments are called items and prev in the example that follows. Essentially, the reduce parameter adds a 1 to the sum of every item it encounters that matches a title it has already found.

The group() function is ideal when you’re looking for a tagcloud kind of function. For example, assume you want to obtain a list of all unique titles of any type of item in your collection. Additionally, assume you want to group them together if any doubles are found, based on the title:

> db.media.group (
{
    key: {Title : true},
    initial: {Total : 0},
    reduce : function (items,prev)
    {
        prev.Total += 1
    }
}
)

[
    {
        "Title" : "Nevermind",
        "Total" : 1
    },
    {
        "Title" : "Definitive Guide to MongoDB 3rd ed., The",
        "Total" : 2
    }
]

In addition to the key, initial, and reduce parameters, you can specify three more optional parameters:

• keyf: You can use this parameter to replace the key parameter if you do not wish to group the results on an existing key in your documents. Instead, you would group them using another function you design that specifies how to do grouping.
• cond: You can use this parameter to specify an additional statement that must be true before a document will be grouped. You can use this much as you use the find() query to search for documents in your collection. If this parameter isn’t set (the default), then all documents in the collection will be checked.
• finalize: You can use this parameter to specify a function you want to execute before the final results are returned. For instance, you might calculate an average or perform a count and include this information in the results.

Working with Conditional Operators

MongoDB supports a large set of conditional operators to better filter your results. The following sections provide an overview of these operators, including some basic examples that show you how to use them. Before walking through these examples, however, you should add a few more items to the database; doing so will let you see the effects of these operators more plainly:

> dvd = ( { "Type" : "DVD", "Title" : "Matrix, The", "Released" : 1999,
    "Cast" : ["Keanu Reeves","Carrie-Anne Moss","Laurence Fishburne","Hugo
     Weaving","Gloria Foster","Joe Pantoliano"] } )
{
        "Type" : "DVD",
        "Title" : "Matrix, The",
        "Released" : 1999,
        "Cast" : [
                "Keanu Reeves",
                "Carrie-Anne Moss",
                "Laurence Fishburne",
                "Hugo Weaving",
                "Gloria Foster",
                "Joe Pantoliano"
        ]
}
> db.media.insertOne(dvd)

> dvd = ( { "Type" : "DVD", Title : "Blade Runner", Released : 1982 } )
{ "Type" : "DVD", "Title" : "Blade Runner", "Released" : 1982 }
> db.media.insertOne(dvd)

> dvd = ( { "Type" : "DVD", Title : "Toy Story 3", Released : 2010 } )
{ "Type" : "DVD", "Title" : "Toy Story 3", "Released" : 2010 }
> db.media.insertOne(dvd)

Performing Greater-Than and Less-Than Comparisons

You can use the following special parameters to perform greater-than and less-than comparisons in queries: $gt, $lt, $gte, and $lte. In this section, we’ll look at how to use each of these parameters.

The first one we’ll cover is the $gt (greater-than) parameter. You can use this to specify that a certain integer should be greater than a specified value in order to be returned:

> db.media.find ( { Released : {$gt : 2000} }, { "Cast" : 0 } )
{ "_id" : ObjectId("4c4369a3c603000000007ed3"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

Note that the year 2000 itself will not be included in the preceding query. For that, you use the $gte (greater-than or equal-to) parameter:

> db.media.find ( { Released : {$gte : 1999 } }, { "Cast" : 0 } )
{ "_id" : ObjectId("4c43694bc603000000007ed1"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999 }
{ "_id" : ObjectId("4c4369a3c603000000007ed3"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

Likewise, you can use the $lt (less-than) parameter to find items in your collection that predate the year 1999:

> db.media.find ( { Released : {$lt : 1999 } }, { "Cast" : 0 } )
{ "_id" : ObjectId("4c436969c603000000007ed2"), "Type" : "DVD", "Title" : "Blade Runner", "Released" : 1982 }

You can also get a list of items older than or equal to the year 1999 by using the $lte (less-than or equal-to) parameter:

> db.media.find( {Released : {$lte: 1999}}, { "Cast" : 0 })
{ "_id" : ObjectId("4c43694bc603000000007ed1"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999 }
{ "_id" : ObjectId("4c436969c603000000007ed2"), "Type" : "DVD", "Title" :
"Blade Runner", "Released" : 1982 }

You can also combine these parameters to specify a range:

> db.media.find( {Released : {$gte: 1990, $lt : 2010}}, { "Cast" : 0 })
{ "_id" : ObjectId("4c43694bc603000000007ed1"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999 }

These parameters might strike you as relatively simple to use; however, you will be using them a lot when querying for a specific range of data.

Retrieving All Documents but Those Specified

You can use the $ne (not-equals) parameter to retrieve every document in your collection, except for the ones that match certain criteria. It should be noted that $ne may be performance heavy when the field of choice has many potential values. For example, you can use this snippet to obtain a list of all books where the author is not Eelco Plugge:

> db.media.find( { Type : "Book", Author: {$ne : "Plugge, Eelco"}})

Specifying an Array of Matches

You can use the $in operator to specify an array of possible matches. The SQL equivalent is the IN operator.

You can use the following snippet to retrieve data from the media collection using the $in operator:

> db.media.find( {Released : {$in : [1999,2008,2009] } }, { "Cast" : 0 } )
{ "_id" : ObjectId("4c43694bc603000000007ed1"), "Type" : "DVD", "Title" : "Matrix, The", "Released" : 1999 }

This example returns only one item, because only one item matches the release year of 1999, and there are no matches for the years 2008 and 2009.

Finding a Value Not in an Array

The $nin operator functions similarly to the $in operator, except that it searches for the objects where the specified field does not have a value in the specified array:

> db.media.find( {Released : {$nin : [1999,2008,2009] },Type : "DVD" },
{ "Cast" : 0 } )
{ "_id" : ObjectId("4c436969c603000000007ed2"), "Type" : "DVD", "Title" :
"Blade Runner", "Released" : 1982 }
{ "_id" : ObjectId("4c4369a3c603000000007ed3"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

Matching All Attributes in a Document

The $all operator also works similarly to the $in operator. However, $all requires that all attributes match in the documents, whereas only one attribute must match for the $in operator. Let’s look at an example that illustrates these differences. First, here’s an example that uses $in:

> db.media.find ( { Released : {$in : ["2010","2009"] } }, { "Cast" : 0 } )
{ "_id" : ObjectId("4c4369a3c603000000007ed3"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

One document is returned for the $in operator because there’s a match for 2010, but not for 2009. However, the $all parameter doesn’t return any results, because there are no matching documents with 2009 in the value:

> db.media.find ( { Released : {$all : ["2010","2009"] } }, { "Cast" : 0 } )

Searching for Multiple Expressions in a Document

You can use the $or operator to search for multiple expressions in a single query, where only one criterion needs to match to return a given document. Unlike the $in operator, $or allows you to specify both the key and the value, rather than only the value:

> db.media.find({ $or : [ { "Title" : "Toy Story 3" }, { "ISBN" :"978-1-4842-1183-0" } ] } )
{ "_id" : ObjectId("4c5fc7d8db290000000067c5"), "Type" : "Book", "Title" :
"Definitive Guide to MongoDB 3rd ed., The", "ISBN" : "978-1-4842-1183-0",
"Publisher" : "Apress", "Author" : ["Hows, David", "Membrey, Peter", "Plugge, Eelco",
"Hawkins, Tim" ] }
{ "_id" : ObjectId("4c5fc943db290000000067ca"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

It’s also possible to combine the $or operator with another query parameter. This will restrict the returned documents to only those that match the first query (mandatory), and then either of the two key/value pairs specified at the $or operator, as in this example:

> db.media.find({ "Type" : "DVD", $or : [ { "Title" : "Toy Story 3" }, {
"ISBN" : "978-1-4842-1183-0" } ] })
{ "_id" : ObjectId("4c5fc943db290000000067ca"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

You could say that the $or operator allows you to perform two queries at the same time, combining the results of two otherwise unrelated queries on the same collection. It is worth noting here that, if all the queries in an $or clause can be supported by indexes, MongoDB will perform index scans. If not, a collection scan will be used instead. Lastly, each clause of the $or can use its own index.

Retrieving a Document with $slice

You can use the $slice projection to limit an array field to a subset of the array for each matching result. This can be particularly useful if you want to limit a certain set of items added to save bandwidth. The operator also lets you retrieve the results of n items per page, a feature generally known as paging.

The operator takes two parameters; the first indicates the total number of items to be returned. The second parameter is optional; if used, it ensures that the first parameter defines the offset, while the second defines the limit. The $slice limit parameter also accepts a negative value to return items starting from the end of an array instead of the beginning.

The following example limits the items from the Cast list to the first three items:

> db.media.find({"Title" : "Matrix, The"}, {"Cast" : {$slice: 3}})
{ "_id" : ObjectId("4c5fcd3edb290000000067cb"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999, "Cast" : [ "Keanu Reeves", "Carrie-Anne
Moss", "Laurence Fishburne" ] }

You can also get only the last three items by making the integer negative:

> db.media.find({"Title" : "Matrix, The"}, {"Cast" : {$slice: -3}})
{ "_id" : ObjectId("4c5fcd3edb290000000067cb"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999, "Cast" : [ "Hugo Weaving", "Gloria Foster",
"Joe Pantoliano" ] }

Or you can skip the first two items and limit the results to three from that particular point (pay careful attention to the brackets):

> db.media.find({"Title" : "Matrix, The"}, {"Cast" : {$slice: [2,3] }})
{ "_id" : ObjectId("4c5fcd3edb290000000067cb"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999, "Cast" : [ "Laurence Fishburne", "Hugo
Weaving", "Gloria Foster" ] }

Finally, when specifying a negative integer, you can skip to the last five items and limit the results to four, as in this example:

> db.media.find({"Title" : "Matrix, The"}, {"Cast" : {$slice: [-5,4] }})
{ "_id" : ObjectId("4c5fcd3edb290000000067cb"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999, "Cast" : [ "Carrie-Anne Moss","Laurence
Fishburne","Hugo Weaving","Gloria Foster"] }

Searching for Odd/Even Integers

The $mod operator lets you search for specific data that consists of an even or uneven number. This works because the operator takes the modulus of 2 and checks for a remainder of 0, thereby providing even-numbered results only.

For example, the following code returns any item in the collection that has an even-numbered integer set to its Released field:

> db.media.find ( { Released : { $mod: [2,0] } }, {"Cast" : 0 } )
{ "_id" : ObjectId("4c45b5c18e0f0000000062aa"), "Type" : "DVD", "Title" :
"Blade Runner", "Released" : 1982 }
{ "_id" : ObjectId("4c45b5df8e0f0000000062ab"), "Type" : "DVD", "Title" :
"Toy Story 3", "Released" : 2010 }

Likewise, you can find any documents containing an uneven value in the Released field by changing the parameters in $mod, as follows:

> db.media.find ( { Released : { $mod: [2,1] } }, { "Cast" : 0 } )
{ "_id" : ObjectId("4c45b5b38e0f0000000062a9"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999 }

Filtering Results with $size

The $size operator lets you filter your results to match an array with the specified number of elements in it. For example, you might use this operator to do a search for those CDs that have exactly two songs on them:

> db.media.find ( { Tracklist : {$size : 2} } )
{ "_id" : ObjectId("4c1a86bb2955000000004076"), "Type" : "CD", "Artist" :
"Nirvana", "Title" : "Nevermind", "Tracklist" : [
        {
                "Track" : "1",
                "Title" : "Smells Like Teen Spirit",
                "Length" : "5:02"
        },
        {
                "Track" : "2",
                "Title" : "In Bloom",
                "Length" : "4:15"
        }
] }

Returning a Specific Field Object

The $exists operator allows you to return a specific object if a specified field is either missing or found. The following example returns all items in the collection with a key named Author:

> db.media.find ( { Author : {$exists : true } } )

Similarly, if you invoke this operator with a value of false, then all documents that don’t have a key named Author will be returned:

> db.media.find ( { Author : {$exists : false } } )

Matching Results Based on the BSON Type

The $type operator lets you match results based on their BSON type. For instance, the following snippet lets you find all items that have a track list of the type Embedded Object (that is, it contains a list of information):

> db.media.find ( {  Tracklist: { $type : 3 } } )
{ "_id" : ObjectId("4c1a86bb2955000000004076"), "Type" : "CD", "Artist" :
"Nirvana", "Title" : "Nevermind", "Tracklist" : [
        {
                "Track" : "1",
                "Title" : "Smells Like Teen Spirit",
                "Length" : "5:02"
        },
        {
                "Track" : "2",
                "Title" : "In Bloom",
                "Length" : "4:15"
        }
] }

The known data types are defined in Table 4-1.

Matching an Entire Array

If you want to match an entire array within a document, you can use the $elemMatch operator. This is particularly useful if you have multiple documents within your collection, some of which have some of the same information. This can make a default query incapable of finding the exact document you are looking for. This is because the standard query syntax doesn’t restrict itself to a single document within an array.

Let’s look at an example that illustrates this principle. For this to work, you need to add another document to the collection, one that has an identical item in it but is otherwise different. Specifically, let’s add another CD from Nirvana that happens to have the same track on it as the aforementioned CD (“Smells Like Teen Spirit”). However, on this version of the CD, the song is track 5, not track 1:

{
        "Type" : "CD",
        "Artist" : "Nirvana",
        "Title" : "Nirvana",
        "Tracklist" : [
                {
                        "Track" : "1",
                        "Title" : "You Know You're Right",
                        "Length" : "3:38"
                },
                {
                       "Track" : "5",
                       "Title" : "Smells Like Teen Spirit",
                       "Length" : "5:02"
                }
        ]
}

> nirvana = ( { "Type" : "CD", "Artist" : "Nirvana", "Title" : "Nirvana",
"Tracklist" : [ { "Track" : "1", "Title" : "You Know You're Right", "Length"
: "3:38"}, {"Track" : "5", "Title" : "Smells Like Teen Spirit", "Length" :
"5:02" } ] } )

> db.media.insertOne(nirvana)

If you want to search for an album from Nirvana that has the song “Smells Like Teen Spirit” as Track 1 on the CD, you might think that the following query would do the job:

> db.media.find ( { "Tracklist.Title" : "Smells Like Teen Spirit","Tracklist.Track" : "1" } )

Unfortunately, the preceding query will return both documents. The reason for this is that both documents have a track with the title called “Smells Like Teen Spirit” and both have a track number 1. If you want to match an entire document within the array, you can use $elemMatch, as in this example:

> db.media.find ( { Tracklist: { "$elemMatch" : { Title:"Smells Like Teen Spirit", Track : "1" } } } )

{ "_id" : ObjectId("4c1a86bb2955000000004076"), "Type" : "CD", "Artist" :
"Nirvana", "Title" : "Nevermind", "Tracklist" : [
        {
                "Track" : "1",
                "Title" : "Smells Like Teen Spirit",
                "Length" : "5:02"
        },
        {
                "Track" : "2",
                "Title" : "In Bloom",
                "Length" : "4:15"
        }
] }

This query gave the desired result and only returned the first document.

Using the $not Metaoperator

You can use the $not metaoperator to negate any check performed by a standard operator. It should be noted that $not may be performance heavy when the field of choice has many potential values. The following example returns all documents in your collection, except for the one seen in the $elemMatch example:

> db.media.find ( { Tracklist : { $not : { "$elemMatch" : { Title:
"Smells Like Teen Spirit", "Track" : "1" } } } } )

Specifying Additional Query Expressions

Apart from the structured query syntax you’ve seen so far, you can also specify additional query expressions in JavaScript. The big advantage of this is that JavaScript is extremely flexible and allows you to do tons of additional things. The downside of using JavaScript is that it’s a tad slower than the native operators baked into MongoDB, as it cannot take advantage of indexes.

For example, assume you want to search for a DVD within your collection that is older than 1995. All of the following code examples would return this information:

db.media.find ( { "Type" : "DVD", "Released" : { $lt : 1995 } } )

db.media.find ( { "Type" : "DVD", $where: "this.Released < 1995" } )

db.media.find ("this.Released < 1995")

f = function() { return this.Released < 1995 }
db.media.find(f)

And that’s how flexible MongoDB is! Using these operators should enable you to find just about anything throughout your collections.

Leveraging Regular Expressions

Regular expressions are another powerful tool you can use to query information. Regular expressions—regex, for short—are special text strings that you can use to describe your search pattern. These work much like wildcards, but they are far more powerful and flexible.

MongoDB allows you to use these regular expressions when searching for data in your collections; however, to improve performance it will attempt to use an index whenever possible for simple prefix expressions. Prefix expressions are those regular expressions that start with either a left anchor (“A”) or a caret (“^”) followed by a few characters (example: “^Matrix”). Querying with regular expressions that are not prefix expressions cannot efficiently make use of an index.

The following example uses regex in a query to find all items in the media collection that start with the word “Matrix” (case insensitive):

> db.media.find ( { Title : /^Matrix/i } )

Using regular expressions from MongoDB can make your life much simpler, so we recommend exploring this feature in greater detail as time permits or your circumstances can benefit from it.

Updating Data

So far you’ve learned how to insert and query for data in your database. Next, you’ll learn how to update those data. MongoDB supports quite a few update operators that you’ll learn how to use in the following sections.

Updating with update()

MongoDB comes with the update() function for performing updates to your data. The update() function takes three primary arguments: criteria, objNew, and options.

The criteria argument lets you specify the query that selects the record you want to update. You use the objNew argument to specify the updated information; or you can use an operator to do this for you. The options argument lets you specify your options when updating the document, and it has two possible values: upsert and multi. The upsert option lets you specify whether the update should be an upsert—that is, it tells MongoDB to update the record if it exists and create it if it doesn’t. Finally, the multi option lets you specify whether all matching documents should be updated or just the first one (the default action).

The following simple example uses the update() function without any fancy operators:

> db.media.updateOne( { "Title" : "Matrix, The"}, {"Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999, "Genre" : "Action"}, { upsert: true} )

This example updates a matching document in the collection if one exists or saves a new document with the new values specified. Note that any fields you leave out are removed (the document is basically being rewritten).

In case there happens to be multiple documents matching the criteria and you wish to upsert them all, the updateMany function can be used instead of updateOne() while using the $set modifier operator, as shown here:

> db.media.updateMany( { "Title" : "Matrix, The"}, {$set: {"Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999, "Genre" : "Action"} }, {upsert: true} )

Implementing an Upsert with the save() Command

You can also perform an upsert with the save() command. To do this, you need to specify the _id value; you can have this value added automatically or specify it manually yourself. If you do not specify the _id value, the save() command will assume it’s an insert and simply add the document into your collection.

The main benefit of using the save() command is that you do not need to specify that the upsert method should be used in conjunction with the update() command. Thus, the save() command gives you a quicker way to upsert data. In practice, the save() and update() commands look similar:

> db.media.updateOne( { "Title" : "Matrix, The"}, {"Type" : "DVD", "Title" :"Matrix, The", "Released" : "1999", "Genre" : "Action"}, { upsert: true} )

> db.media.save( { "Title" : "Matrix, The"}, {"Type" : "DVD", "Title" :"Matrix, The", "Released" : "1999", "Genre" : "Action"})

Obviously, this example assumes that the Title value acts as the id field.

Updating Information Automatically

You can use the modifier operations to update information quickly and simply in your documents, without needing to type everything in manually. For example, you might use these operations to increase a number or to remove an element from an array.

We’ll be exploring these operators next, providing practical examples that show you how to use them.

Incrementing a Value with $inc

The $inc operator enables you to perform an (atomic) update on a key to increase the value by the given increment, assuming that the field exists. If the field doesn’t exist, it will be created. To see this in action, begin by adding another document to the collection:

> manga = ( { "Type" : "Manga", "Title" : "One Piece", "Volumes" : 612,"Read" : 520 } )
{
        "Type" : "Manga",
        "Title" : "One Piece",
        "Volumes" : "612",
        "Read" : "520"
}
> db.media.insertOne(manga)

Now you’re ready to update the document. For example, assume you’ve read another four volumes of the One Piece manga, and you want to increment the number of Read volumes in the document. The following example shows you how to do this:

> db.media.updateOne ( { "Title" : "One Piece"}, {$inc: {"Read" : 4} } )
> db.media.find ( { "Title" : "One Piece" } )
{
        "Type" : "Manga",
        "Title" : "One Piece ",
        "Volumes" : "612",
        "Read" : "524"
}

Setting a Field’s Value

You can use the $set operator to set a field’s value to one you specify. This works for any datatype, as in the following example:

> db.media.update ( { "Title" : "Matrix, The" }, {$set : { Genre :"Sci-Fi" } } )

This snippet would update the genre in the document created earlier, setting it to Sci-Fi instead.

Deleting a Specified Field

The $unset operator lets you delete a given field, as in this example:

> db.media.updateOne ( {"Title": "Matrix, The"}, {$unset : { "Genre" : 1 } } )

This snippet would delete the Genre key and its value from the document.

Appending a Value to a Specified Field

The $push operator allows you to append a value to a specified field. If the field is an existing array, then the value will be added. If the field doesn’t exist yet, then the field will be set to the array value. If the field exists but it isn’t an array, then an error condition will be raised.

Begin by adding another author to your entry in the collection:

> db.media.updateOne ( {"ISBN" : "978-1-4842-1183-0"}, {$push: { Author : "Griffin,
Stewie"} } )

The next snippet raises an error message because the Title field is not an array:

> db.media.updateOne ( {"ISBN" : "978-1-4842-1183-0"}, {$push: { Title :
"This isn't an array"} } )
Cannot apply $push/$pushAll modifier to non-array

The following example shows how the document looks in the meantime:

> db.media.find ( { "ISBN" : "978-1-4842-1183-0" } )
{
    "Author" :
    [
        "Hows, David",
        "Membrey, Peter",
        "Plugge, Eelco",
        "Griffin, Stewie",
    ],
    "ISBN" : "978-1-4302-5821-6",
    "Publisher" : "Apress",
    "Title" : "Definitive Guide to MongoDB 3rd ed., The",
    "Type" : "Book",
    "_id" : ObjectId("4c436231c603000000007ed0")
}

Specifying Multiple Values in an Array

When working with arrays, the $push operator will append the value specified to the given array, expanding the data stored within the given element. If you wish to add several separate values to the given array, you can use the optional $each modifier, as in this example:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0" }, { $push: { Author : { $each: ["Griffin, Peter", "Griffin, Brian"] } } } )
{
    "Author" :
    [
        "Hows, David",
        "Membrey, Peter",
        "Plugge, Eelco",
        "Hawkins, Tim",
        "Griffin, Stewie",
        "Griffin, Peter",
        "Griffin, Brian"
    ],
    "ISBN" : "978-1-4842-1183-0",
    "Publisher" : "Apress",
    "Title" : "Definitive Guide to MongoDB 3rd ed., The",
    "Type" : "Book",
    "_id" : ObjectId("4c436231c603000000007ed0")
}

Optionally, you can use the $slice operator when using $each. This allows you to limit the number of elements within an array during a $push operation. The $slice operator takes either a negative number or zero. Using a negative number ensures that only the last n elements will be kept within the array, whereas using zero would empty the array. Note that the $slice operator has to be the first modifier to the $push operator in order to function as such:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0" }, { $push: { Author : { $each: ["Griffin, Meg", "Griffin, Louis"], $slice: -2 } } } )
{
    "Author" :
    [
        "Griffin, Meg",
        "Griffin, Louis"
    ],
    "ISBN" : "978-1-4842-1183-0",
    "Publisher" : "Apress",
    "Title" : "Definitive Guide to MongoDB 3rd ed., The",
    "Type" : "Book",
    "_id" : ObjectId("4c436231c603000000007ed0")
}

As you can see, the $slice operator ensured that not only were the two new values pushed, but that the data kept within the array was also limited to the value specified (2). The $slice operator can be a valuable tool when working with fixed-sized arrays.

Adding Data to an Array with $addToSet

The $addToSet operator is another command that lets you add data to an array. However, this operator only adds the data to the array if the data are not already there. In this way, $addToSet is unlike $push. By default, the $addToSet operator takes one argument. However, you can use the $each operator to specify additional arguments when using t$addToSet. The following snippet adds the author Griffin, Brian into the authors array because it isn’t there yet:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0" }, {$addToSet : { Author :"Griffin, Brian" } } )

Executing the snippet again won’t change anything because the author is already in the array.

To add more than one value, however, you should take a different approach and use the $each operator as well:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0" }, {$addToSet : { Author :{ $each : ["Griffin, Brian","Griffin, Meg"] } } } )

At this point, our document, which once looked tidy and trustworthy, has been transformed into something like this:

{
    "Author" :
    [
        "Hows, David",
        "Membrey, Peter",
        "Plugge, Eelco",
        "Hawkins, Tim",
        "Griffin, Stewie",
        "Griffin, Peter",
        "Griffin, Brian",
        "Griffin, Louis",
        "Griffin, Meg"
    ],
    "ISBN" : "978-1-4842-1183-0",
    "Publisher" : "Apress",
    "Title" : "Definitive Guide to MongoDB 3rd ed., The",
    "Type" : "Book",
    "_id" : ObjectId("4c436231c603000000007ed0")
}

Removing Elements from an Array

MongoDB also includes several methods that let you remove elements from an array, including $pop, $pull, and $pullAll. In the sections that follow, you’ll learn how to use each of these methods for removing elements from an array.

The $pop operator lets you remove a single element from an array. This operator lets you remove the first or last value in the array, depending on the parameter you pass down with it. For example, the following snippet removes the last element from the array:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0" }, {$pop : {Author : 1 } } )

In this case, the $pop operator will pop Meg’s name off the list of authors. Passing down a negative number would remove the first element from the array. The following example removes Peter Membrey’s name from the list of authors:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0" }, {$pop : {Author : -1 } } )

Removing Each Occurrence of a Specified Value

The $pull operator lets you remove each occurrence of a specified value from an array. This can be particularly useful if you have multiple elements with the same value in your array. Let’s begin this example by using the $push parameter to add Stewie back to the list of authors:

> db.media.updateOne ( {"ISBN" : "978-1-4842-1183-0"}, {$push: { Author :"Griffin, Stewie"} } )

Stewie will be in and out of the database a couple more times as we walk through this book’s examples. You can remove all occurrences of this author in the document with the following code:

> db.media.updateOne ( {"ISBN" : "978-1-4842-1183-0"}, {$pull : { Author : "Griffin,Stewie" } } )

Removing Multiple Elements from an Array

You can also remove multiple elements with different values from an array. The $pullAll operator enables you to accomplish this. The $pullAll operator takes an array with all the elements you want to remove, as in the following example:

> db.media.updateOne( { "ISBN" : "978-1-4842-1183-0"}, {$pullAll : { Author :["Griffin, Louis","Griffin, Peter","Griffin, Brian"] } } )

The field from which you remove the elements (Author in the preceding example) needs to be an array. If it isn’t, you’ll receive an error message.

Specifying the Position of a Matched Array

You can use the $ operator in your queries to specify the position of the matched array item in your query. You can use this operator for data manipulation after finding an array member. For instance, assume you’ve added another track to your track list, but you accidently made a typo when entering the track number:

> db.media.updateOne( { "Artist" : "Nirvana" }, {$addToSet : { Tracklist :{"Track" : 2,"Title": "Been a Son", "Length":"2:23"} } } )

{
    "Artist" : "Nirvana",
    "Title" : "Nevermind",
    "Tracklist" : [
        {
                "Track" : "1",
                "Title" : "You Know You're Right",
                "Length" : "3:38"
        },
        {
                "Track" : "5",
                "Title" : "Smells Like Teen Spirit",
                "Length" : "5:02"
        },
        {
                "Track" : 2,
                "Title" : "Been a Son",
                "Length" : "2:23"
        }
    ],
    "Type" : "CD",
    "_id" : ObjectId("4c443ad6c603000000007ed5")
}

It so happens you know that the track number of the most recent item should be 3 rather than 2. You can use the $inc method in conjunction with the $ operator to increase the value from 2 to 3, as in this example:

> db.media.updateOne( { "Tracklist.Title" : "Been a Son"},{$inc:{"Tracklist.$.Track" : 1} } )

Note that only the first item it matches will be updated. Thus, if there are two identical elements in the comments array, only the first element will be increased.

Atomic Operations

MongoDB supports atomic operations executed against single documents. An atomic operation is a set of operations that can be combined in such a way that the set of operations appears to be merely one single operation to the rest of the system. This set of operations will have either a positive or a negative outcome as the final result.

You can call a set of operations an atomic operation if it meets the following pair of conditions:

1. No other process knows about the changes being made until the entire set of operations has completed.
2. If one of the operations fails, the entire set of operations (the entire atomic operation) will fail, resulting in a full rollback, where the data are restored to their state prior to running the atomic operation.

A standard behavior when executing atomic operations is that the data will be locked and therefore unable to be reached by other queries. However, MongoDB does not support locking or complex transactions for a number of reasons:

• In sharded environments (see Chapter 12 for more information on such environments), distributed locks can be expensive and slow. MongoDB’s goal is to be lightweight and fast, so expensive and slow go against this principle.
• MongoDB developers don’t like the idea of deadlocks. In their view, it’s preferable for a system to be simple and predictable instead.
• MongoDB is designed to work well for real-time problems. When an operation is executed that locks large amounts of data, it would also stop some smaller light queries for an extended period of time. Again, this goes against the MongoDB goal of speed.

MongoDB includes several update operators (as noted previously), all of which can atomically update an element:

• $set: Sets a particular value.
• $unset: Removes a particular value.
• $inc: Increments a particular value by a certain amount.
• $push: Appends a value to an array.
• $pull: Removes one or more values from an existing array.
• $pullAll: Removes several values from an existing array.

Using the Update-If-Current Method

Another strategy that atomic update uses is the update-if-current method. This method takes the following three steps:

1. It fetches the object from the document.
2. It modifies the object locally (with any of the previously mentioned operations, or a combination of them).
3. It sends an update request to update the object to the new value, in case the current value still matches the old value fetched.

You can check the WriteResult output to see whether all went well. Note that all of this happens automatically. Let’s take a new look at an example shown previously:

> db.media.updateOne( { "Tracklist.Title" : "Been a Son"},{$inc:{"Tracklist.$.Track" : 1} } )

Here, you can use the WriteResult output to check whether the update went smoothly:

WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

In this example, you incremented Tracklist.Track using the track list title as an identifier. But now consider what happens if the track list data are changed by another user using the same method while MongoDB was modifying your data. Because Tracklist.Title remains the same, you might assume (incorrectly) that you are updating the original data, when in fact you are overwriting the changes.

This is known as the ABA problem. This scenario might seem unlikely, but in a multiuser environment, where many applications are working on data at the same time, this can be a significant problem.

To avoid this problem, you can do one of the following:

• Use the entire object in the update’s query expression, instead of just the _id and comments.by fields.
• Use $set to set the field you care about. If other fields have changed, they won’t be affected by this.
• Put a version variable in the object and increment it on each update.
• When possible, use a $ operator instead of an update-if-current sequence of operations.

Modifying and Returning a Document Atomically

The findAndModify command also allows you to perform an atomic update on a document. This command modifies the document and returns it. The command takes three main operators: <query>, which is used to specify the document you’re executing it against; <sort>, which is used to sort the matching documents when multiple documents match, and <operations>, which is used to specify what needs to be done.

Now let’s look at a handful of examples that illustrate how to use this command. The first example finds the document you’re searching for and removes it once it is found:

> db.media.findAndModify( { "Title" : "One Piece",sort:{"Title": -1}, remove:true} )
{
        "_id" : ObjectId("4c445218c603000000007ede"),
        "Type" : "Manga",
        "Title" : "One Piece",
        "Volumes" : 612,
        "Read" : 524
}

This code returned the document it found matching the criteria. In this case, it found and removed the first item it found with the title “One Piece.” If you execute a find() function now, you will see that the document is no longer within the collection.

The next example modifies the document rather than removing it:

> db.media.findAndModify( { query: { "ISBN" : "978-1-4842-1183-0" }, sort:{"Title":-1}, update: {$set: {"Title" : " Different Title"} } } )

The preceding example updates the title from “Definitive Guide to MongoDB, The” to “Different Title”—and returns the old document (as it was before the update) to your shell. If you would rather see the results of the update on the document, you can add the new operator after your query:

> db.media.findAndModify( { query: { "ISBN" : "978-1-4842-1183-0" }, sort:{"Title":-1}, update: {$set: {"Title" : " Different Title"} }, new:true } )

Note that you can use any modifier operation with this command, not just $set.

Processing Data in Bulk

MongoDB also allows you to perform write operations in bulk. This way, you can first define the dataset prior to writing it all in a single go. Bulk write operations are limited to a single collection only and can be used to insert, update, or remove data.

Before you can write your data in bulk, you will first need to tell MongoDB how those data are to be written: ordered or unordered. When executing the operation in an ordered fashion, MongoDB will go over the list of operations serially. That is, were an error to occur while processing one of the write operations, the remaining operations will not be processed. In contrast, using an unordered write operation, MongoDB will execute the operations in a parallel manner. Were an error to occur during one of the writing operations here, MongoDB will continue to process the remaining write operations.

For example, let’s assume you want to insert data in bulk to your media collection in an ordered fashion, so that if an error were to occur the operation would halt. You first will need to initialize your ordered list using the initializeOrderedBulkOp() functionx, as follows:

> var bulk = db.media.initializeOrderedBulkOp();

Now you can continue to insert the data into your ordered list, named bulk, before finally executing the operations using the execute() command, like so:

> bulk.insertOne({ "Type" : "Movie", "Title" : "Deadpool", "Released" : 2016});
> bulk.insertOne({ "Type" : "CD", "Artist" : "Iron Maiden", "Title" : "Book of Souls, The" });
> bulk.insertOne({ "Type" : "Book", "Title" : "Paper Towns", "Author" : "Green, John" });

Executing Bulk Operations

Now that the list has been filled, you will notice that the data themselves have not been written into the collection yet. You can verify this by doing a simple find() on the media collection, which will only show the previously added content:

> db.media.find()
{ "_id" : ObjectId("55e6d1d8b54fe7a2c96567d4"),
"Type" : "Book",
"Title" : "Definitive Guide to MongoDB 3rd ed., The",
"ISBN" : "978-1-4842-1183-0",
"Publisher" : "Apress",
"Author" : [
   "Hows, David",
   "Plugge, Eelco",
   "Membrey, Peter",
   "Hawkins, Tim"
] }

{ "_id" : "ObjectId("4c1a86bb2955000000004076"),
"Type" : "CD",
"Artist" : "Nirvana",
"Title" : "Nevermind",
"Tracklist" : [
    {
        "Track" : "1",
            "Title" : "Smells Like Teen Spirit",
            "Length" : "5:02"
    },
    {
        "Track" : "2",
        "Title" : "In Bloom",
        "Length" : "4:15"
    }
] }

To process the list of operations, the execute() command can be used like so:

> bulk.execute();
BulkWriteResult({
            "writeErrors" : [ ],
            "writeConcernErrors" : [ ],
            "nInserted" : 3,
            "nUpserted" : 0,
            "nMatched" : 0,
            "nModified" : 0,
            "nRemoved" : 0,
            "upserted" : [ ]
})

As you can tell from the output, nInserted reports 3, meaning three items were inserted into your collection. If your list were to include other operations such as upserts or removals, those would have been listed here as well.

Evaluating the Output

Once the bulk operations have been executed using the execute() command, you are also able to review the write operations performed. This can be used to evaluate whether all the data were written successfully and in what order this was done. Moreover, when something does go wrong during the write operation, the output will help you understand what has been executed. To review the write operations executed through execute(), you can use the getOperations() command, like so:

> bulk.getOperations();
[
   {
      "originalZeroIndex" : 0,
      "batchType" : 1,
      "operations" : [
         {
            "_id" : ObjectId("55e7fa1db54fe7a2c96567d6"),
            "Type" : "Movie",
            "Title" : "Deadpool",
            "Released" : 2016
         },
         {
            "_id" : ObjectId("55e7fa1db54fe7a2c96567d7"),
            "Type" : "CD",
            "Artist" : "Iron Maiden",
            "Title" : "Book of Souls, The"
         },
         {
            "_id" : ObjectId("55e7fa1db54fe7a2c96567d8"),
            "Type" : "Book",
            "Title" : "Paper Towns",
            "ISBN" : "978-0142414934",
            "Author" : "Green, John"
         }
      ]
   }
]

Notice how the array returned includes all the data processed under the operations key, as well as the batchType key indicating the type of operation performed. Here, its value is 1, indicating the items were inserted into the collection. Table 4-2 describes the types of operations performed and their subsequent batchType values.

Bulk operations can be extremely useful for processing a large set of data in a single go without influencing the available dataset beforehand.

Renaming a Collection

It might happen that you discover you have named a collection incorrectly, but you’ve already inserted some data into it. This might make it troublesome to remove and read the data again from scratch.

Instead, you can use the renameCollection() function to rename your existing collection. The following example shows you how to use this simple and straightforward command:

> db.media.renameCollection("newname")
{ "ok" : 1 }

If the command executes successfully, an OK will be returned. If it fails, however (if the collection doesn’t exist, for example), then the following message is returned:

{ "errmsg" : "assertion: source namespace does not exist", "ok" : 0 }

The renameCollection command doesn’t take many parameters (unlike some commands you’ve seen so far); however, it can be quite useful in the right circumstances.

Deleting Data

So far we’ve explored how to add, search for, and modify data. Next, we’ll examine how to delete documents, entire collections, and the databases themselves.

Previously, you learned how to delete data from a specific document (using the $pop command, for instance). In this section, you will learn how to delete full documents and collections. Just as the insertOne() function is used for inserting and updateOne() is used for modifying a document, deleteOne() is used to delete a document.

To delete a single document from your collection, you need to specify the criteria you’ll use to find the document. A good approach is to perform a find() first; this ensures that the criteria used are specific to your document. Once you are sure of the criterion, you can invoke the deleteOne() function using that criterion as a parameter:

> db.newname.deleteOne( { "Title" : "Different Title" } )

This statement removes a single matching document. Any other item in your collection that matches the criteria will not be removed when using the deleteOne() function. To delete multiple documents matching your criteria, you can use the deleteMany() function instead.

Or you can use the following snippet to delete all documents from the newname library (remember, we renamed the media collection this previously):

> db.newname.deleteMany({})

If you want to delete an entire collection, you can use either the drop() or remove() function. Using remove() will be a lot slower than drop() as all indexes will be kept this way. A drop() will be faster if you need to remove all data as well as indexes from a collection. The following snippet removes the entire newname collection, including all of its documents:

> db.newname.drop()
true

The drop() function returns either true or false, depending on whether the operation has completed successfully. Likewise, if you want to remove an entire database from MongoDB, you can use the dropDatabase() function, as in this example:

> db.dropDatabase()
{ "dropped" : "library", "ok" : 1 }

Note that this snippet will remove the database you are currently working in (again, be sure to check db to see which database is your current database).

Referencing a Database

At this point, you have an empty database again. You’re also familiar with inserting various kinds of data into a collection. Now you’re ready to take things a step further and learn about database referencing (DBRef). As you’ve already seen, there are plenty of scenarios where embedding data into your document will suffice for your application (such as the track list or the list of authors in the book entry). However, sometimes you do need to reference information in another document. The following sections will explain how to go about doing so.

Just as with SQL, references between documents in MongoDB are resolved by performing additional queries on the server. MongoDB gives you two ways to accomplish this: referencing them manually or using the DBRef standard, which many drivers also support.

Referencing Data Manually

The simplest and most straightforward way to reference data is to do so manually. When referencing data manually, you store the value from the _id of the other document in your document, either through the full ID or through a simpler common term. Before proceeding with an example, let’s add a new document and specify the publisher’s information in it (pay close attention to the _id field):

> apress = ( { "_id" : "Apress", "Type" : "Technical Publisher", "Category" :["IT", "Software","Programming"] } )
{
        "_id" : "Apress",
        "Type" : "Technical Publisher",
        "Category" : [
                "IT",
                "Software",
                "Programming"
        ]
}
> db.publisherscollection.insertOne(apress)

Once you add the publisher’s information, you’re ready to add an actual document (for example, a book’s information) into the media collection. The following example adds a document, specifying Apress as the name of the publisher:

> book = ( { "Type" : "Book", "Title" : "Definitive Guide to MongoDB 3rd ed., The","ISBN" : "978-1-4842-1183-0", "Publisher" : "Apress","Author" : ["Hows, David","Plugge, Eelco","Membrey,Peter","Hawkins, Tim"] } )
{
        "Type" : "Book",
        "Title" : "Definitive Guide to MongoDB 3rd ed., The",
        "ISBN" : "978-1-4842-1183-0",
        "Publisher": "Apress",
        "Author" : [
                "Hows, David"
                "Membrey, Peter",
                "Plugge, Eelco",
                "Hawkins, Tim"
        ]
}
> db.media.insertOne(book)

All the information you need has been inserted into the publisherscollection and media collections, respectively. You can now start using the database reference. First, specify the document that contains the publisher’s information to a variable:

> book = db.media.findOne()
{
        "_id" : ObjectId("4c458e848e0f00000000628e"),
        "Type" : "Book",
        "Title" : "Definitive Guide to MongoDB 3rd ed., The",
        "ISBN" : "978-1-4842-1183-0",
        "Publisher" : "Apress",
        "Author" : [
                "Hows, David"
                "Membrey, Peter",
                "Plugge, Eelco",
                "Hawkins, Tim"
        ]
}

To obtain the information itself, you combine the findOne function with some dot notation:

> db.publisherscollection.findOne( { _id : book.Publisher } )
{
        "_id" : "Apress",
        "Type" : "Technical Publisher",
        "Category" : [
                "IT",
                "Software",
                "Programming"
        ]
}

As this example illustrates, referencing data manually is straightforward and doesn’t require much brainwork. Here, the _id in the documents placed in the users collection has been manually set and has not been generated by MongoDB (otherwise, the _id would be an object ID).

Referencing Data with DBRef

The DBRef standard provides a more formal specification for referencing data between documents. The main reason for using DBRef over a manual reference is that the collection can change from one document to the next. So, if your referenced collection will always be the same, referencing data manually (as just described) is fine.

With DBRef, the database reference is stored as a standard embedded (JSON/BSON) object. Having a standard way to represent references means that drivers and data frameworks can add helper methods that manipulate the references in standard ways.

The syntax for adding a DBRef reference value looks like this:

{ $ref : <collectionname>, $id : <id value>[, $db : <database name>] }

Here, <collectionname> represents the name of the collection referenced (for example, publisherscollection); <id value> represents the value of the _id field for the object you are referencing; and the optional $db allows you to reference documents that are placed in other databases.

Let’s look at another example using DBRef from scratch. Begin by emptying your two collections and adding a new document:

> db.publisherscollection.drop()
true
> db.media.drop()
true
> apress = ( { "Type" : "Technical Publisher", "Category" :
["IT","Software","Programming"] } )
{
        "Type" : "Technical Publisher",
        "Category" : [
                "IT",
                "Software",
                "Programming"
        ]
}
> db.publisherscollection.save(apress)

So far you’ve defined the variable apress and saved it using the save() function. Next, display the updated contents of the variable by typing in its name:

> apress
{
"Type" : "Technical Publisher",
"Category" : [
     "IT",
     "Software",
     "Programming"
],
"_id" : ObjectId("4c4597e98e0f000000006290")
}

So far you’ve defined the publisher and saved it to the publisherscollection collection. Now you’re ready to add an item to the media collection that references the data:

> book = { "Type" : "Book", "Title" : "Definitive Guide to MongoDB 3rd ed., The","ISBN" : "978-1-4842-1183-0", "Author": ["Hows, David","Membrey, Peter","Plugge,Eelco","Hawkins, Tim"], Publisher : [ new DBRef ('publisherscollection',apress._id) ] }

{
        "Type" : "Book",
        "Title" : "Definitive Guide to MongoDB 3rd ed., The",
        "ISBN" : "978-1-4842-1183-0",
        "Author" : [
                "Hows, David”
                "Membrey, Peter",
                "Plugge, Eelco",
                "Hawkins, Tim"
        ],
        "Publisher" : [
                DBRef("publishercollection", "Apress")
        ]
}
> db.media.save(book)

And that’s it! Granted, the example looks a little less simple than the manual method of referencing data; however, it’s a good alternative for cases where collections can change from one document to the next.

Implementing Index-Related Functions

In Chapter 3, you looked at what indexes can do for your database. Now it’s time to briefly learn how to create and use indexes. Indexing will be discussed in greater detail in Chapter 10, but for now let’s look at the basics. MongoDB includes a fair number of functions available for maintaining your indexes; we’ll begin by creating an index with the createIndex() function.

The createIndex() function takes at least one parameter, which is the name of a key in one of your documents that you will use to build the index. In the previous example, you added a document to the media collection that used the Title key. This collection would be well served by an index on this key.

You can create an index for this collection by invoking the following command:

> db.media.createIndex( { Title : 1 } )

This command ensures that an index will be created for all the Title values from all documents in the media collection. The :1 at the end of the line specifies the direction of the index: 1 would order the index entries in ascending order, whereas -1 would order the index entries in descending order:

// Ensure ascending index
db.media.createIndex( { Title :1 } )

// Ensure descending index
db.media.createIndex( { Title :-1 } )

BSON allows you to store full arrays in a document; however, it would also be beneficial to be able to create an index on an embedded key. Luckily, the developers of MongoDB thought of this, too, and added support for this feature. Let’s build on one of the earlier examples in this chapter, adding another document into the database that has embedded information:

> db.media.insertOne( { "Type" : "CD", "Artist" : "Nirvana","Title" :"Nevermind", "Tracklist" : [ { "Track" : "1", "Title" : "Smells Like TeenSpirit", "Length" : "5:02" }, {"Track" : "2","Title" : "In Bloom", "Length" :"4:15" } ] } )

{ "_id" : ObjectId("4c45aa2f8e0f000000006293"), "Type" : "CD", "Artist" :
"Nirvana", "Title" : "Nevermind", "Tracklist" : [
        {
                "Track" : "1",
                "Title" : "Smells Like Teen Spirit",
                "Length" : "5:02"
        },
        {
                "Track" : "2",
                "Title" : "In Bloom",
                "Length" : "4:15"
        }
] }

Next, you can create an index on the Title key for all entries in the track list:

> db.media.createIndex( { "Tracklist.Title" : 1 } )

The next time you perform a search for any of the titles in the collection—assuming they are nested under Tracklist—the titles will show up instantly. Next, you can take this concept one step further and use an entire (sub)document as a key, as in this example:

> db.media.createIndex( { "Tracklist" : 1 } )

This statement indexes each element of the array, which means you can now search for any object in the array. These types of keys are also known as multikeys. You can also create an index based on multiple keys in a set of documents. This process is known as compound indexing. The method you use to create a compound index is mostly the same; the difference is that you specify several keys instead of one, as in this example:

> db.media.createIndex({"Tracklist.Title": 1, "Tracklist.Length": -1})

The benefit of this approach is that you can make an index on multiple keys (as in the previous example, where you indexed an entire subdocument). Unlike the subdocument method, however, compound indexing lets you specify whether you want one of the two fields to be indexed in descending order. If you perform your index with the subdocument method, you are limited to ascending or descending order only. There is more on compound indexes in Chapter 10.

Surveying Index-Related Commands

So far you’ve taken a quick glance at one of the index-related commands, createIndex(). Without a doubt, this is the command you will primarily use to create your indexes. However, you might also find a pair of additional functions useful: hint() and min()/max(). You use these functions to query for data. We haven’t covered them to this point because they won’t function without a custom index. But now let’s take a look at what they can do for you.

Forcing a Specified Index to Query Data

You can use the hint() function to force the use of a specified index when querying for data. The intended benefit of using this command is to improve the query performance where the query planner does not consistently use a good index for a given query. This option should be used with caution, as you can also force an index to be used, which will result in poor performance.

To see this principle in action, try performing a find with the hint() function without defining an index:

> db.media.find( { ISBN: " 978-1-4842-1183-0"} ) . hint ( { ISBN: -1 } )
error: { "$err" : "bad hint", "code" : 10113 }

If you create an index on ISBN numbers, this technique will be more successful. Note that the first command’s background parameter ensures that the indexing is done on the background. This is useful as by default initial index builds are done on the foreground, which is a blocking operation for other writes. The background option allows the initial index build to happen without blocking other writes:

> db.media.ensureIndex({ISBN: 1}, {background: true});
> db.media.find( { ISBN: "978-1-4842-1183-0"} ) . hint ( { ISBN: 1 } )

{ "_id" : ObjectId("4c45a5418e0f000000006291"), "Type" : "Book", "Title" : "Definitive Guide to MongoDB 3rd ed., The", "ISBN" : "978-1-4842-1183-0", "Author" : ["Hows, David","Membrey, Peter", "Plugge, Eelco","Hawkins,Tim"], "Publisher" : [
       {
                "$ref" : "publisherscollection",
                "$id" : ObjectId("4c4597e98e0f000000006290")
        }
] }

To confirm that the given index is being used, you can optionally add the explain() function, returning information about the query plan chosen. Here, the indexBounds value tells you about the index used:

> db.media.find( { ISBN: "978-1-4842-1183-0"} ) . hint ( { ISBN: 1 } ).explain()
{
  "waitedMS" : NumberLong(0),
  "queryPlanner" : {
    "plannerVersion" : 1,
    "namespace" : "library.media",
    "indexFilterSet" : false,
    "parsedQuery" : {
      "$and" : [ ]
    },
    "winningPlan" : {
      "stage" : "COLLSCAN",
      "filter" : {
        "$and" : [ ]
      },
      "direction" : "forward"
    },
    "rejectedPlans" : [ ]
  },
  "serverInfo" : {
    "host" : "localhost",
    "port" : 27017,
    "version" : "3.1.7",
    "gitVersion" : "7d7f4fb3b6f6a171eacf53384053df0fe728db42"
  },
  "ok" : 1
}

Constraining Query Matches

The min() and max() functions enable you to constrain query matches to only those that have index keys between the min and max keys specified. Therefore, you will need to have an index for the keys you are specifying. Also, you can either combine the two functions or use them separately. Let’s begin by adding a few documents that enable you to take advantage of these functions. First, create an index on the Released field:

> db.media.insertOne( { "Type" : "DVD", "Title" : "Matrix, The", "Released" :1999} )
> db.media.insertOne( { "Type" : "DVD", "Title" : "Blade Runner", "Released" :1982 } )
> db.media.insertOne( { "Type" : "DVD", "Title" : "Toy Story 3", "Released" :2010} )
> db.media.ensureIndex( { "Released": 1 } )

You can now use the max() and min() commands, as in this example:

> db.media.find() . min ( { Released: 1995 } ) . max ( { Released : 2005 } )
{ "_id" : ObjectId("4c45b5b38e0f0000000062a9"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999 }

If no index is created, then an error message will be returned, saying that no index has been found for the specified key pattern. Obviously, you will need to define which index must be used with the hint() function:

> db.media.find() . min ( { Released: 1995 } ) .max ( { Released : 2005 } ). hint ( { Released : 1 } )
{ "_id" : ObjectId("4c45b5b38e0f0000000062a9"), "Type" : "DVD", "Title" :
"Matrix, The", "Released" : 1999 }

Generally speaking, it is recommended that you use $gt and $lt (greater than and less than, respectively) rather than min() and max() because $gt and $lt don’t require an index. The min() and max() functions are used primarily for compound keys.

Summary

In this chapter, we’ve taken a look at the most commonly used commands and options that can be performed with the MongoDB shell to manipulate data. We also examined how to search for, add, modify, and delete data, and how to modify your collections and databases. Next, we took a quick look at atomic operations, how to use aggregation, and when to use operators such as $elemMatch. Finally, we explored how to create indexes and when to use them. We examined what indexes are used for, how you can drop them, how to search for your data using the indexes created, and how to check for running indexing operations.

In the next chapter, we’ll look into the fundamentals of GridFS, including what it is, what it does, and how it can be used to your benefit.