In a NoSQL database, such as MongoDB, planning queries is a very important task, and depending on the query you want to perform, your document can vary greatly.
As you saw in Chapter 2, Data Modeling with MongoDB, the decision to refer or include documents in a collection is, in a large part, the result of our planning. It is essential to determine whether we will give a preference to reading or writing in a collection.
Here, we will see how planning queries can help us create documents in a more efficient and effective way, and we will also consider more sensible questions such as atomicity and transactions.
This chapter will focus on the following subjects:
- Read operations
- Write operations
- Write concerns
- Bulk writing documents
Read is the most common and fundamental operation in a database. It's very hard to imagine a database that is used only to write information, where this information is never read. By the way, I have never heard of such an approach.
In MongoDB, we can execute queries through the find interface. The find interface can accept queries as criteria and projections as parameters. This will result in a cursor. Cursors have methods that can be used as modifiers of the executed query, such as limit, map, skip, and sort. For example, take a look at the following query:
db.customers.find({"username": "johnclay"})
This would return the following document:
{ "_id" : ObjectId("54835d0ff059b08503e200d4"), "username" : "johnclay", "email" : "[email protected]", "password" : "bf383e8469e98b44895d61b821748ae1", "details" : { "firstName" : "John", "lastName" : "Clay", "gender" : "male", "age" : 25 }, "billingAddress" : [ { "street" : "Address 1, 111", "city" : "City One", "state" : "State One" } ], "shippingAddress" : [ { "street" : "Address 2, 222", "city" : "City Two", "state" : "State Two" }, { "street" : "Address 3,333", "city" : "City Three", "state" : "State Three" } ] }
We can use the find interface to execute a query in MongoDB. The find interface will select the documents in a collection and return a cursor for the selected documents.
Compared with the SQL language, the find interface should be seen as a select statement. And, similar to a select statement where we can determinate clauses with expressions and predicates, the find interface allows us to use criteria and projections as parameters.
As mentioned before, we will use JSON documents in these find interface parameters. We can use the find interface in the following way:
db.collection.find( {criteria}, {projection} )
In this example:
criteriais a JSON document that will specify the criteria for the selection of documents inside a collection by using some operatorsprojectionis a JSON document that will specify which document's fields in a collection will be returned as the query result
Both are optional parameters, and we will go into more detail regarding these later.
Let's execute the following example:
db.customers.find( {"username": "johnclay"}, {_id: 1, username: 1, details: 1} )
In this example:
{"username": "johnclay"}is the criteria{_id: 1, username: 1, details: 1}is the projection
This query will result in this document:
{ "_id" : ObjectId("54835d0ff059b08503e200d4"), "username" : "johnclay", "details" : { "firstName" : "John", "lastName" : "Clay", "gender" : "male", "age" : 25 } }
As mentioned in the previous section, in the find interface, both the criteria and projection parameters are optional. To use the find interface without any parameters means selecting all the documents in a collection.
So, a query in the products collection executes in this way:
db.products.find()
{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" } } { "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" } } …
Despite the convenience, selecting all the documents in a collection can turn out to be a bad idea due to a given collection's length. If we take as an example a collection with hundreds, thousands, or millions of records, it is essential to create a criterion in order to select only the documents we want.
However, nothing prevents the query result from being huge. In this case, depending on the chosen drive that is executing the query, we must iterate the returned cursor.
Let's check the following example query. We want to select the documents where the attribute name is Product 1:
db.products.find({name: "Product 1"});
This will give us as a result:
{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" } }
The preceding query selects the documents through the equality {name: "Product 1"}. It's also possible to use operators on the criteria interface.
The following example demonstrates how it's possible to select all documents where the price is greater than 10:
db.products.find({price: {$gt: 10}});
This produces as a result:
{ "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" } } { "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" } }
When we execute a query using the operator $gt, only documents that have an information price greater than 10 will be returned as a result in the cursor.
In addition, there are other operators such as comparison, logical, element, evaluation, geographical, and arrays.
Let's take, for example, the documents from the products collection, shown as follows:
{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 6 } ] } { "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] } { "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 9 } ] }
MongoDB provides us with a way to define equality between values. With comparison operators, we can compare BSON type values. Let's look at these operators:
- The
$gteoperator is responsible for searching values that are equal or greater than the value specified in the query. If we execute the querydb.products.find({price: {$gte: 20}}), it will return:{ "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] } { "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 9 } ] }
- With the
$ltoperator, it's possible to search for values that are inferior to the requested value in the query. The querydb.products.find({price: {$lt: 20}})will return:{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 6 } ] }
- The
$lteoperator searches for values that are less than or equal to the requested value in the query. If we execute the querydb.products.find({price: {$lte: 20}}), it will return:{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 6 } ] } { "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] }
- The
$inoperator is able to search any document where the value of a field equals a value that is specified in the requested array in the query. The execution of the querydb.products.find({price:{$in: [5, 10, 15]}})will return:{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 6 } ] }
- The
$ninoperator will match values that are not included in the specified array. The execution of thedb.products.find({price:{$nin: [10, 20]}})query will produce:{ "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 9 } ] }
- The
$neoperator will match any values that are not equal to the specified value in the query. The execution of thedb.products.find({name: {$ne: "Product 1"}})query will produce:{ "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] } { "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 9 } ] }
Logical operators are how we define the logic between values in MongoDB. These are derived from Boolean algebra, and the truth value of a Boolean value can be either true or false. Let's look at the logical operators in MongoDB:
- The
$andoperator will make a logical AND operation in an expressions array, and will return the values that match all the specified criteria. The execution of thedb.products.find({$and: [{price: {$lt: 30}}, {name: "Product 2"}]})query will produce:{ "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] }
- The
$oroperator will make a logical OR operation in an expressions array, and will return all the values that match either of the specified criteria. The execution of thedb.products.find({$or: [{price: {$gt: 50}}, {name: "Product 3"}]})query will produce:{ "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 9 } ] }
- The
$notoperator inverts the query effect and returns the values that do not match the specified operator expression. It is used to negate any operation. The execution of thedb.products.find({price: {$not: {$gt: 10}}})query will produce:{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description", "price" : 10, "supplier" : { "name" : "Supplier 1", "telephone" : "+552199998888" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 6 } ] }
- The
$noroperator will make a logical NOR operation in an expressions array, and will return all the values that fail to match all the specified expressions in the array. The execution of thedb.products.find({$nor:[{price:{$gt: 35}}, {price:{$lte: 20}}]})query will produce:{ "_id" : ObjectId("54837b69f059b08503e200dd"), "name" : "Product 3", "description" : "Product 3 description", "price" : 30, "supplier" : { "name" : "Supplier 3", "telephone" : "+552177776666" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 }, { "customer" : { "email" : "[email protected]" }, "stars" : 9 } ] }
To query a collection about our documents fields, we can use element operators.
The $exists operator will return all documents that have the specified field in the query. The execution of db.products.find({sku: {$exists: true}}) will not return any document, because none of them have the field sku.
Evaluation operators are how we perform an assessment of an expression in MongoDB. We must take care with this kind of operator, especially if there is no index for the field we are using on the criteria. Let's consider the evaluation operator:
- The
$regexoperator will return all values that match a regular expression. The execution ofdb.products.find({name: {$regex: /2/}})will return:{ "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] }
When we are working with arrays on a query, we should use array operators. Let's consider the array operator:
- The
$elemMatchoperator will return all documents where the specified array field values have at least one element that match the query criteria conditions.The
db.products.find({review: {$elemMatch: {stars: {$gt: 5}, customer: {email: "[email protected]"}}}})query will look at all the collection documents where thereviewfield has documents, thestarsfield value is greater than5, andcustomer emailis[email protected]:{ "_id" : ObjectId("54837b65f059b08503e200dc"), "name" : "Product 2", "description" : "Product 2 description", "price" : 20, "supplier" : { "name" : "Supplier 2", "telephone" : "+552188887777" }, "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 10 }, { "customer" : { "email" : "[email protected]" }, "stars" : 2 } ] }
Note
Besides the presented operators, we have: $mod, $text, $where, $all, $geoIntersects, $geoWithin, $nearSphere, $near, $size, and $comment. You can find more information regarding this in the MongoDB manual reference at http://docs.mongodb.org/manual/reference/operator/query/.
Until now, we have executed queries where the presented result is the document as it is persisted in MongoDB. But, in order to optimize the network overhead between MongoDB and its clients, we should use projections.
As you saw at the beginning of the chapter, the find interface allows us to use two parameters. The second parameter is projections.
By using the same sample collection we used in the previous session, an example of a query with projection would be:
db.products.find({price: {$not: {$gt: 10}}}, {name: 1, description: 1})
This query produces:
{ "_id" : ObjectId("54837b61f059b08503e200db"), "name" : "Product 1", "description" : "Product 1 description" }
The projection is a JSON document with all the fields we would like to present or hide, followed by 0 or 1, depending on what we want.
When a field is followed by a 0, then this field will not be shown in the resulting document. On the other hand, if the field is followed by a 1, then this means that it will be shown in the resulting document.
The db.products.find({price: {$not: {$gt: 10}}}, {_id: 0, name: 1, "supplier.name": 1}) query will show the following document:
{ "name" : "Product 1", "supplier" : { "name" : "Supplier 1" } }
In fields that have an array as a value, we can use operators such as $elemMatch, $split, $slice, and $.
The db.products.find({price: {$gt: 20}}, {review: {$elemMatch: {stars: 5}}}) query will produce:
{ "_id" : ObjectId("54837b69f059b08503e200dd"), "review" : [ { "customer" : { "email" : "[email protected]" }, "stars" : 5 } ] }