In this chapter, I’ll take up MongoDB, the database layer and the M in the MERN stack. Until now, we had an array of issues in the Express server’s memory that we used as the database. We’ll replace this with real persistence and read and write the list of issues from a MongoDB database.
To achieve this, we’ll need to install or use MongoDB on the cloud, get used to its shell commands, install a Node.js driver to access it from Node.js, and finally modify the server code to replace the API calls to read and write from a MongoDB database instead of the in-memory array of issues.
MongoDB Basics
This is an introductory section, where we will not be modifying the application. We’ll look at these core concepts in this section: MongoDB, documents, and collections. Then, we’ll set up MongoDB and explore these concepts with examples using the mongo shell to read and write to the database.
Documents
MongoDB is a document database, which means that the equivalent of a record is a document, or an object. In a relational database, you organize data in terms of rows and columns, whereas in a document database, an entire object can be written as a document.
For simple objects, this may seem no different from a relational database. But let’s say you have objects with nested objects (called embedded documents) and arrays. Now, when using a relational database, this will typically need multiple tables. For example, in a relational database, an Invoice object may be stored in a combination of an invoice table (to store the invoice details such as the customer address and delivery details) and an invoice_items table (to store the details of each item that is part of the shipment). In MongoDB, the entire Invoice object would be stored as one document. That’s because a document can contain arrays and other objects in a nested manner and the contained objects don’t have to be separated out into other documents.
A document is a data structure composed of field and value pairs. The values of fields may include objects, arrays, and arrays of objects and so on, as deeply nested as you want it to be. MongoDB documents are similar to JSON objects, so it is easy to think of them as JavaScript objects. Compared to a JSON object, a MongoDB document has support not only for the primitive data types—Boolean, numbers, and strings—but also other common data types such as dates, timestamps, regular expressions, and binary data.
In this document, there are numbers, strings, and a date data type. Further, there is a nested object (billingAddress) and an array of objects (items).
Collections
A collection is like a table in a relational database: it is a set of documents. Just like in a relational database, the collection can have a primary key and indexes. But there are a few differences compared to a relational database.
A primary key is mandated in MongoDB, and it has the reserved field name _id. Even if _id field is not supplied when creating a document, MongoDB creates this field and auto-generates a unique key for every document. More often than not, the auto-generated ID can be used as is, since it is convenient and guaranteed to produce unique keys even when multiple clients are writing to the database simultaneously. MongoDB uses a special data type called the ObjectId for the primary key.
The _id field is automatically indexed. Apart from this, indexes can be created on other fields, and this includes fields within embedded documents and array fields. Indexes are used to efficiently access a subset of documents in a collection.
Unlike a relational database, MongoDB does not require you to define a schema for a collection. The only requirement is that all documents in a collection must have a unique _id, but the actual documents may have completely different fields. In practice, though, all documents in a collection do have the same fields. Although a flexible schema may seem very convenient for schema changes during the initial stages of an application, this can cause problems if some kind of schema checking is not added in the application code.
As of version 3.6, MongoDB has supported a concept of schema, even though it is optional. You can read all about MongoDB schemas at https://docs.mongodb.com/manual/core/schema-validation/index.html . A schema can enforce allowed and required fields and their data types, just like GraphQL can. But it can also validate other things like string length and minimum and maximum values for integers.
But the errors generated because of schema violations do not give enough details as to which of the validation checks fail as of version 3.6. This may improve in future versions of MongoDB, at which point in time it is worth considering adding full-fledged schema checks. For the Issue Tracker application, we’ll not use the schema validation feature of MongoDB, instead, we’ll implement all necessary validations in the back-end code.
Databases
A database is a logical grouping of many collections. Since there are no foreign keys like in a SQL database, the concept of a database is nothing but a logical partitioning namespace. Most database operations read or write from a single collection, but $lookup, which is a stage in an aggregation pipeline, is equivalent to a join in SQL databases. This stage can combine documents within the same database.
Further, taking backups and other administrative tasks work on the database as a unit. A database connection is restricted to accessing only one database, so to access multiple databases, multiple connections are required. Thus, it is useful to keep all the collections of an application in one database, though a database server can host multiple databases.
Query Language
Unlike the universal English-like SQL in a relational database, the MongoDB query language is made up of methods to achieve various operations. The main methods for read and write operations are the CRUD methods. Other methods include aggregation, text search, and geospatial queries.
All methods operate on a collection and take parameters as JavaScript objects that specify the details of the operation. Each method has its own specification. For example, to insert a document, the only argument needed is the document itself. For querying, the parameters are a query filter and a list of fields to return (also called the projection).
Since there is no "language" for querying or updating, the query filters can be very easily constructed programmatically.
Unlike relational databases, MongoDB encourages denormalization, that is, storing related parts of a document as embedded subdocuments rather than as separate collections (tables) in a relational database. Take an example of people (name, gender, etc.) and their contact information (primary address, secondary address etc.). In a relational database, this would require separate tables for People and Contacts, and then a join on the two tables when all of the information is needed together. In MongoDB, on the other hand, it can be stored as a list of contacts within the same People document. That’s because a join of collections is not natural to most methods in MongoDB: the most convenient find() method can operate only on one collection at a time.
Installation
MongoDB Atlas ( https://www.mongodb.com/cloud/atlas ): I refer to this as Atlas for short. A small database (shared RAM, 512 MB storage) is available for free.
mLab (previously MongoLab) ( https://mlab.com/ ): mLab has announced an acquisition by MongoDB Inc. and may eventually be merged into Atlas itself. A sandbox environment is available for free, limited to 500 MB storage.
Compose ( https://www.compose.com ): Among many other services, Compose offers MongoDB as a service. A 30-day trial period is available, but a permanently free sandbox kind of option is not available.
Of these three, I find Atlas the most convenient because there are many options for the location of the host. When connecting to the database, it lets me choose one closest to my location, and that minimizes the latency. mLab does not give a cluster—a database can be created individually. Compose is not permanently free, and it is likely that you may need more than 30 days to complete this book.
The downside of any of the hosted options is that, apart from the small extra latency when accessing the database, you need an Internet connection. Which means that you may not be able to test your code where Internet access is not available, for example, on a flight. In comparison, installing MongoDB on your computer may work better, but the installation takes a bit more work than signing up for one of the cloud-based options.
Even when using one of the cloud options, you will need to download and install the mongo shell to be able to access the database remotely. Each of the services come with instructions on this step as well. Choose version 3.6 or higher of MongoDB when signing up for any of these services. Test the signup by connecting to the cluster or database using the mongo shell, by following instructions given by the service provider.
If you choose to install MongoDB on your computer (it can be installed easily on OS X, Windows, and most distributions based on Linux), look up the installation instructions, which are different for each operating system. You may install MongoDB by following the instructions at the MongoDB website ( https://docs.mongodb.com/manual/installation/ or search for “mongodb installation” in your search engine).
Choose MongoDB version 3.6 or higher, preferably the latest, as some of the examples use features introduced only in version 3.6. Most local installation options let you install the server, the shell, and tools all in one. Check that this is the case; if not, you may have to install them separately.
The message you see can be slightly different from this, especially if you have installed a different version of MongoDB. But you do need to see the prompt > where you can type further commands. If, instead, you see an error message, revisit the installation and the server starting procedure.
The Mongo Shell
The mongo shell is an interactive JavaScript shell, very much like the Node.js shell. In the interactive shell, a few non-JavaScript conveniences are available over and above the full power of JavaScript. In this section, we’ll discuss the basic operations that are possible via the shell, those that are most commonly used. For a full reference of all the capabilities of the shell, you can take a look at the mongo shell documentation at https://docs.mongodb.com/manual/mongo/ .
The commands that we will be typing in the mongo shell have been collected together in a file called mongo_commands.txt. These commands have been tested to work as is on Atlas or a local installation, but you may find variations in the other options. For example, mLab lets you connect only to a database (as opposed to a cluster), so it does not allow of switching between databases in mLab.
Note
If you find that something is not working as expected when typing a command, cross-check the commands with the same in the GitHub repository ( https://github.com/vasansr/pro-mern-stack-2 ). This is because typos may have been introduced during the production of the book, or last-minute corrections may have missed making it to the book. The GitHub repository, on the other hand, reflects the most up-to-date and tested set of code and commands.
You will find that there are no collections in this database, since it is a fresh installation. Further, you will also find that the database test was not listed when we listed the available databases. That’s because databases and collections are really created only on the first write operation to any of these.
This is the auto-completion feature of the mongo shell at work. Note that you can let the mongo shell auto-complete the name of any method by pressing the Tab character after entering the beginning few characters of the method.
The shell by itself does very little apart from providing a mechanism to access methods of the database and collections. It is the JavaScript engine, which forms the basis of the shell and gives a lot of flexibility and power to the shell.
In the next section, we will discuss more methods on the collection, such as insertOne() that you just learned about. These methods are accessible from many programming languages via a driver. The mongo shell is just another tool that can access these methods. You will find that the methods and arguments available in other programming languages are very similar to those in the mongo shell.
Exercise: MongoDB Basics
- 1.
Using the shell, display a list of methods available on the cursor object. Hint: Look up the mongo shell documentation for mongo Shell Help at https://docs.mongodb.com/manual/tutorial/access-mongo-shell-help/ .
Answers are available at the end of the chapter.
MongoDB CRUD Operations
This is different from removing all the documents in the collection, because it also removes any indexes that are part of the collection.
Create
This works just fine, and using find() , you can see that two documents exist in the collection, but they are not necessarily the same schema. This is the advantage of a flexible schema: the schema can be enhanced whenever a new data element that needs to be stored is discovered, without having to explicitly modify the schema.
In this case, it is implicit that any employee document where the middle field under name is missing indicates an employee without a middle name. If, on the other hand, a field was added that didn’t have an implicit meaning when absent, its absence would have to be handled in the code. Or a migration script would have to be run that defaults the field’s value to something.
You will also find that the format of the _id field is different for the two documents, and even the data type is different. For the first document, the data type is an integer. For the second, it is of type ObjectID (which is why it is shown as ObjectID(...). Thus, it’s not just the presence of fields that can differ between two documents in the same collection, even the data types of the same field can be different.
In most cases, leaving the creation of the primary key to MongoDB works just great, because you don’t have to worry about keeping it unique: MongoDB does that automatically. But, this identifier is not human-readable. In the Issue Tracker application, we want the identifier to be a number so that it can be easily remembered and talked about. But instead of using the _id field to store the human-readable identifier, let’s use a new field called id and let MongoDB auto-generate _id.
Read
Now that there are multiple documents in the collection, let’s see how to retrieve a subset of the documents as opposed to the full list. The find() method takes in two more arguments. The first is a filter to apply to the list, and the second is a projection, a specification of which fields to retrieve.
Note that we did not use pretty() here, yet, the output is prettified. This is because findOne() returns a single object and the mongo shell prettifies objects by default.
The number of documents returned now should be reduced to only one, since there is only one document that matched both the criteria, the last name being equal to 'Doe' as well as age being greater than 30. Note that we used the dot notation for specifying a field embedded in a nested document. And this also made us use quotes around the field name, since it is a regular JavaScript object property.
To match multiple values of the same field—for example, to match age being greater than 30 and age being less than 60—the same strategy cannot be used. That’s because the filter is a regular JavaScript object, and two properties of the same name cannot exist in a document. Thus, a filter like { age: { $gte: 30 }, age: { $lte: 60 } } will not work (JavaScript will not throw an error, instead, it will pick just one of the values for the property age). An explicit $and operator has to be used, which takes in an array of objects specifying multiple field-value criteria. You can read all about the $and operator and many more operators in the operators section of the reference manual of MongoDB at https://docs.mongodb.com/manual/reference/operator/query/ .
With this index, any query that uses a filter that has the field age in it will be significantly faster because MongoDB will use this index instead of scanning through all documents in the collection. But this was not a unique index, as many people can be the same age.
Projection
All this while, we retrieved the entire document that matched the filter. In the previous section, when we had to print only a subset of the fields of the document, we did it using a forEach() loop. But this means that the entire document is fetched from the server even when we needed only some parts of it for printing. When the documents are large, this can use up a lot of network bandwidth. To restrict the fetch to only some fields, the find() method takes a second argument called the projection. A projection specifies which fields to include or exclude in the result.
Update
There are two methods—updateOne() and updateMany()—available for modifying a document. The arguments to both methods are the same, except that updateOne() stops after finding and updating the first matching document. The first argument is a query filter, the same as the filter that find() takes. The second argument is an update specification if only some fields of the object need to be changed.
The matchedCount returned how many documents matched the filter. If the filter had matched more than one, that number would have been returned. But since the method is supposed to modify only one document, the modified count should always be 1, unless the modification had no effect. If you run the command again, you will find that modifiedCount will be 0, since the age was already 23 for the employee with ID 2.
Note that even though the field organization did not exist in the documents, the new value MyCompany would have been applied to all of them. If you execute the command find() to show the companies alone in the projection, this fact will be confirmed.
You can see that it no longer has the fields name.last and organization, because these were not specified in the document that was supplied to the command replaceOne(). It just replaces the document with the one supplied, except for the field ObjectId. Being the primary key, this field cannot be changed via an updateOne() or a replaceOne().
Delete
The delete operation takes a filter and removes the document from the collection. The filter format is the same, and the variations deleteOne() and deleteMany() are both available, just as in the update operation.
Aggregate
The find() method is used to return all the documents or a subset of the documents in a collection. Many a time, instead of the list of documents, we need a summary or an aggregate, for example, the count of documents that match a certain criterion.
The count() method can surely take a filter. But what about other aggregate functions, such as sum? That is where the aggregate() comes into play. When compared to relational databases supporting SQL, the aggregate() method performs the function of the GROUP BY clause. But it can also perform other functions such as a join, or even an unwind (expand the documents based on arrays within), and much more.
You can look up the advanced features that the aggregate() function supports in the MongoDB documentation at https://docs.mongodb.com/manual/reference/operator/aggregation-pipeline/ but for now, let’s look at the real aggregation and grouping construct that it provides.
The aggregate() method works in a pipeline. Every stage in the pipeline takes the input from the result of the previous stage and operates as per its specification to result in a new modified set of documents. The initial input to the pipeline is, of course, the entire collection. The pipeline specification is in the form of an array of objects, each element being an object with one property that identifies the pipeline stage type and the value specifying the pipeline’s effect.
For example, the find() method can be replicated using aggregate() by using the stages $match (the filter) and $project (the projection). To perform an actual aggregation, the $group stage needs to be used. The stage’s specification includes the grouping key identified by the property _id and other fields as keys, whose values are aggregation specifications and fields on which the aggregation needs to be performed. The _id can be null to group on the entire collection.
There are other aggregation functions, including minimum and maximum. For the complete set, refer to the documentation at https://docs.mongodb.com/manual/reference/operator/aggregation/group/#accumulator-operator .
Exercise: MongoDB Crud Operations
- 1.
Write a simple statement to retrieve all employees who have middle names. Hint: Look up the MongoDB documentation for query operators at https://docs.mongodb.com/manual/reference/operator/query/ .
- 2.
Is the filter specification a JSON? Hint: Think about date objects and quotes around field names.
- 3.
Say an employee’s middle name was set mistakenly, and you need to remove it. Write a statement to do this. Hint: Look up the MongoDB documentation for update operators at https://docs.mongodb.com/manual/reference/operator/update/ .
- 4.
During index creation, what did the 1 indicate? What other valid values are allowed? Hint: Look up the MongoDB indexes documentation at https://docs.mongodb.com/manual/indexes/ .
Answers are available at the end of the chapter.
MongoDB Node.js Driver
This is the Node.js driver that lets you connect and interact with the MongoDB server. It provides methods very similar to what you saw in the mongo shell, but not exactly the same. Instead of the low-level MongoDB driver, we could use an Object Document Mapper called Mongoose, which has a higher level of abstraction and more convenient methods. But learning about the lower-level MongoDB driver may give you a better handle on the actual working of MongoDB itself, so I’ve chosen to use the low-level driver for the Issue Tracker application.
Let’s also start a new Node.js program just to try out the different ways that the driver’s methods can be used. In the next section, we’ll use some code from this trial to integrate the driver into the Issue Tracker application. Let’s call this sample Node.js program trymongo.js and place it in a new directory called scripts, to distinguish it from other files that are part of the application.
The URL should start with mongodb:// followed by the hostname or the IP address of the server to connect to. An optional port can be added using : as the separator, but it’s not required if the MongoDB server is running on the default port, 27017. It’s good practice to separate the connection parameters into a configuration file rather than keep them in a checked-in file, but we’ll do this in the next chapter. For the moment, let’s hard code this. If you have used one of the cloud providers, the URL can be obtained from the corresponding connection instructions. For the local installation, the URL will be mongodb://localhost/issuetracker. Note that the MongoDB Node.js driver accepts the database name as part of the URL itself, and it is best to specify it this way, even though a cloud provider may not show this explicitly.
With this collection, we can do the same things we did with the mongo shell’s equivalent db.employees in the previous section. The methods are also very similar, except that they are all asynchronous. This means that the methods take in the regular arguments, but also a callback function that’s called when the operation completes. The convention in the callback functions is to pass the error as the first argument and the result of the operation as the second argument. You already saw this pattern of callback in the previous connection method.
Note that accessing the collection and the insert operation can only be called within the callback of the connection operation, because only then do we know that the connection has succeeded. There also needs to be some amount of error handling, but let’s deal with this a little later.
Let’s put all this together in a function called testWithCallbacks(). We will soon also use a different method of using the Node.js driver using async/await. Also, as is customary, let’s pass a callback function to this function, which we will call from the testWithCallbacks() function once all the operations are completed. Then, if there are any errors, these can be passed to the callback function.
Close the connection to the server
Call the callback
Return from the call, so that no more operations are performed
With all the error handling and callbacks introduced, the final code in the trymongo.js file is shown in Listing 6-1.
Note
Although no effort has been spared to ensure that all code listings are accurate, there may be typos or even corrections that did not make it to the book before it went to press. So, always rely on the GitHub repository ( https://github.com/vasansr/pro-mern-stack-2 ) as the tested and up-to-date source for all code listings, especially if something does not work as expected.
trymongo.js: Using Node.js driver, Using the Callbacks Paradigm
As you probably felt yourself, the callback paradigm is a bit unwieldy. But the advantage is that it works in the older JavaScript version (ES5), and therefore, older versions of Node.js. The callbacks are bit too deeply nested and the error handling makes for repetitive code. ES2015 started supporting Promises, which is supported by the Node.js MongoDB driver as well, and this was an improvement over callbacks. But in ES2017 and Node.js from version 7.6, full support for the async/await paradigm appeared, and this is the recommended and most convenient way to use the driver.
Errors will be thrown and can be caught. We can place all the operations in a single try block and catch any error in one place (the catch block) rather than after each call. There is no need for the function to take a callback, because if the caller needs to wait for the result, an await can be added before the call to this function, and errors can be thrown.
trymongo.js, testWithAsync Function
A good way to test whether errors are being caught and displayed is by running the program again. There will be errors because we have a unique index on the field id, so MongoDB will throw a duplicate key violation. If you have dropped the collection after creating the index, you could run the createIndex() command to reinstate this index.
As you can see, the async/await paradigm is much smaller in terms of code, as well as a lot clearer and easier to read. In fact, although we caught the error within this function, we didn’t have to do it. We could as well have let the caller handle it.
Given the benefits of the async/await paradigm, let’s use this in the Issue Tracker application when interacting with the database.
Schema Initialization
The mongo shell is not only an interactive shell, but is also a scripting environment. Using this, scripts can be written to perform various tasks such as schema initialization and migration. Because the mongo shell is in fact built on top of a JavaScript engine, the power of JavaScript is available in the scripts, just as in the shell itself.
One difference between the interactive and the non-interactive mode of working is that the non-interactive shell does not support non-JavaScript shortcuts, such as use <db> and show collections commands. The script has to be a regular JavaScript program adhering to the proper syntax.
Let’s create a schema initialization script called init.mongo.js within the script directory. Since MongoDB does not enforce a schema, there is really no such thing as a schema initialization as you may do in relational databases, like creation of tables. The only thing that is really useful is the creation of indexes, which are one-time tasks. While we’re at it, let’s also initialize the database with some sample documents to ease testing. We will use the same database called issuetracker that we used to try out the mongo shell, to store all the collections relevant to the Issue Tracker application.
Let’s copy the array of issues from server.js and use the same array to initialize the collection using insertMany() on a collection called issues. But before that, let’s clear existing issues it by calling a remove() with an empty filter (which will match all documents) on the same collection. Then, let’s create a few indexes on useful fields that we will be using to search the collection with.
init.mongo.js: Schema Initialization
For the other methods of using MongoDB, there are instructions as comments on the top of the script. In essence, the entire connection string has to be specified in the command line, including the username and password that you use to connect to the hosted service. Following the connection string, you can type the name of the script, scripts/init.mongo.js.
You can run this any time you wish to reset the database to its pristine state. You should see an output that indicates that two issues were inserted, among other things such as the MongoDB version and the shell version. Note that creating an index when one already exists has no effect, so it is safe to create the index multiple times.
Exercise: Schema Initialization
- 1.
The same schema initialization could have been done using a Node.js script and the MongoDB driver. What are the pros and cons of each of these methods: using the mongo shell vs. the Node.js MongoDB driver?
- 2.
Are there any other indexes that may be useful? Hint: What if we needed a search bar in the application? Read about MongoDB index types at https://docs.mongodb.com/manual/indexes/#index-types .
Answers are available at the end of the chapter.
Reading from MongoDB
In the previous section, you saw how to use the Node.js driver to perform basic CRUD tasks. With this knowledge, let’s now change the List API to read from the MongoDB database rather than the in-memory array of issues in the server. Since we’ve initialized the database with the same initial set of issues, while testing, you should see the same set of issues in the UI.
server.js: Changes for Reading the Issue List from MongoDB
Note
We did not have to do anything special due to the fact that the resolver issueList() is now an async function, which does not immediately return a value. The graphql-tools library handles this automatically. A resolver can return a value immediately or return a Promise (which is what an async function returns immediately). Both are acceptable return values for a resolver.
schema.graphql: Changes to add _id as a Field in Issue
Now, assuming that the server is still running (or that you have restarted the server and the compilation), if you refresh the browser, you will find that the two initial sets of issues are listed in a table, as before. The UI itself will show no change, but to convince yourself that the data is indeed coming from the database, you could modify the documents in the collection using the mongo shell and the updateMany() method on the collection. If, for example, you update effort to 100 for all the documents and refresh the browser, you should see that the effort is indeed showing 100 for all the rows in the table.
Exercise: Reading from MongoDB
- 1.
We are saving the connection in a global variable. What happens when the connection is lost? Stop the MongoDB server and start it again to see what happens. Does the connection still work?
- 2.
Shut down the MongoDB server, wait for a minute or more, and then start the server again. Now, refresh the browser. What happens? Can you explain this? What if you wanted a longer period for the connection to work even if the database server is down? Hint: Look up the connection settings parameters at http://mongodb.github.io/node-mongodb-native/3.1/reference/connecting/connection-settings/ .
- 3.
We used toArray() to convert the list of issues into an array. What if the list is too big, say, a million documents? How would you deal with this? Hint: Look up the documentation for the MongoDB Node.js driver's Cursor at http://mongodb.github.io/node-mongodb-native/3.1/api/Cursor.html . Note that the find() method returns a Cursor.
Writing to MongoDB
In order to completely replace the in-memory database on the server, we’ll also need to change the Create API to use the MongoDB database. As you saw in the MongoDB CRUD Operations section, the way to create a new document is to use the insertOne() method on the collection.
We used the size of the in-memory array to generate the new document’s id field. We could do the same, using the count() method of the collection to get the next ID. But there is a small chance when there are multiple users using the application that a new document is created between the time we call the count() method and the time we call the insertOne() method. What we really need is a reliable way of generating a sequence of numbers that cannot give us duplicates, much like sequences in popular relational databases.
MongoDB does not provide such a method directly. But it does support an atomic update operation, which can return the result of the update. This method is called findOneAndUpdate(). Using this method, we can update a counter and return the updated value, but instead of using the $set operator, we can use the $inc operator, which increments the current value.
Let’s first create a collection with the counter that holds a value for the latest Issue ID generated. To make it a bit generic, let’s assume we may have other such counters and use a collection with an ID set to the name of the counter and a value field called current holding the current value of the counter. In the future, we could add more counters in the same collections, and these would translate to one document for each counter.
init.mongo.js: Initialize Counters for Issues
Now, a call to findOneAndUpdate() that increments the current field is guaranteed to return a unique value that is next in the sequence. Let’s create a function in server.js that does this, but in a generic manner. We’ll let it take the ID of the counter and return the next sequence. In this function, all we have to do is call findOneAndUpdate(). It identifies the counter to use using the ID supplied, increments the field called current, and returns the new value. By default, the result of the findOneAndUpdate() method returns the original document. To make it return the new, modified document instead, the option returnOriginal has to be set to false.
Note
The option for returning the current or new value is called differently in the Node.js driver and in the mongo shell. In the mongo shell, the option is called returnNewDocument and the default is false. In the Node.js driver, the option is called returnOriginal and the default is true. In both cases, the default behavior is to return the original, so the option must be specified to return the new document.
server.js: Changes for Create API to Use the Database
Testing this set of changes will show that new issues can be added, and even on a restart of the Node.js server, or the database server, the newly added issues are still there. As a cross-check, you could use the mongo shell to look at the contents of the collection after every change from the UI.
Exercise: Writing to MongoDB
- 1.
Could we have just added the _id to the passed-in object and returned that instead of doing a find() for the inserted object?
Answers are available at the end of the chapter.
Summary
In this chapter, you learned about the installation and other ways of getting access to an instance of a database in MongoDB. You saw how to use the mongo shell and the Node.js driver to access the basic operations in MongoDB: the CRUD operations. We then modified the Issue Tracker application to use some of these methods to read and write to the MongoDB database, thus making the issue list persistent.
I covered only the very basics of MongoDB, only the capabilities and features that will be useful to build the Issue Tracker application, which is a rather simple CRUD application. In reality, the capabilities of the database as well as the Node.js driver and the mongo shell are vast, and many more features of MongoDB may be required for a complex application. I encourage you to take a look at the MongoDB documentation ( https://docs.mongodb.com/manual/ ) and the Node.js driver documentation ( http://mongodb.github.io/node-mongodb-native/ ) to familiarize yourself with what else the database and the Node.js drivers are capable of.
Now that we have used the essentials of the MERN stack and have a working application, let’s take a break from implementing features and get a bit organized instead. Before the application gets any bigger and becomes unwieldy, let’s modularize the code and use tools to improve our productivity.
We’ll do this in the next chapter, by using Webpack, one of the best tools that can be used to modularize both the front-end and the back-end code.
Answers to Exercises
Exercise: MongoDB Basics
- 1.
As per the mongo shell documentation under "Access the mongo shell Help", you can find that there is a method called help() on many objects, including the cursor object. The way to get help on this is using db.collection.find().help().
But since this is also a JavaScript shell like Node.js, pressing Tab will auto-complete and a double-Tab will show a list of possible completions. Thus, if you assign a cursor to a variable and press Tab twice after typing the variable name and a dot after that, the shell will list the possible completions, and that is a list of methods available on the cursor.
Exercise: MongoDB CRUD Operations
- 1.
This can be done using the $exists operator like this:
> db.employees.find({ "name.middle": { $exists: true } })
- 2.
The filter specification is not a JSON document, because it is not a string. It is a regular JavaScript object, which is why you are able to skip the quotes around the property names. You will also be able to have real Date objects as field values, unlike a JSON string.
- 3.The $unset operator in an update can be used to unset a field (which is actually different from setting it to null). Here is an example:> db.employees.update(({_id: ObjectId("57b1caea3475bb1784747ccb")},{"name.middle": {$unset: null}})
Although we supplied null as the value for $unset, this value is ignored. It can be anything.
- 4.
The 1 indicates an ascending sort order for traversing the index. -1 is used to indicate a descending sort order. This is useful only for compound (aka composite) indexes, because a simple index on one field can be used to traverse the collection in both directions.
Exercise: Schema Initialization
- 1.
The advantage of using the Node.js driver is that there is one way of doing things across the application and the scripts, and the familiarity will help prevent errors. But running the program requires a proper Node.js environment, including npm modules installed, whereas the mongo shell script can be run from anywhere, provided the machine has the mongo shell installed.
- 2.
A search bar is quite helpful when searching for issues. A text index (an index based on the words) on the title field would be useful in this case. We’ll implement a text index toward the end of the book.
Exercise: Reading from MongoDB
- 1.
The connection object is in fact a connection pool. It automatically determines the best thing to do: reuse an existing TCP connection, reestablish a new connection when the connection is broken, etc. Using a global variable (at least, reusing the connection object) is the recommended usage.
- 2.
If the database is unavailable for a short period (less than 30 seconds), the driver retries and reconnects when the database is available again. If the database is unavailable for a longer period, the read throws an error. The driver is also unable to reestablish a connection when the database is restored. The application server needs to be restarted in this case.
The default interval of 30 seconds can be changed using the connection settings reconnectTries or reconnectInterval.
- 3.
One option is to use limit() on the result to limit the return value to a maximum number of records. For example, find().limit(100) returns the first 100 documents. If you were to paginate the output in the UI, you could also use the skip() method to specify where to start the list.
If, on the other hand, you think the client can handle large lists but you don’t want to expend that much memory in the server, you could deal with one document at a time using hasNext() and next() and stream the results back to the client.
Exercise: Writing to MongoDB
- 1.
Adding the _id and returning the object passed in would have worked, so long as you know for a fact that the write was a success and the object was written to the database as is. In most cases, this would be true, but it’s good practice to get the results from the database, as that is the ultimate truth.