Chapter 6. Writing a REST API: Exposing the MongoDB database to the application

This chapter covers

• Rules of REST APIs
• API patterns
• Typical CRUD functions (create, read, update, delete)
• Using Express and Mongoose to interact with MongoDB
• Testing API endpoints

As we come in to this chapter we have a MongoDB database set up, but we can only interact with it through the MongoDB shell. During this chapter we’ll build a REST API so that we can interact with our database through HTTP calls and perform the common CRUD functions: create, read, update, and delete.

We’ll mainly be working with Node and Express, using Mongoose to help with the interactions. Figure 6.1 shows where this chapter fits into the overall architecture.

Figure 6.1. This chapter will focus on building the API that will interact with the database, exposing an interface for the applications to talk to.

We’ll start off by looking at the rules of a REST API. We’ll discuss the importance of defining the URL structure properly, the different request methods (GET, POST, PUT, and DELETE) that should be used for different actions, and how an API should respond with data and an appropriate HTTP status code. Once we have that knowledge under our belts we’ll move on to building our API for Loc8r, covering all of the typical CRUD operations. As we go, we’ll discuss a lot about Mongoose, and get into some Node programming and more Express routing.

Note

If you haven’t yet built the application from chapter 5, you can get the code from GitHub on the chapter-05 branch at github.com/simonholmes/getting-MEAN. In a fresh folder in terminal the following commands will clone it and install the npm module dependencies:

$ git clone -b chapter-05 https://github.com/simonholmes/getting-MEAN.git
$ cd getting-MEAN
$ npm install

6.1. The rules of a REST API

Let’s start with a recap of what a REST API is. From chapter 2 you may remember:

• REST stands for REpresentational State Transfer, which is an architectural style rather than a strict protocol. REST is stateless—it has no idea of any current user state or history.
• API is an abbreviation for application program interface, which enables applications to talk to each other.

So a REST API is a stateless interface to your application. In the case of the MEAN stack the REST API is used to create a stateless interface to your database, enabling a way for other applications to work with the data.

REST APIs have an associated set of standards. While you don’t have to stick to these for your own API it’s generally best to, as it means that any API you create will follow the same approach. It also means you’re used to doing things in the “right” way if you decide you’re going to make your API public.

In basic terms a REST API takes an incoming HTTP request, does some processing, and always sends back an HTTP response, as shown in figure 6.2.

Figure 6.2. A REST API takes incoming HTTP requests, does some processing, and returns HTTP responses.

The standards that we’re going to follow for Loc8r revolve around the requests and the responses.

6.1.1. Request URLs

Request URLs for a REST API have a simple standard. Following this standard will make your API easy to pick up, use, and maintain.

The way to approach this is to start thinking about the collections in your database, as you’ll typically have a set of API URLs for each collection. You may also have a set of URLs for each set of subdocuments. Each URL in a set will have the same basic path, and some may have additional parameters.

Within a set of URLs you need to cover a number of actions, generally based around the standard CRUD operations. The common actions you’ll likely want are

• Create a new item
• Read a list of several items
• Read a specific item
• Update a specific item
• Delete a specific item

Using Loc8r as an example, the database has a Locations collection that we want to interact with. Table 6.1 shows how the URLs and parameters might look for this collection.

Table 6.1. URL paths and parameters for an API to the Locations collection; all have the same base path, and several have the same location ID parameter

Action	URL path	Parameters	Example
Create new location	/locations		http://loc8r.com/api/locations
Read list of locations	/locations		http://loc8r.com/api/locations
Read a specific location	/locations	locationid	http://loc8r.com/api/locations/123
Update a specific location	/locations	locationid	http://loc8r.com/api/locations/123
Delete a specific location	/locations	locationid	http://loc8r.com/api/locations/123

As you can see from table 6.1, each action has the same URL path, and three of them expect the same parameter to specify a location. This poses a very obvious question: How do you use the same URL to initiate different actions? The answer lies in request methods.

6.1.2. Request methods

HTTP requests can have different methods that essentially tell the server what type of action to take. The most common type of request is a GET request—this is the method used when you enter a URL into the address bar of your browser. Another common method is POST, often used when submitting form data.

Table 6.2 shows the methods we’ll be using in our API, their typical use cases, and what you’d expect returned.

Table 6.2. Four request methods used in a REST API

Request method	Use	Response
POST	Create new data in the database	New data object as seen in the database
GET	Read data from the database	Data object answering the request
PUT	Update a document in the database	Updated data object as seen in the database
DELETE	Delete an object from the database	Null

The four HTTP methods that we’ll be using are POST, GET, PUT, and DELETE. If you look at the first word in the “Use” column you’ll notice that there’s a different method for each of the four CRUD operations.

Tip

Each of the four CRUD operations uses a different request method.

The method is important, because a well-designed REST API will often have the same URL for different actions. In these cases it’s the method that tells the server which type of operation to perform. We’ll discuss how to build and organize the routes for this in Express later in this chapter.

So if we take the paths and parameters and map across the appropriate request method we can put together a plan for our API, as shown in table 6.3.

Table 6.3. Request method is used to link the URL to the desired action, enabling the API to use the same URL for different actions

Action	Method	URL path	Parameters	Example
Create new location	POST	/locations		http://loc8r.com/api/locations
Read list of locations	GET	/locations		http://loc8r.com/api/locations
Read a specific location	GET	/locations	locationid	http://loc8r.com/api/locations/123
Update a specific location	PUT	/locations	locationid	http://loc8r.com/api/locations/123
Delete a specific location	DELETE	/locations	locationid	http://loc8r.com/api/locations/123

Table 6.3 shows the paths and methods we’ll use for the requests to interact with the location data. As there are five actions but only two different URL patterns, we can use the request methods to get the desired results.

Loc8r only has one collection right now, so this is our starting point. But the documents in the Locations collection do have reviews as subdocuments, so let’s quickly map those out too.

API URLs for subdocuments

Subdocuments are treated in a similar way, but require an additional parameter. Each request will need to specify the ID of the location, and some will also need to specify the ID of a review. Table 6.4 shows the list of actions and their associated methods, URL paths, and parameters.

Table 6.4. API URL specifications for interacting with subdocuments; each base URL path must contain the ID of the parent document

Action	Method	URL path	Parameters	Example
Create new review	POST	/locations/locationid/reviews	locationid	http://loc8r.com/api/locations/123/reviews
Read a specific review	GET	/locations/locationid/reviews	locationid reviewid	http://loc8r.com/api/locations/123/reviews/abc
Update a specific review	PUT	/locations/locationid/reviews	locationid reviewid	http://loc8r.com/api/locations/123/reviews/abc
Delete a specific review	DELETE	/locations/locationid/reviews	locationid reviewid	http://loc8r.com/api/locations/123/reviews/abc

You may have noticed that for the subdocuments we don’t have a “read a list of reviews” action. This is because we’ll be retrieving the list of reviews as part of the main document. The preceding tables should give you an idea of how to create basic API request specifications. The URLs, parameters, and actions will be different from one application to the next, but the approach should remain consistent.

That’s requests covered. The other half of the flow, before we get stuck in some code, is responses.

6.1.3. Responses and status codes

A good API is like a good friend. If you go for a high-five a good friend will not leave you hanging. The same goes for a good API. If you make a request, a good API will always respond and not leave you hanging. Every single API request should return a response. This contrast is shown in figure 6.3.

Figure 6.3. A good API always returns a response and shouldn’t leave you hanging.

For a successful REST API, standardizing the responses is just as important as standardizing the request format. There are two key components to a response:

• The returned data
• The HTTP status code

Combining the returned data with the appropriate status code correctly should give the requester all of the information required to continue.

Returning data from an API

Your API should return a consistent data format. Typical formats for a REST API are XML and JSON. We’ll be using JSON for our API because it’s the natural fit for the MEAN stack, and it’s more compact than XML, so it can help speed up the response times of an API.

Our API will return one of three things for each request:

• A JSON object containing data answering the request query
• A JSON object containing error data
• A null response

During this chapter we’ll discuss how to do all of these things as we build the Loc8r API. As well as responding with data, any REST API should return the correct HTTP status code.

Using HTTP status codes

A good REST API should return the correct HTTP status code. The status code most people are familiar with is 404, which is what is returned by a web server when a user requests a page that can’t be found. This is probably the most prevalent error code on the internet, but there are dozens of other codes relating to client errors, server errors, redirections, and successful requests. Table 6.5 shows the 10 most popular HTTP status codes and where they might be useful when building an API.

Table 6.5. Most popular HTTP status codes and how they might be used when sending responses to an API request

Status code	Name	Use case
200	OK	A successful GET or PUT request
201	Created	A successful POST request
204	No content	A successful DELETE request
400	Bad request	An unsuccessful GET, POST, or PUT request, due to invalid content
401	Unauthorized	Requesting a restricted URL with incorrect credentials
403	Forbidden	Making a request that isn’t allowed
404	Not found	Unsuccessful request due to an incorrect parameter in the URL
405	Method not allowed	Request method not allowed for the given URL
409	Conflict	Unsuccessful POST request when another object already exists with the same data
500	Internal server error	Problem with your server or the database server

As we go through this chapter and build the Loc8r API we’ll make use of several of these status codes, while also returning the appropriate data.

6.2. Setting up the API in Express

We’ve already got a good idea about the actions we want our API to perform, and the URL paths needed to do so. As we know from chapter 4, to get Express to do something based on an incoming URL request we need to set up controllers and routes. The controllers will do the action, and the routes will map the incoming requests to the appropriate controllers.

We have files for routes and controllers already set up in the application, so we could use those. A better option, though, is to keep the API code separate so that we don’t run the risk of confusion and complication in our application. In fact, this is one of the reasons for creating an API in the first place. Also, by keeping the API code separate it makes it easier to strip it out and put it into a separate application at a future point, should you choose to do so. We really do want easy decoupling here.

So the first thing we want to do here is create a separate area inside the application for the files that will create the API. At the top level of the application create a new folder called app_api. If you’ve been following along and building up the application as you go, this will sit alongside the app_server folder.

This folder will hold everything specific to the API: routes, controllers, and models. When you’ve got this all set up we’ll have a look at some ways to test these API placeholders.

6.2.1. Creating the routes

Like we did with the routes for the main Express application, we’ll have an index.js file in the app_api/routes folder that will hold all of the routes we’ll use in the API. Let’s start by referencing this file in the main application file app.js.

Including the routes in the application

The first step is to tell our application that we’re adding more routes to look out for, and when it should use them. We already have a line in app.js to require the server application routes, which we can simply duplicate and set the path to the API routes as follows:

var routes = require('./app_server/routes/index');
var routesApi = require('./app_api/routes/index');

Next we need to tell the application when to use the routes. We currently have the following line in app.js telling the application to check the server application routes for all incoming requests:

app.use('/', routes);

Notice the '/' as the first parameter. This enables us to specify a subset of URLs for which the routes will apply. For example, we’ll define all of our API routes starting with /api/. By adding the line shown in the following code snippet we can tell the application to use the API routes only when the route starts with /api:

app.use('/', routes);
app.use('/api', routesApi);

Okay, let’s set up these URLs.

Specifying the request methods in the routes

Up to now we’ve only used the GET method in the routes, like in the following code snippet from our main application routes:

router.get('/location', ctrlLocations.locationInfo);

Using the other methods of POST, PUT, and DELETE is as simple as switching out the get with the respective keywords of post, put, and delete. The following code snippet shows an example using the POST method for creating a new location:

router.post('/locations', ctrlLocations.locationsCreate);

Note that we don’t specify /api at the front of the path. We specify in app.js that these routes should only be used if the path starts with /api, so it’s assumed that all routes specified in this file will be prefixed with /api.

Specifying required URL parameters

It’s quite common for API URLs to contain parameters for identifying specific documents or subdocuments—locations and reviews in the case of Loc8r. Specifying these parameters in routes is really simple; you just prefix the name of the parameter with a colon when defining each route.

Say you’re trying to access a review with the ID abc that belongs to a location with the ID 123; you’d have a URL path like this:

/api/locations/123/reviews/abc

Swapping out the IDs for the parameter names (with a colon prefix) gives you a path like this:

/api/locations/:locationid/reviews/:reviewid

With a path like this Express will only match URLs that match that pattern. So a location ID must be specified and must be in the URL between locations/ and /reviews, and a review ID must also be specified at the end of the URL. When a path like this is assigned to a controller the parameters will be available to use in the code, with the names specified in the path, locationid and reviewid in this case.

We’ll review exactly how you get to them in just a moment, but first we need to set up the routes for our Loc8r API.

Defining the Loc8r API routes

Now we know how to set up routes to accept parameters, and we also know what actions, methods, and paths we want to have in our API. So we can combine all of this to create the route definitions for the Loc8r API.

If you haven’t done so yet, you should create an index.js file in the app_api/routes folder. To keep the size of individual files under control we’ll separate the locations and reviews controllers into different files. The following listing shows how the defined routes should look.

Listing 6.1. Routes defined in app_api/routes/locations.js

In this router file we need to require the related controller files. We haven’t created these controller files yet, and will do so in just a moment. This is a good way to approach it, because by defining all of the routes and declaring the associated controller functions here we develop a high-level view of what controllers are needed.

The application now has two sets of routes: the main Express application routes and the new API routes. The application won’t start at the moment though, because none of the controllers referenced by the API routes exist.

6.2.2. Creating the controller placeholders

To enable the application to start we can create placeholder functions for the controllers. These functions won’t really do anything, but they will stop the application from falling over while we’re building the API functionality.

The first step, of course, is to create the controller files. We know where these should be and what they should be called because we’ve already declared them in the app_api/routes folder. We need two new files called locations.js and reviews.js in the app_api/controllers folder.

You can create a placeholder for each of the controller functions as a blank export function, like in the following code snippet. Remember to put each controller into the correct file, depending on whether it’s for a location or a review.

module.exports.locationsCreate = function (req, res) { };

To test the routing and the functions, though, we’ll need to return a response.

Returning JSON from an Express request

When building the Express application we rendered a view template to send HTML to the browser, but with an API we instead want to send a status code and some JSON data. Express makes this task really easy with the following commands:

You can use these two commands in the placeholder functions to test the success, as shown in the following code snippet:

module.exports.locationsCreate = function (req, res) {
  res.status(200);
  res.json({"status" : "success"});
};

Returning JSON and a response status is a very common task for an API, so it’s a good idea to move these two statements into their own function. It also makes the code easier to test. So create a sendJsonResponse function in both controller files and call this from each of the controller placeholders as follows:

Now we can send a JSON response and associated status code with a single line. We’ll use this a lot in our API!

6.2.3. Including the model

It’s vitally important that the API can talk to the database; without it the API isn’t going to be much use! To do this with Mongoose, we first need to require Mongoose into the controller files, and then bring in the Location model. Right at the top of the controller files, above all of the placeholder functions, add the following two lines:

var mongoose = require('mongoose');
var Loc = mongoose.model('Location');

The first line gives the controllers access to the database connection, and the second brings in the Location model so that we can interact with the Locations collection.

If we take a look at the file structure of our application, we see the app_api/models folder containing the database connection and the Mongoose setup is inside the app_server folder. But it’s the API that’s dealing with the database, not the main Express application. If the two applications were separate the model would be kept part of the API, so that’s where it should live.

Just move the app_api/models folder from the app_server folder into the app_api folder, giving the folder structure like that shown in figure 6.4.

Figure 6.4. Folder structure of the application at this point: app_api has models, controllers, and routes, and app_server has views, controllers, and routes

We need to tell the application that we’ve moved the app_api/models folder, of course, so we need to update the line in app.js that requires the model to point to the correct place:

require('./app_api/models/db');

With that done, the application should start again and still connect to your database. The next question is, how can we test the API?

6.2.4. Testing the API

You can quickly test the GET routes in your browser by heading to the appropriate URL, such as http://localhost:3000/api/locations/1234. You should see the success response being delivered to the browser as shown in figure 6.5.

Figure 6.5. Testing a GET request of the API in the browser

This is okay for testing GET requests, but it doesn’t get you very far with POST, PUT, and DELETE methods. There are a few tools to help you test API calls like this, but my current favorite is an extension for Chrome called Postman REST Client.

Postman enables you to test API URLs with a number of different request methods, allowing you to specify additional query string parameters or form data. After you click the Send button it will make a request to the URL you’ve specified and display the response data and status code.

Figure 6.6 shows a screenshot of Postman making a PUT request to the same URL as before.

Figure 6.6. Using the Postman REST Client in Chrome to test a PUT request to the API

It’s a good idea to get Postman or another REST client up and running now. You’ll need to use one a lot during this chapter as we build up a REST API. Let’s get started by using the GET requests to read data from MongoDB.

6.3. GET methods: Reading data from MongoDB

GET methods are all about querying the database and returning some data. In our routes for Loc8r we have three GET requests doing different things, as listed in table 6.6.

Table 6.6. Three GET requests of the Loc8r API

Action	Method	URL path	Parameters	Example
Read list of locations	GET	/locations		http://loc8r.com/api/locations
Read a specific location	GET	/locations	locationid	http://loc8r.com/api/locations/123
Read a specific review	GET	/locations/locationid/reviews	locationid reviewid	http://loc8r.com/api/locations/123/reviews/abc

We’ll look at how to find a single location first, because it provides a good introduction to the way Mongoose works. Next we’ll locate a single document using an ID, and then we’ll expand into searching for multiple documents.

6.3.1. Finding a single document in MongoDB using Mongoose

Mongoose interacts with the database through its models, which is why we imported the Locations model as Loc at the top of the controller files. A Mongoose model has several associated methods to help manage the interactions as noted in the following sidebar.

For finding a single database document with a known ID in MongoDB, Mongoose has the findById method.

Applying the findById method to the model

The findById method is relatively straightforward, accepting a single parameter, the ID to look for. As it’s a model method, it’s applied to the model like this:

Loc.findById(locationid)

This will not start the database query operation; it just tells the model what the query will be. To start the database query Mongoose models have an exec method.

Running the query with the exec method

The exec method executes the query and passes a callback function that will run when the operation is complete. The callback function should accept two parameters, an error object and the instance of the found document. As it’s a callback function the names of these parameters can be whatever you like.

The methods can be chained as follows:

This approach ensures that the database interaction is asynchronous, and therefore doesn’t block the main Node process.

Tip

If you’re not 100% comfortable with callbacks, scopes, and where the variables come from, take a look online at appendix D, section D.4, “Understanding JavaScript Callbacks.”

Using the findById method in a controller

The controller we’re working with to find a single location by ID is locationsReadOne, in the locations.js file in app_api/controllers.

We know the basic construct of the operation: apply the findById and exec methods to the Location model. To get this working in the context of the controller we need to do two things:

• Get the locationid parameter from the URL and pass it to the findById method.
• Provide an output function to the exec method.

Express makes it really easy to get the URL parameters we defined in the routes. The parameters are held inside a params object attached to the request object. With our route being defined like so

app.get('/api/locations/:locationid', ctrlLocations.locationsReadOne);

we can access the locationid parameter from inside the controller like this:

req.params.locationid

For the output function we can use the sendJsonResponse function that we created earlier. Putting this all together gives us the following:

And now we have a very basic API controller. You can try it out by getting the ID of one of the locations in MongoDB and going to the URL in your browser, or by calling it in Postman. To get one of the ID values you can run the command db.locations.find() in the Mongo shell and it will list all of the locations you have, which will each include the _id value. When you’ve put the URL together the output should be a full location object as stored in MongoDB; you should see something like figure 6.7.

Figure 6.7. Basic controller for finding a single location by ID returns a JSON object to the browser if the ID is found

Did you try out the basic controller? Did you put an invalid location ID into the URL? If you did you’ll have seen that you got nothing back. No warning, no message, just a 200 status telling you that everything is okay, but no data returned.

Catching errors

The problem with that basic controller is that it only outputs a success response, regardless of whether it was successful or not. This isn’t good behavior for an API. A good API should respond with an error code when something goes wrong.

To respond with error messages the controller needs to be set up to trap potential errors and send an appropriate response. Error trapping in this fashion typically involves if statements. Every if statement must either have a corresponding else statement, or it must include a return statement.

Tip

Your API code must never leave a request unanswered.

With our basic controller there are three errors we need to trap:

• The request parameters don’t include locationid.
• The findById method doesn’t return a location.
• The findById method returns an error.

The status code for an unsuccessful GET request is 404. Bearing this in mind the final code for the controller to find and return a single location looks like the following listing.

Listing 6.2. locationsReadOne controller

Listing 6.2 uses both of the two methods of trapping with if statements. Error trap 1 uses an if to check that the params object exists in the request object, and that the params object contains a locationid value. This loop is closed off with an else for when either the params object or the locationid value isn’t found. Error trap 2 and error trap 3 both use an if to check for an error returned by Mongoose. Each if includes a return statement, which will prevent any following code in the callback scope from running. If no error was found the return statement is ignored and the code moves on to send the successful response .

Each of these traps provides a response for success and failure, leaving no room for the API to leave a requester hanging. If you wish you can also throw in a few console.log statements so that it’s easier to track what’s going on in terminal; the source code in GitHub will have some.

Figure 6.8 shows the difference between a successful request and a failed request, using the Postman extension in Chrome.

Figure 6.8. Testing successful (left) and failed (right) API responses using Postman

That’s one complete API route dealt with. Now it’s time to look at the second GET request to return a single review.

6.3.2. Finding a single subdocument based on IDs

To find a subdocument you first have to find the parent document like we’ve just done to find a single location by its ID. Once you’ve found the document you can look for a specific subdocument.

This means that we can take the locationsReadOne controller as the starting point and add a few modifications to create the reviewsReadOne controller. These modifications are

• Accept and use an additional reviewid URL parameter.
• Select only the name and reviews from the document, rather than having MongoDB return the entire document.
• Look for a review with a matching ID.
• Return the appropriate JSON response.

To do these things we can use a couple of new Mongoose methods.

Limiting the paths returned from MongoDB

When you retrieve a document from MongoDB you don’t always need the full document; sometimes you just want some specific data. Limiting the data being passed around is also better for bandwidth consumption and speed.

Mongoose does this through a select method chained to the model query. For example, the following code snippet will tell MongoDB that we only want to get the name and the reviews of a location:

Loc
  .findById(req.params.locationid)
  .select('name reviews')
  .exec();

The select method accepts a space-separated string of the paths we want to retrieve.

Using Mongoose to find a specific subdocument

Mongoose also offers a helper method for finding a subdocument by ID. Given an array of subdocuments Mongoose has an id method that accepts the ID you want to find. The id method will return the single matching subdocument, and it can be used as follows:

In this code snippet a single review would be returned to the review variable in the callback.

Adding some error trapping and putting it all together

Now we’ve got the ingredients needed to make the reviewsReadOne controller. Starting with a copy of the locationsReadOne controller we can make the modifications required to return just a single review.

The following listing shows the reviewsReadOne controller in review.js (modifications in bold).

Listing 6.3. Controller for finding a single review

When this is saved and ready you can test it using Postman again. You need to have correct ID values, which you can get directly from MongoDB via the Mongo shell. The command db.locations.find() will return all of the locations and their reviews. You can test what happens if you put in a false ID for a location or a review, or try a review ID from a different location.

6.3.3. Finding multiple documents with geospatial queries

The homepage of Loc8r should display a list of locations based on the user’s current geographical location. MongoDB and Mongoose have some special geospatial query methods to help find nearby places.

Here we’ll use the Mongoose method geoNear to find a list of locations close to a specified point, up to a specified maximum distance. geoNear is a model method that accepts three parameters:

• A geoJSON geographical point
• An options object
• A callback function

The following code snippet shows the basic construct:

Loc.geoNear(point, options, callback);

Unlike the findById method, geoNear doesn’t have an exec method. Instead, geoNear is executed immediately and the code to run on completion is sent through in the callback.

Constructing a geoJSON point

The first parameter of the geoNear method is a geoJSON point. A geoJSON point is a simple JSON object containing a latitude and a longitude in an array. The construct for a geoJSON point is shown in the following code snippet:

The route set up here to get a list of locations doesn’t have the coordinates in the URL parameters, meaning that they’ll have to be specified in a different way. A query string is ideal for this data type, meaning that the request URL will look more like this:

api/locations?lng=-0.7992599&lat=51.378091

Express, of course, gives you access to the values in a query string, putting them into a query object attached to the request object—for example, req.query.lng. The longitude and latitude values will be strings when retrieved, but they need to be added to the point object as numbers. JavaScript’s parseFloat function can see to this. Putting it all together, the following code snippet shows how to get the coordinates from the query string and create the geoJSON point required by the geoNear function:

Naturally, this controller will not work yet as options and callback are both currently undefined. We’ll work on these now, starting with the options.

Adding required query options to geoNear

The geoNear method only has one required option: spherical. This determines whether the search will be done based on a spherical object or a flat plane. It’s generally accepted these days that Earth is round, so we’ll set the spherical option to be true.

In creating an object to hold the options we have the following code snippet:

var geoOptions = {
  spherical: true
};

Now the search will be based on coordinates on a sphere.

Limiting geoNear results by number

You’ll often want to look after the API server—and the responsiveness seen by end users—by limiting the number of results when returning a list. In the geoNear method adding an option called num does this. You simply specify the maximum number of results you want to have returned.

The following code snippet shows this added to the previous geoOptions object, limiting the size of the returned data set to 10 objects:

var geoOptions = {
  spherical: true,
  num: 10
};

Now the search will bring back no more than the 10 closest results.

Limiting geoNear results by distance

When returning location-based data, another way to keep the processing of the API under control is to limit the list of results by distance from the central point. In theory, this is just a case of adding another option called maxDistance. The challenge is that MongoDB does the calculations in radians rather than meters or miles, and expects the maxDistance to be supplied as radians. This allows MongoDB to easily do these calculations on any size sphere, not just Earth, but that doesn’t help us here.

The calculations to convert between physical distances and radians are quite straightforward, as shown in figure 6.9.

Figure 6.9. Relationship between distance and radians, and how to convert one to the other given a known radius

The radius of Earth is 6,371 kilometers, or 3,959 miles, but we’ll be using kilometers in this book because, frankly, they’re easier to work with! Given this information we can create a function called theEarth, exposing two methods to make the calculations for us. The following code snippet should go near the top of the API controller locations.js file, just after the Mongoose is required and the model set up:

Now we have some reusable functions for making the distance calculations.

Tip

If this pattern of coding isn’t familiar to you take a look at appendix D, section D.5, “Writing Modular JavaScript.”

We can now add the maxDistance value to the options, and add these options to the controller as follows:

That’s the last of the options we need for our geoNear database search, so now it’s time to start working with the output.

Looking at the geoNear output

The completion callback for the geoNear method has three parameters, in this order:

1. An error object
2. A results object
3. A stats object

With a successful query the error object will be undefined, the results object will contain an array of results, and the stats object will contain information about the query, like time taken, number of documents scanned, the average distance, and the maximum distance of the documents returned. We’ll start by working with a successful query before adding in the error trapping.

Following a successful geoNear query MongoDB returns an array of objects. Each object contains a distance value and a returned document from the database. In other words, MongoDB doesn’t add the distance to the data. The following code snippet shows an example of the returned data, truncated for brevity:

[{
  dis: 0.002532674663406363,
  obj: {
    name: 'Starcups',
    address: '125 High Street, Reading, RG6 1PS'
  }
}]

This array only has one object, but a successful query is likely to have several objects returned at once. The geoNear method actually returns the entire document in the obj object. There are two problems here:

• The API shouldn’t return more data than necessary.
• We want to return the distance in a meaningful way (not radians) as an integral part of the returned data set.

So rather than simply sending the returned data back as the response there’s some processing to do first.

Processing the geoNear output

Before the API can send a response we need to make sure it’s sending the right thing, and only what’s needed. We know what data is needed by the homepage listing as we’ve already built the homepage controller in app_server/controllers/location.js. The homelist function sends a number of location objects like the following example:

{
 name: 'Starcups',
 address: '125 High Street, Reading, RG6 1PS',
 rating: 3,

 facilities: ['Hot drinks', 'Food', 'Premium wifi'],
 distance: '100m'
}

To create an object along these lines from the results, we simply need to loop through the results and push the relevant data into a new array. This processed data can then be returned with a status 200 response. The following code snippet shows how this might look:

If you test this API route with Postman—remembering to add longitude and latitude coordinates to the query string—you’ll see something like figure 6.10.

Figure 6.10. Testing the location list route in Postman should give a 200 status and a list of results, depending on the geographical coordinates sent in the query string.

If you test this by sending coordinates too far away from the test data you should still get a 200 status, but the returned array will be empty.

Adding the error trapping

Once again we’ve started by building the success functionality, and now we need to add in some error traps to make sure that the API always sends the appropriate response.

The traps we need to set should check that

• The parameters have all been sent correctly.
• The geoNear function hasn’t returned an error.

The listing on the next page shows the final controller all put together, including these error traps.

Listing 6.4. Locations list controller locationsListByDistance

This completes the GET requests that our API needs to service, so moving forward it’s time to tackle the POST requests.

6.4. POST methods: Adding data to MongoDB

POST methods are all about creating documents or subdocuments in the database, and then returning the saved data as confirmation. In the routes for Loc8r we have two POST requests doing different things, listed in Table 6.7.

Table 6.7. Two POST requests of the Loc8r API

Action	Method	URL path	Parameters	Example
Create new location	POST	/locations		http://api.loc8r.com/locations
Create new review	POST	/locations/locationid/reviews	locationid	http://api.loc8r.com/locations/123/reviews

POST methods work by taking form data posted to them and adding it to the database. In the same way that URL parameters are accessed using req.params and query strings are accessed via req.query, Express controllers access posted form data via req.body.

Let’s make a start by looking at how to create documents.

6.4.1. Creating new documents in MongoDB

In the database for Loc8r each location is a document, so this is what we’ll be creating in this section. Mongoose really couldn’t make the process of creating MongoDB documents much easier for you. You take your model, apply the create method, and send it some data and a callback function. This is the minimal construct, as it would be attached to our Loc model:

So that’s pretty simple. There are two main steps to the creation process:

1. Take the posted form data and use it to create a JavaScript object that matches the schema.
2. Send an appropriate response in the callback depending on the success or failure of the create operation.

Looking at step 1 we already know that we can get data sent to us in a form by using req.body, and step 2 should be pretty familiar by now. So let’s jump straight into the code. The following listing shows the full locationsCreate controller for creating a new document.

Listing 6.5. Complete controller for creating a new location

This shows how easy it can be to create a new document in MongoDB and save some data. For the sake of brevity we’ve limited the openingTimes array to two entries, but this could easily be extended, or better yet put into a loop checking for the existence of the values.

You might also notice that there’s no rating being set. Remember in the schema that we set a default of 0, as in the following snippet:

rating: {type: Number, "default": 0, min: 0, max: 5},

This is applied when the document is created, setting the initial value to be 0. Something else about this code might be shouting out at you. There’s no validation!

Validating the data using Mongoose

This controller has no validation code inside it, so what’s to stop somebody from entering loads of empty or partial documents? Again, we started this off in the Mongoose schemas. In the schemas we set a required flag to true in a few of the paths. When this flag is set, Mongoose will not send the data to MongoDB.

Given the following base schema for locations, for example, we can see that name and coords are both required fields:

var locationSchema = new mongoose.Schema({
  name: {type: String, required: true},
  address: String,
  rating: {type: Number, "default": 0, min: 0, max: 5},
  facilities: [String],
  coords: {type: [Number], index: '2dsphere', required: true},
  openingTimes: [openingTimeSchema],
  reviews: [reviewSchema]
});

If either of these fields is missing, the create method will raise an error and not attempt to save the document to the database.

Testing this API route in Postman looks like figure 6.11. Note that the method is set to post, and that the data type selected (above the list of names and values) is x-www-form-urlencoded.

Figure 6.11. Testing a POST method in Postman, ensuring that the method and form data settings are correct

6.4.2. Creating new subdocuments in MongoDB

In the context of Loc8r locations, reviews are subdocuments. Subdocuments are created and saved through their parent document. Put another way, to create and save a new subdocument you have to

1. Find the correct parent document.
2. Add a new subdocument.
3. Save the parent document.

Finding the correct parent isn’t a problem as we’ve already done that, and can use it as the skeleton for the next controller, reviewsCreate. When we’ve found the parent we can call an external function to do the next step, as shown in the following listing.

Listing 6.6. Controller for creating a review

This isn’t doing anything particularly new; we’ve seen it all before. By putting in a call to a new function we can keep the code neater by reducing the amount of nesting and indentation, and also make it easier to test.

Adding and saving a subdocument

Having found the parent document, and retrieved the existing list of subdocuments, we then need to add a new one. Subdocuments are essentially arrays of objects, and the easiest way to add a new object to an array is to create the data object and use the JavaScript push method. The following code snippet demonstrates this:

location.reviews.push({
  author: req.body.author,
  rating: req.body.rating,
  reviewText: req.body.reviewText
});

This is getting posted form data, hence using req.body.

Once the subdocument has been added, the parent document must be saved because subdocuments cannot be saved on their own. To save a document Mongoose has a model method save, which expects a callback with an error parameter and a returned object parameter. The following code snippet shows this in action:

The document returned by the save method is the full parent document, not just the new subdocument. To return the correct data in the API response—that is, the subdocument—we need to retrieve the last subdocument from the array .

When adding documents and subdocuments you need to keep in mind any impact this may have on other data. In Loc8r adding a review will add a new rating. This new rating will impact the overall rating for the document. So on the successful save of a review we’ll call another function to update the average rating.

Putting everything we have together in the doAddReview function, plus a little extra error trapping, gives us the following listing.

Listing 6.7. Adding and saving a subdocument

Updating the average rating

Calculating the average rating isn’t particularly complicated, so we won’t dwell on it too long. The steps are

1. Find the correct document given a provided ID.
2. Loop through the review subdocuments adding up the ratings.
3. Calculate the average rating value.
4. Update the rating value of the parent document.
5. Save the document.

Turning this list of steps into code gives us something along the lines of the following listing, which should be placed in the reviews.js controller file along with the review-based controllers.

Listing 6.8. Calculating and updating the average rating

You might have noticed that we’re not sending any JSON response here, and that’s because we’ve already sent it. This entire operation is asynchronous and doesn’t need to impact sending the API response confirming the saved review.

Adding a review isn’t the only time we’ll need to update the average rating. This is why it makes extra sense to make these functions accessible from the other controllers, and not tightly coupled to the actions of creating a review.

What we’ve just done here offers a sneak peak at using Mongoose to update data in MongoDB, so let’s now move on to the PUT methods of the API.

6.5. PUT methods: Updating data in MongoDB

PUT methods are all about updating existing documents or subdocuments in the database, and then returning the saved data as confirmation. In the routes for Loc8r we have two PUT requests doing different things, listed in table 6.8.

Table 6.8. Two PUT requests of the Loc8r API for updating locations and reviews

Action	Method	URL path	Parameters	Example
Update a specific location	PUT	/locations	locationid	http://loc8r.com/api/locations/123
Update a specific review	PUT	/locations/locationid/reviews	locationid reviewid	http://loc8r.com/api/locations/123/reviews/abc

PUT methods are similar to POST methods because they work by taking form data posted to them. But instead of using the data to create new documents in the database, PUT methods use the data to update existing documents.

6.5.1. Using Mongoose to update a document in MongoDB

In Loc8r we might want to update a location to add new facilities, change the open times, or amend any of the other data. The approach to updating data in a document is probably starting to look familiar, following these steps:

1. Find the relevant document.
2. Make some changes to the instance.
3. Save the document.
4. Send a JSON response.

This approach is made possible by the way that an instance of a Mongoose model maps directly to a document in MongoDB. When your query finds the document you get a model instance. If you make changes to this instance and then save it, Mongoose will update the original document in the database with your changes.

Using the Mongoose save method

We’ve actually already seen this in action, when updating the average rating value. The save method is applied to the model instance that the find function returns. It expects a callback with the standard parameters of an error object and a returned data object.

A cut-down skeleton of this approach is shown in the following code snippet:

Here we can clearly see the separate steps of finding, updating, saving, and responding. Fleshing out this skeleton into the locationsUpdateOne controller with some error trapping and the data we want to save gives us the following listing.

Listing 6.9. Making changes to an existing document in MongoDB

There’s clearly a lot more code here, now that it’s fully fleshed out, but we can still quite easily identify the key steps of the update process.

The eagle-eyed among us may have noticed something strange in the select statement:

.select('-reviews -rating')

Previously we’ve used the select method to say which columns we do want to select. By adding a dash in front of a path name we’re stating that we don’t want to retrieve it from the database. So this select statement says to retrieve everything except the reviews and the rating.

6.5.2. Updating an existing subdocument in MongoDB

Updating a subdocument is exactly the same as updating a document, with one exception. After finding the document you then have to find the correct subdocument to make your changes. After this, the save method is applied to the document, not the subdocument. So the steps to updating an existing subdocument are

1. Find the relevant document.
2. Find the relevant subdocument.
3. Make some changes to the subdocument.
4. Save the document.
5. Send a JSON response.

For Loc8r the subdocuments we’re updating are reviews, so when a review is changed we’ll have to remember to recalculate the average rating. That’s the only additional thing we’ll need to add in, above and beyond the five steps. The following listing shows this all put into place in the reviewsUpdateOne controller.

Listing 6.10. Updating a subdocument in MongoDB

The five steps for updating are clear to see in this listing: find the document, find the subdocument, make changes, save, and respond. Once again a lot of the code here’s error trapping, but it’s vital for creating a stable and responsive API. You really don’t want to save incorrect data, send the wrong responses, or delete data you don’t want to. Speaking of deleting data, let’s move on to the final of the four API methods we’re using: DELETE.

6.6. DELETE method: Deleting data from MongoDB

The DELETE method is, unsurprisingly, all about deleting existing documents or subdocuments in the database. In the routes for Loc8r we have a DELETE request for deleting a location, and another for deleting a review. The details are listed in Table 6.9.

Table 6.9. Two DELETE requests of the Loc8r API for deleting locations and reviews

Action	Method	URL path	Parameters	Example
Delete a specific location	DELETE	/locations	locationid	http://loc8r.com/api/locations/123
Delete a specific review	DELETE	/locations/locationid/reviews	locationid reviewid	http://loc8r.com/api/locations/123/reviews/abc

We’ll start by taking a look at deleting documents.

6.6.1. Deleting documents in MongoDB

Mongoose makes deleting a document in MongoDB extremely simple by giving us the method findByIdAndRemove. This method expects just a single parameter—the ID of the document to be deleted.

The API should respond with a 404 in case of an error and a 204 in case of success. The following listing shows this all in place in the locationsDeleteOne controller.

Listing 6.11. Deleting a document from MongoDB given an ID

That’s the quick and easy way to delete a document, but you can break it into a two-step process and find it then delete it if you prefer. This does give you the chance to do something with the document before deleting if you need to. This would look like the following code snippet:

Loc
  .findById(locationid)
  .exec(
    function (err, location) {
      // Do something with the document
      Loc.remove(function(err, location){

       // Confirm success or failure
     });
   }
);

So there’s an extra level of nesting there, but with it comes an extra level of flexibility should you need it.

6.6.2. Deleting a subdocument from MongoDB

The process for deleting a subdocument is no different from the other work we’ve done with subdocuments—everything is managed through the parent document. The steps for deleting a subdocument are

1. Find the parent document.
2. Find the relevant subdocument.
3. Remove the subdocument.
4. Save the parent document.
5. Confirm success or failure of operation.

Actually deleting the subdocument itself is really easy, as Mongoose gives us another helper method. You’ve already seen that we can find a subdocument by its ID with the id method like this:

location.reviews.id(reviewid)

Mongoose allows you to chain a remove method to the end of this statement like so:

location.reviews.id(reviewid).remove()

This will delete the subdocument from the array. Remember, of course, that the parent document will need saving after this to persist the change back to the database. Putting all the steps together—with a load of error trapping—into the reviewsDeleteOne controller looks like the following listing.

Listing 6.12. Finding and deleting a subdocument from MongoDB

Again, most of the code here’s error trapping; there are seven possible responses the API could give and only one of them is the successful one. Actually deleting the subdocument is really easy; you just have to make absolutely sure that you’re deleting the right one.

As we’re deleting a review here, which will have a rating associated to it, we also have to remember to call the updateAverageRating function to recalculate the average rating for the location. This should only be called if the delete operation is successful, of course.

And that is it. We’ve now built a REST API in Express and Node that can accept GET, POST, PUT, and DELETE HTTP requests to perform CRUD operations on a MongoDB database.

6.7. Summary

In this chapter we’ve covered

• The best practices for creating a REST API, including URLs, request methods, and response codes
• How the POST, GET, PUT, and DELETE HTTP request methods map onto common CRUD operations
• Mongoose helper methods for creating the helper methods
• Interacting with the data through Mongoose models, and how one instance of the model maps directly to one document in the database
• Managing subdocuments through their parent documents because you cannot access or save a subdocument in isolation
• Making the API robust by checking for any possible errors you can think of, so that a request is never left unanswered

Coming up next in chapter 7 we’re going to see how to use this API from inside the Express application, finally making the Loc8r site database-driven!