Lesson 17. Improving Your Data Models

In this lesson, you take advantage of Mongoose’s schema- and model-creation tools. To start, you improve on your simple model and add properties to the models to restrict what data can be saved to the database. Next, you see how to associate data in a NoSQL database such as MongoDB. At the end, you build out static and instance methods for the model. You can run these methods directly on Mongoose model objects, and create the necessary controller actions for them to work with the application.

This lesson covers

• Adding validations to your models
• Creating static and instance methods for your models
• Testing your models in REPL
• Implementing data associations on multiple models

17.1. Adding validations on the model

So far, you’ve built a model with Mongoose. The model you created is an abstraction from the data, represented as a document, in your MongoDB database. Because of this abstraction, you can create a blueprint of how you want your data to look and behave using Mongoose schemas.

Take a look at the subscriber data model for your recipe application in listing 17.1. The subscriber’s schema lets your application know that it’s looking for three properties of a certain data type. It doesn’t specify, however, whether the properties can be duplicates, if a size limit exist (the ZIP code could be saved as 15 digits, for example), or whether the properties are even required for saving to the database. It won’t be any help to have subscriber records in your database if they’re mostly blank. Next, you add some ways to validate that your properties ensure that your data is consistent.

Listing 17.1. Defining a subscriber schema in subscriber.js

const mongoose = require("mongoose");              1
const subscriberSchema = mongoose.Schema({
  name: String,
  email: String,
  zipCode: Number
});
module.exports = mongoose.model("Subscriber", subscriberSchema);

• 1 Define a subscriberSchema to contain name, email, and zipCode properties.

The schema defined so far works, but it also allows you to save an instance of the Subscriber model without any meaningful data.

To start improving your model, add some Mongoose validators. Validators are rules that you apply to model properties, preventing them from saving to your database unless those rules are met. See the amended schema in listing 17.2. Notice that each model property can have a type assigned directly or a bunch of options passed as a JavaScript object.

You want to require the name property and make it type String. The email property should be required because no two records can have the same email, and it’s also of type String.

Note

In this example, require means that data must exist for the model instance before it can be saved to the database. It’s not the same way I’ve been using the term to require modules.

You also add the lowercase property set to true to indicate that all emails saved to the database are not case-sensitive. Last, the ZIP code property won’t be required, but it has a minimum and maximum number of digits. If a number less than 10000 is entered, the error message "Zip Code too Short" is used. If the number exceeds 99999, or 5 digits in length, you get a generic error from Mongoose, and the data won’t save.

Listing 17.2. Adding validators to the subscriber schema in subscriber.js

const mongoose = require("mongoose");

const subscriberSchema = new mongoose.Schema({
  name: {                                       1
    type: String,
    required: true
  },
  email: {                                      2
    type: String,
    required: true,
    lowercase: true,
    unique: true
  },
  zipCode: {                                   3
    type: Number,
    min: [10000, "Zip code too short"],
    max: 99999
  }
});

• 1 Require the name property.
• 2 Require the email property, and add the lowercase property.
• 3 Set up the zipCode property with a custom error message.

Note

The unique option used on the email property isn’t a validator, but rather a Mongoose schema helper. Helpers are like methods that perform tasks that behave like a validator in this case.

Because the subscriber’s schema defines how instances of the Subscriber model behave, you can also add instance and static methods to the schema. As in traditional objectoriented programming, instance methods operate on an instance (a Mongoose document) of the Subscriber model and are defined by subscriberSchema.methods. Static methods are used for general queries that may relate to many Subscriber instances and are defined with subscriberSchema.statics.

Next, you add two instance methods from listing 17.3 to your recipe application.

getInfo can be called on a Subscriber instance to return the subscriber’s information in one line, which could be useful to get a quick read of the subscribers in your database. findLocalSubscribers works the same way but returns an array of subscribers. This instance method involves a Mongoose query where this refers to the instance of Subscriber on which the method is called. Here, you’re asking for all subscribers with the same ZIP code. exec ensures that you get a promise back instead of needing to add an asynchronous callback here.

Listing 17.3. Adding instance methods to the schema in subscriber.js

subscriberSchema.methods.getInfo = function() {                   1
  return `Name: ${this.name} Email: ${this.email} Zip Code:
  ${this.zipCode}`;
};

subscriberSchema.methods.findLocalSubscribers = function() {      2
  return this.model("Subscriber")
    .find({zipCode: this.zipCode})
    .exec();                                                      3
};

• 1 Add an instance method to get the full name of a subscriber.
• 2 Add an instance method to find subscribers with the same ZIP code.
• 3 Access the Subscriber model to use the find method.

Warning

As of the writing of this book, when using methods with Mongoose, you won’t be able to use ES6 arrow functions without drawbacks. Mongoose makes use of binding this, which is removed with arrow functions. Inside the function, you can use ES6 again.

Note

Recall that you need to export the Subscriber model by using module.exports = mongoose.model("Subscriber", subscriberSchema) after setting up these methods. This line allows you to require the Subscriber model directly by importing this module in another file.

Mongoose provides dozens of other query methods. You could add more methods and validations in subscriber.js, but Mongoose already offers many methods for you to query documents. Table 17.1 lists a few query methods that you may find useful.

Table 17.1. Mongoose queries

Query	Description
find	Returns an array of records that match the query parameters. You can search for all subscribers with the name "Jon" by running Subscriber.find({name: "Jon"}).
findOne	Returns a single record when you don’t want an array of values. Running Subscriber.findOne({name: "Jon"}) results in one returned document.
findById	Allows you to query the database by an ObjectId. This query is your most useful tool for modifying existing records in your database. Assuming that you know a subscriber’s ObjectId, you can run Subscriber.findById("598695b29ff27740c5715265").
remove	Allows you to delete documents in your database by running Subscriber.remove({}) to remove all documents. Be careful with this query. You can also remove specific instances such as subscriber.remove({}).

Note

Each of these queries returns a promise, so you need to use then and catch to handle the resulting data or errors.

For more information about Mongoose queries, visit http://mongoosejs.com/docs/-queries.html.

Before you get to programming the routes and user interface to interact with your new models, try another way to test whether everything is working: REPL. In the next section, you apply the code from earlier in this lesson to a new REPL session.

17.2. Testing models in REPL

To start interacting with your database by using the Subscriber model, you need to go into REPL by typing the node keyword in a new terminal window and adding the lines in listing 17.4. Set up the environment by requiring Mongoose. (You need to be in your project’s directory in terminal for this procedure to work.) Next, set up the connection to MongoDB. Enter the name of your database—in this case, recipe_db.

Listing 17.4. Set up subscriber model in REPL in terminal

const mongoose = require("mongoose"),           1
  Subscriber = require("./models/subscriber");  2
mongoose.connect(                               3
  "mongodb://localhost:27017/recipe_db",
  {useNewUrlParser: true}
);
mongoose.Promise = global.Promise;              4

• 1 Require Mongoose in REPL.
• 2 Assign the Subscriber model to a variable, using the model name and local project file.
• 3 Set up a database connection, using recipe_db.
• 4 Tell Mongoose to use native promises as you did in main.js.

Now you’re all set to test whether your model and its methods work. In REPL, run the commands and queries in listing 17.5 to see whether you’ve set up your model correctly.

Create a new subscriber document with the name "Jon" and email "[email protected]". Try running this line twice. The first time, you should see the saved document logged back to the console. The second time, you should see an error message saying the email already exists in the database, which means that your email validator is working.

Next, set up a variable to which you can assign the following results of your query. Using Mongoose’s findOne query, you’re searching for the document you just created. Then assign the resulting record to your subscriber variable. You can test that this code works by logging the subscriber record or, better, the results of your custom getInfo method on this instance.

The resulting text should read: Name: Jon Email: [email protected] Zip Code: 12345.

Note

Because emails must be unique, you may run into a duplicate key error when saving new records with the same information. In that case, you can run Subscriber .remove({}) to clear all subscriber data from your database.

Listing 17.5. Testing model methods and Mongoose queries in REPL in terminal

Subscriber.create({
  name: "Jon",
  email: "[email protected]",
  zipCode: "12345"
})
  .then(subscriber => console.log(subscriber))
  .catch(error => console.log(error.message));       1

var subscriber;                                      2
Subscriber.findOne({
  name: "Jon"
}).then(result => {
  subscriber = result;                               3
  console.log(subscriber.getInfo());                 4
});

• 1 Create a new subscriber document.
• 2 Set up a variable to hold query results.
• 3 Search for the document you just created.
• 4 Log the subscriber record.

Your terminal console window should resemble the one in figure 17.1.

Figure 17.1. Example response for Mongoose REPL commands

Try to create new records with different content. Check that your validators for the zipCode property are working by creating a new Subscriber with ZIP code 890876 or 123. Then try to delete one or all of your subscriber records directly from REPL.

Next, I show you how to associate this new model with other new models.

Tip

The code in this section can be saved and reused. Add your REPL code to a file called repl.js in your project directory. The next time you open REPL, you can load the contents of this file into the environment. Remember: Node.js runs asynchronously, so if you try to create a record in one command and query for that record immediately afterward, those two commands run virtually at the same time. To avoid any errors, run the commands individually, or nest queries within each other’s then blocks.

17.3. Creating model associations

In unit 3, I discussed how data is structured with MongoDB and how Mongoose acts as a layer over the database to map documents to JavaScript objects. The Mongoose package saves you a lot of time in development by offering methods that make it easy to query the database and generate results quickly in an object-oriented way.

If your background is relational databases, you may be familiar with the ways you can associate data in your applications, as shown in figure 17.2.

Figure 17.2. Relational database associations

Because you’re working with a document-based database, you have no tables—and definitely no join tables. But you do have fairly simple ways to use Mongoose to set up the data relationships laid out in table 17.2.

Table 17.2. Data relationships

Relationship	Description
One-to-one	When one model can have an association to another model. This association could be a User with one Profile; that profile belongs only to the user.
One-to-many	When one model can have many associations to another model, but the other model can have only a single association back to the first model. This association could be a Company with many instances of Employee. In this example, the employees work for only one company, and that company has many employees.
Many-to-many	When many instances of one model can have multiple associations to another model, and vice versa. Many Theatre instances could show the same Movie instances, and each Movie can be traced to many Theatre instances. Typically, a join table is used to map records to one another in a relational database.

If two models are associated in some way—a user has many pictures, an order has a single payment, many classes share multiple enrolled students—you add a property with the associated model’s name, where the type is Schema.Types.ObjectId, the ref attribute is set to the associated model’s name, and Schema is mongoose.Schema. The following code might represent a schema property for users with many pictures: pictures: [{type: Schema.Types .ObjectId, ref: "Picture"}].

Add another model to this recipe application called Course, and associate it with Subscriber. This course model represents recipe courses to choose from in the application. Each course has different food offerings in different locations. Add the code from listing 17.6 to a new model file called course.js in your models folder.

Courses have titles that are required and must not match another course’s title. Courses have a description property to inform users of the site of what the course offers. They also have an items property, which is an array of strings to reflect items and ingredients they include. The zipCode property makes it easier for people to choose the courses that are nearest them.

Listing 17.6. Creating a new schema and model in course.js

const mongoose = require("mongoose");

const courseSchema = new mongoose.Schema({
  title: {                                  1
    type: String,
    required: true,
    unique: true
  },
  description: {
    type: String,
    required: true
  },
  items: [],
  zipCode: {
    type: Number,
    min: [10000, "Zip code too short"],
    max: 99999
  }
});
module.exports = mongoose.model("Course", courseSchema);

• 1 Add properties to the course schema.

You could add a subscribers property to the Course model that stores a reference to the subscribers by each subscriber’s ObjectId, which comes from MongoDB. Then you’d reference the Mongoose model name, Subscriber, like so: subscribers: [{type: mongoose .Schema.Types.ObjectId, ref: "Subscriber"}]. Technically, though, you don’t need the models to reference each other; one model referencing the other is enough. Therefore, add the association on the Subscriber model.

Head back over to subscriber.js, and add the following property to the subscriberSchema: courses: [{type: mongoose.Schema.Types.ObjectId, ref: "Course"}]

Add a courses property to subscribers that stores a reference to each associated course by that course’s ObjectId. The ID comes from MongoDB. Then reference the Mongoose model name, Course.

Note

Notice how the property’s name is plural to reflect the potential to have many associations between subscribers and courses.

If you wanted to restrict subscribers to one course at a time, you could remove the brackets around the property. The brackets signify an array of multiple referenced objects. If a subscriber could sign up for only a single course, the course property would look like the following: course: {type: mongoose.Schema.Types.ObjectId, ref: "Course"}.

In this case, each subscriber could be associated with only a single course. You can think of this as allowing subscribers to sign up for only one course at a time. In a way, this database limitation can also behave like a feature, preventing subscribers from signing up for multiple courses at a time. Nothing prevents different subscribers from signing up for the same course, however, as long as each subscriber has one course association.

To associate two instances of separate models in practice, rely on JavaScript assignment operators. Suppose that you have a subscriber assigned to the variable subscriber1 and a course instance represented as course1. To associate these two instances, assuming the subscriber model can have many course associations, you need to run subscriber1.courses.push(course1). Because subscriber1.courses is an array, use the push method to add the new course.

Alternatively, you can push the ObjectId into subscriber.courses instead of using the whole course object. If course1 has ObjectID "5c23mdsnn3k43k2kuu", for example, your code would look like the following: subscriber1.courses.push("5c23mdsnn3k43k2kuu").

To retrieve course data from a subscriber, you can use the course’s ObjectID and query on the Course model or use the populate method to query the subscriber along with the contents of its associated courses. Your subscriber1 MongoDB document would come with the course1 document nested within it. As a result, you get the ObjectIDs of associated models only.

In the next section, you explore the populate method a little further.

17.4. Populating data from associated models

Population is a method in Mongoose that allows you to get all the documents associated with your model and add them to your query results. When you populate query results, you’re replacing the ObjectIds of associated documents with the documents’ contents. To accomplish this task, you need to chain the populate method to your model queries. Subscriber.populate(subscriber, "courses"), for example, takes all the courses associated with the subscriber object and replaces their ObjectIds with the full Course document in the subscriber’s courses array.

Note

You can find some useful examples at http://mongoosejs.com/docs/populate.html.

With these two models set up, go back to REPL, and test the model associations. See the commands in listing 17.7. First, require the Course model for use in the REPL environment. Set up two variables outside the promise chain scope so that you can assign and use them later. Create a new course instance with values that meet the Course schema requirements. Upon creation, you’re assigning the saved course object to testCourse. Alternatively, if you’ve already created a course, you can get it from the database with Course.findOne({}).then(course => testCourse = course);.

Assuming that you created a subscriber earlier in the lesson, this line pulls a single subscriber from the database and assigns it to testSubscriber. You push the testCourse course into the testSubscriber array of courses. You need to make sure to save the model instance again so that changes take effect in the database. Last, use populate on the Subscriber model to locate all the subscriber’s courses and fill in their data in the subscriber’s courses array.

Listing 17.7. Testing model associations using REPL in terminal

const Course = require("./models/course");                        1
var testCourse, testSubscriber;                                   2
Course.create( {
  title: "Tomato Land",
  description: "Locally farmed tomatoes only",
  zipCode: 12345,
  items: ["cherry", "heirloom"]
}).then(course => testCourse = course);                           3
Subscriber.findOne({}).then(
  subscriber => testSubscriber = subscriber                       4
);
testSubscriber.courses.push(testCourse);                          5
testSubscriber.save();                                            6
Subscriber.populate(testSubscriber, "courses").then(subscriber =>
  console.log(subscriber)                                         7
);

• 1 Require the Course model.
• 2 Set up two variables outside the promise chain.
• 3 Create a new course instance.
• 4 Find a subscriber.
• 5 Push the testCourse course into the courses array of testSubscriber.
• 6 Save the model instance again.
• 7 Use populate on the model.

Note

For these examples, you’re not handling potential errors with catch to keep the code short, though you’ll want to add some error handling while you test. Even a simple catch(error => console.log(error.message)) can help you debug if some error occurs in the promise pipeline.

After running these commands, you should see the results in listing 17.8. Notice that the testSubscriber’s courses array is now populated with the Tomato Land course’s data. To reveal that course’s items, you can log subscriber.courses[0].items in the last REPL populate command you ran.

Listing 17.8. Resulting console log from REPL in terminal

{ _id: 5986b16782180c46c9126287,
  name: "Jon",
  email: "[email protected]",
  zipCode: 12345,
  __v: 1,
  courses:
   [{ _id: 5986b8aad7f31c479a983b42,
       title: "Tomato Land",
       description: "Locally farmed tomatoes only",
       zipCode: 12345,
       __v: 0,
       subscribers: [],
       items: [Array]}]}           1

• 1 Display results for a populated object.

Now that you have access to associated model data, your queries have become more useful. Interested in creating a page to show all subscribers subscribed for the Tomato Land course with ObjectId 5986b8aad7f31c479a983b42? The query you need is Subscriber .find({courses: mongoose.Types.ObjectId("5986b8aad7f31c479a983b42")}).

If you want to run all the examples from this lesson in sequence, you can add the code in listing 17.9 to repl.js, restart your REPL environment by entering node, and load this file by running .load repl.js.

The code in repl.js clears your database of courses and subscribers. Then, in an organized promise chain, a new subscriber is created and saved to an external variable called testSubscriber. The same is done for a course, which is saved to testCourse. At the end, these two model instances are associated, and their association is populated and logged. The commands, in order, demonstrate how powerful REPL can be for testing code.

Listing 17.9. Series of commands in REPL.js

const mongoose = require("mongoose"),
  Subscriber = require("./models/subscriber"),
  Course = require("./models/course");

var testCourse,
  testSubscriber;

mongoose.connect(
  "mongodb://localhost:27017/recipe_db",
  {useNewUrlParser: true}
);

mongoose.Promise = global.Promise;

Subscriber.remove({})                                              1
  .then((items) => console.log(`Removed ${items.n} records!`))
  .then(() => {
    return Course.remove({});
  })
  .then((items) => console.log(`Removed ${items.n} records!`))
  .then(() => {                                                    2
      return Subscriber.create( {
        name: "Jon",
        email: "[email protected]",
        zipCode: "12345"
      });
  })
  .then(subscriber => {
    console.log(`Created Subscriber: ${subscriber.getInfo()}`);
  })
  .then(() => {
    return Subscriber.findOne( {
      name: "Jon"
    });
  })
  .then(subscriber => {
    testSubscriber = subscriber;
    console.log(`Found one subscriber: ${ subscriber.getInfo()}`);
  })
  .then(() => {                                                    3
      return Course.create({
        title: "Tomato Land",
        description: "Locally farmed tomatoes only",
        zipCode: 12345,
        items: ["cherry", "heirloom"]
      });
  })
  .then(course => {
    testCourse = course;
    console.log(`Created course: ${course.title}`);
  })
  .then(() => {                                                    4
      testSubscriber.courses.push(testCourse);
    testSubscriber.save();
  })
  .then( () => {                                                   5
      return Subscriber.populate(testSubscriber, "courses");
  })
  .then(subscriber => console.log(subscriber))
  .then(() => {                                                    6
      return Subscriber.find({ courses: mongoose.Types.ObjectId(
 testCourse._id) });
  })
  .then(subscriber => console.log(subscriber));

• 1 Remove all subscribers and courses.
• 2 Create a new subscriber.
• 3 Create a new course.
• 4 Associate the course with subscriber.
• 5 Populate course document in subscriber.
• 6 Query subscribers where ObjectId is same as course.

Tip

Querying with Mongoose and MongoDB can get complicated. I recommend exploring the sample queries for Mongoose and practicing some of the integrated MongoDB query syntax. You’ll discover the queries that make the most sense to you as you need them in the development process.

In lesson 18, you expand on these associations. You add some controller actions to manage how you interact with your data.

Summary

In this lesson, you learned how to create more-robust Mongoose models. You also created instance methods for your models that can be run from elsewhere in your application on specific model instances. Later, you tested your models for the first time in REPL and created a new Course model with a many-to-many association to your existing Subscriber model. This relationship allows subscribers on the site to show interest in specific recipe courses, allowing you to target your users better by location and interest. In lesson 18, you build a user model along with the CRUD methods that any application needs to manage its data.