Lesson 5. Handling incoming data

In lesson 4, I introduced you to the web server and showed how you can create one with Node.js. Every time a user visits a URL that leads to your application, a request is made, and each request must be processed by the code you write. In this lesson, you learn how to gather and process some of the information in these requests. You also build application routes—code logic to match requests with appropriate responses.

This lesson covers

• Collecting and processing request data
• Submitting a POST request with the curl command
• Building a web application with basic routes

5.1. Reworking your server code

To start this lesson, rearrange the code from lesson 4 to get a better idea of how the server is behaving. Create a new project called second_server within its own project directory, and inside, add a new main.js file.

Note

In this lesson and following lessons, I expect you to initialize your Node.js application with npm init and to follow the guidance in lesson 4 to create a package.json file.

In your code, you have a server object that has a callback function, (req, res) ⇒ {}, which is run every time a request is made to the server. With your server running, if you visit localhost:3000 in your browser and refresh the page, that callback function is run twice—once on every refresh.

Note

req and res represent the HTTP request and response. You can use any variable names here. Keep the order in mind; request always comes before response in this method.

In other words, upon receiving a request, the server passes a request and response object to a function where you can run your code. Another way to write the code for this server is shown in listing 5.1. The server fires the code in a callback function when a request event is triggered. When a user visits your application’s web page, the code within the braces runs. Then the server prepares a response by assigning a response code of 200 and defines the type of content in the response as HTML. Last, the server sends the HTML content within the parentheses and simultaneously closes the connection with the client.

Listing 5.1. A simple server with a request event listener in main.js

const port = 3000,
  http = require("http"),
  httpStatus = require("http-status-codes"),
  app = http.createServer();

app.on("request", (req, res) => {                                  1
  res.writeHead(httpStatus.OK, {
    "Content-Type": "text/html"
  });                                                              2

  let responseMessage = "<h1>This will show on the screen.</h1>";
  res.end(responseMessage);                                        3

});

app.listen(port);
console.log(`The server has started and is listening on port number:
 ${port}`);

• 1 Listen for requests.
• 2 Prepare a response.
• 3 Respond with HTML.

Run node main in terminal and visit http://localhost:3000/ in your web browser to view the response containing one line of HTML on the screen.

Note

You may need to reinstall the http-status-codes package again for this new project by runinng npm i http-status-codes -save-dev.

It’s great to have some content on the screen, but you want to modify the content based on the type of request you get. If the user is visiting the contact page or submitting a form they filled out, for example, they’ll want to see different content on the screen. The first step is determining which HTTP method and URL were in the headers of the request. In the next section, you look at these request attributes.

5.2. Analyzing request data

Routing is a way for your application to determine how to respond to a requesting client. Some routes are designed by matching the URL in the request object. That method is how you’re going to build your routes in this lesson.

Each request object has a url property. You can view which URL the client requested with req.url. Test this property and two other properties by logging them to your console. Add the code in the next listing to the app.on(“request”) code block.

Listing 5.2. Logging request data in main.js

console.log(req.method);        1
console.log(req.url);           2
console.log(req.headers);       3

• 1 Log the HTTP method used.
• 2 Log the request URL.
• 3 Log request headers.

Because some objects in the request can have within them other nested objects, convert the objects to more-readable strings by using JSON.stringify within your own custom wrapper function, getJSONString, as shown in listing 5.3. This function takes a JavaScript object as an argument and returns a string. Now you can change your log statements to use this function. You can print the request method, for example, by using console.log (`Method: ${getJSONString(req.method)}`);.

Listing 5.3. Logging request data in main.js

const getJSONString = obj => {
  return JSON.stringify(obj, null, 2);      1
};

• 1 Convert JavaScript object to string.

When you restart your server, run main.js again, and access http://localhost:3000 in your web browser, you’ll notice in your terminal window information indicating that a GET request was made to the / URL (the home page), followed by that request’s header data. Try entering a different URL, such as http://localhost:3000/testing or http://localhost: 3000/contact. Notice that you still get the same HTML text on the browser, but your console continues to log the URLs you type in the browser.

The types of requests you’re largely dealing with are GET requests. If you were building an application with forms for users to fill out, though, your server should be able to process that form data and respond to the user to let them know that the data has been received.

The request object, like most objects in Node.js, can also listen for events, similarly to the server. If someone makes a POST request to the server (trying to send data to the server), the content of that POST lives in the request’s body. Because a server never knows how much data is being sent, posted data comes into the http server via chunks.

Note

Data chunks allow information to stream into and out of a server. Instead of waiting for a large set of information to arrive at the server, Node.js allows you to work with parts of that information as it arrives via the ReadableStream library.

To collect all the posted data with a server, you need to listen for each piece of data received and arrange the data yourself. Luckily, the request listens for a specific data event. req.on(“data”) is triggered when data is received for a specific request. You need to define a new array, body, outside this event handler and sequentially add the data chunks to it as they arrive at the server. Notice the exchange of posted data in figure 5.1. When all the data chunks are received, they can be collected as a single data item.

Figure 5.1. A web server collects posted data and arranges it.

Within the app.on(“request”) code block, add the new request event handlers in listing 5.4 to read incoming data. In this code example, every time a request is made to the server, you execute the code in the callback function. An array is created and referred to as body, and every time data from the request is received, you process it in another callback function. The received data is added to the body array. When the transmission of data is complete, you execute code in a third callback function. The body array is turned into a String of text, and the request’s contents are logged to your console.

Listing 5.4. Handling posted request data in main.js

app.on("request", (req, res) => {                          1
  var body = [];                                           2
  req.on("data", (bodyData) => {                           3
    body.push(bodyData);                                   4
  });
  req.on("end", () => {                                    5
    body = Buffer.concat(body).toString();                 6
    console.log(`Request Body Contents: ${body}`);
  });                                                      7

  console.log(`Method: ${getJSONString(req.method)}`);
  console.log(`URL: ${getJSONString(req.url)}`);
  console.log(`Headers: ${getJSONString(req.headers)}`);

  res.writeHead(httpStatus.OK, {
    "Content-Type": "text/html"
  });

  let responseMessage = "<h1>This will show on the screen.</h1>";
  res.end(responseMessage);
});
app.listen(port);
console.log(`The server has started and is listening on port number:
 ${port}`);

• 1 Listen for requests.
• 2 Create an array to hold chunk contents.
• 3 Process it in another callback function.
• 4 Add received data to the body array.
• 5 Run code when data transmission ends.
• 6 Convert the body array to a String of text.
• 7 Log the request’s contents to your console.

With this added code, your application is prepared to receive posted data collected into an array and converted back to String format. When an event is triggered, indicating that some chunk of data reached the server, you handle that data by adding the chunk (represented as a Buffer object) to an array. When the event indicating the request’s connected has ended, you follow up by taking all the array’s contents and turn them into text you can read. To test this process, try sending a POST request to your server from terminal.

Because you haven’t built a form yet, you can use a curl command. Follow these steps:

1. With your web server running in one terminal window, open a new terminal window.
2. In the new window. run the following command: curl --data "username= Jon&password=secret" http://localhost:3000

Tip

curl is a simple way of mimicking a browser’s request to a server. Using the curl keyword, you can use different flags, such as –data, to send information to a server via a POST request.

Note

If you’re a Windows user, before you install curl on your computer, install the software and package manager called Chocolatey (https://chocolatey.org/install). Then you can run choco install curl in your command line.

In the first terminal window, you should see the contents of the request’s body logged to the screen, letting you know that a request was received and processed by your server (figure 5.2).

Figure 5.2. Results of running a curl command

Tip

For a more user-friendly interface for submitting data to your application, install Insomnia (https://insomnia.rest/download/).

In lesson 8, you learn about simpler ways to handle request contents. For now, try to control what type of response you write back to the client based on the URL and method in the request.

5.3. Adding routes to a web application

A route is a way of determining how an application should respond to a request made to a specific URL. An application should route a request to the home page differently from a request to submit login information.

You’ve established that a user can make a request to your web server; from there, you can evaluate the type of request and prompt an appropriate response. Consider your simple HTTP web server code, which so far has one response to any request. This example accepts any request made to the server (localhost) at port 3000 and responds with a line of HTML on the screen.

Listing 5.5. Simple server example in main.js

const port = 3000,
  http = require("http"),
  httpStatus = require("http-status-codes"),
  app = http
    .createServer((req, res) => {
      res.writeHead(httpStatus.OK, {
        "Content-Type": "text/html"
      });
      let responseMessage = "<h1>Welcome!</h1>";
      res.end(responseMessage);                    1
    })
    .listen(port);

• 1 Respond with HTML to every request.

As a first web application, this application is a great accomplishment, but you need to start building an application with more functionality. If this project were a legitimate application live on the internet, for example, you might want to show content based on what the user is looking for. If the user wants to see an information page, you may want them to find that information at the /info URL (http://localhost:3000/info). Right now, if users visit those URLs, they’ll be greeted by the same HTML welcome line.

The next step is checking the client’s request and basing the response body on that request’s contents. This structure is otherwise known as application routing. Routes identify specific URL paths, which can be targeted in the application logic and which allow you to specify the information to be sent to the client. Creating these routes is necessary for a fully integrated application experience.

Duplicate the simple_server project folder with a new name: simple_routes. Then add a few routes to the main.js file, as shown in listing 5.6.

You set up a mapping of routes to responses called routeResponseMap. When a request is made to http://localhost:3000/info, you check whether the request’s URL has a match in routeResponseMap and respond with an info page heading. When a request is made to http://localhost:3000/contact, you respond with a contact page heading. To all other requests, you respond with a generic greeting.

Listing 5.6. Simple routing in a web server in main.js

const routeResponseMap = {                              1
  "/info": "<h1>Info Page</h1>",
  "/contact": "<h1>Contact Us</h1>",
  "/about": "<h1>Learn More About Us.</h1>",
  "/hello": "<h1>Say hello by emailing us here</h1>",
  "/error": "<h1>Sorry the page you are looking for is not here.</h1>"
};

const port = 3000,
  http = require("http"),
  httpStatus = require("http-status-codes"),
  app = http.createServer((req, res) => {
    res.writeHead(200, {
      "Content-Type": "text/html"
    });
    if (routeResponseMap[req.url]) {                    2
      res.end(routeResponseMap[req.url]);
    } else {
      res.end("<h1>Welcome!</h1>");                     3
    }
  });

app.listen(port);
console.log(`The server has started and is listening on port number:
 ${port}`);

• 1 Define mapping of routes with responses.
• 2 Check whether a request route is defined in the map.
• 3 Respond with default HTML.

With the additions to your code, you can differentiate between a couple of URLs and offer different content accordingly. You’re still not concerned with the HTTP method used in the request, but you can check whether the user was searching for the/info route or the /contact route. Users can more intuitively determine what URLs they need to type to get to that page’s expected content.

Give the code a try. Save the code in listing 5.6 in a project file called main.js, and run that file in terminal. Then try accessing http://localhost:3000/info or http://localhost: 3000/contact in your web browser. Any other URL should result in the original default welcome HTML line.

To mimic heavy processing or external calls made by your server, you can add the code in the following listing to a route to manually delay your response to the client.

Listing 5.7. Route with a timer in main.js

setTimeout(() => res.end(routeResponseMap[req.url]), 2000);    1

• 1 Wrap a response with setTimeout to delay the response manually.

If you run this file again, you’ll notice that the page’s load time is approximately two seconds longer. You have full control of what code is executed and what content is served to your user. Keep this fact in mind: as your application grows, your web pages’ response times will naturally be longer.

Look at the browser screenshot for the /contact URL in figure 5.3.

Figure 5.3. Browser view for the /contact URL

Summary

In this lesson, you learned how to handle request content, respond with viewable HTML, and build a server route. By identifying a request’s contents, you can process posted data from a request and separate response content based on targeted URLs. The creation of routes shapes your application logic. As a web application expands, its routes expand with it, and so do the types of content that it’s able to deliver.

In the next lesson, I talk about serving individual HTML files, images, and web-page styles.