Node.js uses the event-driven nature of JavaScript to support non-blocking operations in the platform, a feature that enables its excellent efficiency. JavaScript is an event-driven language, which means that you register code to specific events, and that code will be executed once the event is emitted. This concept allows you to seamlessly execute asynchronous code without blocking the rest of the program from running.

To understand this better, take a look at the following Java code example:

System.out.print("What is your name?");
String name = System.console().readLine();
System.out.print("Your name is: " + name);

In this example, the program executes the first and second lines, but any code after the second line will not be executed until the user inputs their name. This is synchronous programming, where I/O operations block the rest of the program from running. However, this is not how JavaScript works.

Because it was originally written to support browser operations, JavaScript was designed around browser events. Even though it has vastly evolved since its early days, the idea was to allow the browser to take the HTML user events and delegate them to JavaScript code. Let's have a look at the following HTML example:

<span>What is your name?</span>
<input type="text" id="nameInput">
<input type="button" id="showNameButton" value="Show Name">
<script type="text/javascript">
var showNameButton = document.getElementById('showNameButton');

showNameButton.addEventListener('click', function() {
    alert(document.getElementById('nameInput').value);
});
// Rest of your code...
</script>

In the preceding example, we have a textbox and a button. When the button is pressed, it will alert the value inside the textbox. The main function to watch here is the addEventListener() method. As you can see it takes two arguments: the name of the event and an anonymous function that will run once the event is emitted. We usually refer to arguments of the latter kind as a callback function. Notice that any code after the addEventListener() method will execute accordingly regardless of what we write in the callback function.

As simple as this example is, it illustrates well how JavaScript uses events to execute a set of commands. Since the browser is single-threaded, using synchronous programming in this example would freeze everything else in the page, which would make every web page extremely unresponsive and impair the web experience in general. Thankfully, this is not how it works. The browser manages a single thread to run the entire JavaScript code using an inner loop, commonly referred to as the event loop. The event loop is a single-threaded loop that the browser runs infinitely. Every time an event is emitted, the browser adds it to an event queue. The loop will then grab the next event from the queue in order to execute the event handlers registered to that event. After all of the event handlers are executed, the loop grabs the next event, executes its handlers, grabs the next event, and so on. You can see a visual representation of this process in the following diagram:

The event loop cycle

While the browser usually deals with user-generated events (such as button clicks), Node.js has to deal with various types of events that are generated from different sources.

When developing web server logic, you will probably notice a lot of your system resources are wasted on blocking code. For instance, let's observe the following PHP database interactions:

$output = mysql_query('SELECT * FROM Users');
echo($output);

Our server will try querying the database that will then perform the select statement and return the result to the PHP code, which will eventually output the data as a response. The preceding code blocks any other operation until it gets the result from the database. This means the process, or more commonly, the thread, will stay idle, consuming system resources while it waits for other processes.

To solve this issue, many web platforms have implemented a thread pool system that usually issues a single thread per connection. This kind of multithreading may seem intuitive at first, but has some significant disadvantages, as follows:

This is tolerable while developing one-sided web applications, where the browser makes a quick request that ends with a server response. But, what happens when you want to build real-time applications that keep a long-living connection between the browser and the server? To understand the real-life consequences of these design choices, take a look at the following graphs. They present a famous performance comparison between Apache, which is a blocking web server, and NGINX, which uses a non-blocking event loop. The following screenshot shows concurrent request handling in Apache versus Nginx (http://blog.webfaction.com/2008/12/a-little-holiday-present-10000-reqssec-with-nginx-2/):

In the preceding screenshot, you can see how Apache's request handling ability is degrading much faster than Nginx's. But, an even clearer impact can be seen in the following screenshot, where you can witness how Nginx's event loop architecture affects memory consumption:

Concurrent connections impact on memory allocation in Apache versus Nginx (http://blog.webfaction.com/2008/12/a-little-holiday-present-10000-reqssec-with-nginx-2/)

As you can see from the results, using event-driven architecture will help you dramatically reduce the load on your server while leveraging JavaScript's asynchronous behavior in building your web application. This approach is made possible thanks to a simple design pattern, which is called closure by JavaScript developers.

CommonJS is a project started in 2009 to standardize the way of working with JavaScript outside the browser. The project has evolved since then to support a variety of JavaScript issues, including the global namespace issue, which was solved through a simple specification of how to write and include isolated JavaScript modules.

The CommonJS standards specify the following three key components when working with modules:

In Node's CommonJS module implementation, each module is written in a single JavaScript file and has an isolated scope that holds its own variables. The author of the module can expose any functionality through the exports object. To understand it better, let's say we created a module file named hello.js that contains the following code snippet:

var message = 'Hello';

exports.sayHello = function(){
  console.log(message);
}

Also, let's say we created an application file named server.js, which contains the following lines of code:

var hello = require('./hello');
hello.sayHello();

In the preceding example, you have the hello module, which contains a variable named message. The message variable is self-contained in the hello module, which only exposes the sayHello() method by defining it as a property of the exports object. Then, the application file loads the hello module using the require() method, which will allow it to call the sayHello() method of the hello module.

A different approach to creating modules is exposing a single function using the module.exports pointer. To understand this better, let's revise the preceding example. A modified hello.js file should look as follows:

module.exports = function() {
  var message = 'Hello';

  console.log(message);
}

Then, the module is loaded in the server.js file as follows:

var hello = require('./hello');
hello();

In the preceding example, the application file uses the hello module directly as a function instead of using the sayHello() method as a property of the hello module.

The CommonJS module standard allows the endless extension of the Node.js platform while preventing the pollution of Node's core; without it, the Node.js platform would become a mess of conflicts. However, not all modules are the same, and while developing a Node application, you will encounter several types of modules.

Core modules are modules that were compiled into the Node binary. They come prebundled with Node and are documented in great detail in its documentation. The core modules provide most of the basic functionalities of Node, including filesystem access, HTTP and HTTPS interfaces, and much more. To load a core module, you just need to use the require method in your JavaScript file. An example code, using the fs core module to read the content of the environment hosts file, would look like the following code snippet:

fs = require('fs');

fs.readFile('/etc/hosts', 'utf8', function (err, data) { 
  if (err) { 
    return console.log(err);
  }

  console.log(data);
});

When you require the fs module, Node will find it in the core modules folder. You'll then be able to use the fs.readFile() method to read the file's content and print it in the command-line output.

To use third-party modules, you can just require them as you would normally require a core module. Node will first look for the module in the core modules folder and then try to load the module from the module folder inside the node_modules folder. For instance, to use the express module, your code should look like the following code snippet:

var express = require('express');
var app = express();

Node will then look for the express module in the node_modules folder and load it into your application file, where you'll be able to use it as a method to generate the express application object.

In previous examples, you saw how Node loads modules directly from files. These examples describe a scenario where the files reside in the same folder. However, you can also place your modules inside a folder and load them by providing the folder path. Let's say you moved your hello module to a modules folder. The application file would have to change, so Node would look for the module in the new relative path:

var hello = require('./modules/hello');

Note that the path can also be an absolute path, as follows:

var hello = require('/home/projects/first-example/modules/hello');

Node will then look for the hello module in that path.

Although this is not common with developers that aren't writing third-party Node modules, Node also supports the loading of folder modules. Requiring folder modules is done in the same way as file modules, as follows:

var hello = require('./modules/hello');

Now, if a folder named hello exists, Node will go through that folder looking for a package.json file. If Node finds a package.json file, it will try parsing it, looking for the main property, with a package.json file that looks like the following code snippet:

{
  "name" : "hello",
  "version" : "1.0.0",
  "main" : "./hello-module.js"
}

Node will try to load the ./hello/hello-module.js file. If the package.json file doesn't exist or the main property isn't defined, Node will automatically try to load the ./hello/index.js file.

Node.js modules have been found to be a great solution to write complex JavaScript applications. They have helped developers organize their code better, while NPM and its third-party modules registry helped them to find and install one of the many third-party modules created by the community. Ryan Dahl's dream of building a better web framework ended up as a platform that supports a huge variety of solutions. But the dream was not abandoned; it was just implemented as a third-party module named express.

Connect is a module built to support interception of requests in a more modular approach. In the first web server example, you learned how to build a simple web server using the http module. If you wish to extend this example, you'd have to write code that manages the different HTTP requests sent to your server, handles them properly, and responds to each request with the correct response.

Connect creates an API exactly for that purpose. It uses a modular component called middleware, which allows you to simply register your application logic to predefined HTTP request scenarios. Connect middleware are basically callback functions, which get executed when an HTTP request occurs. The middleware can then perform some logic, return a response, or call the next registered middleware. While you will mostly write custom middleware to support your application needs, Connect also includes some common middleware to support logging, static file serving, and more.

The way a Connect application works is by using an object called dispatcher. The dispatcher object handles each HTTP request received by the server and then decides, in a cascading way, the order of middleware execution. To understand Connect better, take a look at the following diagram:

The preceding diagram illustrates two calls made to the Connect application: the first one should be handled by a custom middleware and the second is handled by the static files middleware. Connect's dispatcher initiates the process, moving on to the next handler using the next() method, until it gets to middleware responding with the res.end() method, which will end the request handling.

In the next Lesson, you'll create your first Express application, but Express is based on Connect's approach, so in order to understand how Express works, we'll begin with creating a Connect application.

In your working folder, create a file named server.js that contains the following code snippet:

var connect = require('connect');
var app = connect();
app.listen(3000); 

console.log('Server running at http://localhost:3000/');

As you can see, your application file is using the connect module to create a new web server. However, Connect isn't a core module, so you'll have to install it using NPM. As you already know, there are several ways of installing third-party modules. The easiest one is to install it directly using the npm install command. To do so, use your command-line tool, and navigate to your working folder. Then execute the following command:

$ npm install connect

NPM will install the connect module inside a node_modules folder, which will enable you to require it in your application file. To run your Connect web server, just use Node's CLI and execute the following command:

$ node server

Node will run your application, reporting the server status using the console.log() method. You can try reaching your application in the browser by visiting http://localhost:3000. However, you should get a response similar to what is shown in the following screenshot:

Connect application's empty response

What this response means is that there isn't any middleware registered to handle the GET HTTP request. This means two things:

Connect middleware

Connect middleware is just JavaScript function with a unique signature. Each middleware function is defined with the following three arguments:

• req: This is an object that holds the HTTP request information
• res: This is an object that holds the HTTP response information and allows you to set the response properties
• next: This is the next middleware function defined in the ordered set of Connect middleware

When you have a middleware defined, you'll just have to register it with the Connect application using the app.use() method. Let's revise the previous example to include your first middleware. Change your server.js file to look like the following code snippet:

var connect = require('connect');
var app = connect();

var helloWorld = function(req, res, next) {
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World');
};
app.use(helloWorld);

app.listen(3000);
console.log('Server running at http://localhost:3000/');

Then, start your connect server again by issuing the following command in your command-line tool:

$ node server

Try visiting http://localhost:3000 again. You will now get a response similar to that in the following screenshot:

Connect application's response

Congratulations, you've just created your first Connect middleware!

Let's recap. First, you added a middleware function named helloWorld(), which has three arguments: req, res, and next. In your middleware, you used the res.setHeader() method to set the response Content-Type header and the res.end() method to set the response text. Finally, you used the app.use() method to register your middleware with the Connect application.

var connect = require('connect');
var app = connect();

var logger = function(req, res, next) {
  console.log(req.method, req.url);

  next();
};

var helloWorld = function(req, res, next) {
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World');
};

app.use(logger);
app.use(helloWorld);
app.listen(3000);

console.log('Server running at http://localhost:3000/');

$ node server

Mounting Connect middleware

As you may have noticed, the middleware you registered responds to any request regardless of the request path. This does not comply with modern web application development because responding to different paths is an integral part of all web applications. Fortunately, Connect middleware supports a feature called mounting, which enables you to determine which request path is required for the middleware function to get executed. Mounting is done by adding the path argument to the app.use() method. To understand this better, let's revisit our previous example. Modify your server.js file to look like the following code snippet:

var connect = require('connect');
var app = connect();

var logger = function(req, res, next) {
  console.log(req.method, req.url);

  next();
};

var helloWorld = function(req, res, next) {
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World');
};

var goodbyeWorld = function(req, res, next) {
  res.setHeader('Content-Type', 'text/plain');
  res.end('Goodbye World');
};

app.use(logger);
app.use('/hello', helloWorld);
app.use('/goodbye', goodbyeWorld);
app.listen(3000);

console.log('Server running at http://localhost:3000/');

A few things have been changed in the previous example. First, you mounted the helloWorld() middleware to respond only to requests made to the /hello path. Then, you added another (a bit morbid) middleware called goodbyeWorld() that will respond to requests made to the /goodbye path. Notice how, as a logger should do, we left the logger() middleware to respond to all the requests made to the server. Another thing you should be aware of is that any requests made to the base path will not be responded by any middleware because we mounted the helloWorld() middleware to a specific path.

Connect is a great module that supports various features of common web applications. Connect middleware is super simple as it is built with a JavaScript style in mind. It allows the endless extension of your application logic without breaking the nimble philosophy of the Node platform. While Connect is a great improvement over writing your web application infrastructure, it deliberately lacks some basic features you're used to having in other web frameworks. The reason lies in one of the basic principles of the Node community: create your modules lean and let other developers build their modules on top of the module you created. The community is supposed to extend Connect with its own modules and create its own web infrastructures. In fact, one very energetic developer named TJ Holowaychuk, did it better than most when he released a Connect-based web framework known as Express.