In this Lesson will cover the proper way of building your first Node.js web application. You'll go through the basics of JavaScript event-driven nature and how to utilize it to build Node.js applications. You'll also learn about the Node.js module system and how to build your first Node.js web application. You'll then proceed to the Connect module and learn about its powerful middleware approach. By the end of this Lesson, you'll know how to use Connect and Node.js to build simple yet powerful web applications. We'll cover the following topics:
Downloading the example code:
The code files for all the four parts of the course are available at https://github.com/shinypoojary09/AngularJS_Course.git.
At JSConf EU 2009, a developer named Ryan Dahl went onstage to present his project named Node.js. Starting in 2008, Dahl looked at the current web trends and discovered something odd in the way web applications worked. The introduction of the AJAX technology a few years earlier transformed static websites into dynamic web applications, but the fundamental building block of web development didn't follow this trend. The problem was that web technologies didn't support two-way communication between the browser and the server. The test case he used was the Flickr upload file feature, where the browser was unable to know when to update the progress bar as the server could not inform it of how much of the file was uploaded.
Dahl's idea was to build a web platform that would gracefully support the push of data from the server to the browser, but it wasn't that simple. When scaling to common web usage, the platform had to support hundreds (and sometimes thousands) of ongoing connections between the server and the browser. Most web platforms used expensive threads to handle requests, which meant keeping a fair amount of idle threads in order to keep the connection alive. So Dahl used a different approach. He understood that using non-blocking sockets could save a lot in terms of system resources and went as far as proving this could be done using C. Given that this technique could be implemented in any programming language and the fact that Dahl thought working with non-blocking C code was a tedious task, he decided to look for a better programming language.
When Google announced Chrome and its new V8 JavaScript engine in late 2008, it was obvious that JavaScript could run faster than before—a lot faster. V8's greatest advantage over other JavaScript engines was the compiling of JavaScript code to native machine code before executing it. This and other optimizations made JavaScript a viable programming language capable of executing complex tasks. Dahl noticed that and decided to try a new idea: non-blocking sockets in JavaScript. He took the V8 engine, wrapped it with the already solid C code, and created the first version of Node.js.
After a very warm response from the community, he went on to expand the Node core. The V8 engine wasn't built to run in a server environment, so Node.js had to extend it in a way that made more sense in a server context. For example, browsers don't usually need access to the filesystem, but when running server code, this becomes essential. The result was that Node.js wasn't just a JavaScript execution engine, but a platform capable of running complex JavaScript applications that were simple to code, highly efficient, and easily scalable.
Node.js uses the event-driven nature of JavaScript to support non-blocking operations in the platform, a feature that enables its excellent efficiency. JavaScript is an event-driven language, which means that you register code to specific events, and that code will be executed once the event is emitted. This concept allows you to seamlessly execute asynchronous code without blocking the rest of the program from running.
To understand this better, take a look at the following Java code example:
System.out.print("What is your name?"); String name = System.console().readLine(); System.out.print("Your name is: " + name);
In this example, the program executes the first and second lines, but any code after the second line will not be executed until the user inputs their name. This is synchronous programming, where I/O operations block the rest of the program from running. However, this is not how JavaScript works.
Because it was originally written to support browser operations, JavaScript was designed around browser events. Even though it has vastly evolved since its early days, the idea was to allow the browser to take the HTML user events and delegate them to JavaScript code. Let's have a look at the following HTML example:
<span>What is your name?</span> <input type="text" id="nameInput"> <input type="button" id="showNameButton" value="Show Name"> <script type="text/javascript"> var showNameButton = document.getElementById('showNameButton'); showNameButton.addEventListener('click', function() { alert(document.getElementById('nameInput').value); }); // Rest of your code... </script>
In the preceding example, we have a textbox and a button. When the button is pressed, it will alert the value inside the textbox. The main function to watch here is the addEventListener()
method. As you can see it takes two arguments: the name of the event and an anonymous function that will run once the event is emitted. We usually refer to arguments of the latter kind as a callback
function. Notice that any code after the addEventListener()
method will execute accordingly regardless of what we write in the callback
function.
As simple as this example is, it illustrates well how JavaScript uses events to execute a set of commands. Since the browser is single-threaded, using synchronous programming in this example would freeze everything else in the page, which would make every web page extremely unresponsive and impair the web experience in general. Thankfully, this is not how it works. The browser manages a single thread to run the entire JavaScript code using an inner loop, commonly referred to as the event loop. The event loop is a single-threaded loop that the browser runs infinitely. Every time an event is emitted, the browser adds it to an event queue. The loop will then grab the next event from the queue in order to execute the event handlers registered to that event. After all of the event handlers are executed, the loop grabs the next event, executes its handlers, grabs the next event, and so on. You can see a visual representation of this process in the following diagram:
While the browser usually deals with user-generated events (such as button clicks), Node.js has to deal with various types of events that are generated from different sources.
When developing web server logic, you will probably notice a lot of your system resources are wasted on blocking code. For instance, let's observe the following PHP database interactions:
$output = mysql_query('SELECT * FROM Users'); echo($output);
Our server will try querying the database that will then perform the select
statement and return the result to the PHP code, which will eventually output the data as a response. The preceding code blocks any other operation until it gets the result from the database. This means the process, or more commonly, the thread, will stay idle, consuming system resources while it waits for other processes.
To solve this issue, many web platforms have implemented a thread pool system that usually issues a single thread per connection. This kind of multithreading may seem intuitive at first, but has some significant disadvantages, as follows:
This is tolerable while developing one-sided web applications, where the browser makes a quick request that ends with a server response. But, what happens when you want to build real-time applications that keep a long-living connection between the browser and the server? To understand the real-life consequences of these design choices, take a look at the following graphs. They present a famous performance comparison between Apache, which is a blocking web server, and NGINX, which uses a non-blocking event loop. The following screenshot shows concurrent request handling in Apache versus Nginx (http://blog.webfaction.com/2008/12/a-little-holiday-present-10000-reqssec-with-nginx-2/):
In the preceding screenshot, you can see how Apache's request handling ability is degrading much faster than Nginx's. But, an even clearer impact can be seen in the following screenshot, where you can witness how Nginx's event loop architecture affects memory consumption:
As you can see from the results, using event-driven architecture will help you dramatically reduce the load on your server while leveraging JavaScript's asynchronous behavior in building your web application. This approach is made possible thanks to a simple design pattern, which is called closure by JavaScript developers.
Closures are functions that refer to variables from their parent environment. Using the closure pattern enables variables from the parent()
function to remain bound to the closure. Let's take a look at the following example:
function parent() { var message = "Hello World"; function child() { alert (message); } child(); } parent();
In the preceding example, you can see how the child()
function has access to a variable defined in the parent()
function. But this is a simple example, so let's see a more interesting one:
function parent() { var message = 'Hello World'; function child() { alert (message); } return child; } var childFN = parent() childFN();
This time, the parent()
function returned the child()
function, and the child()
function is called after the parent()
function has already been executed. This is counterintuitive to some developers because usually the parent()
function's local variables should only exist while the function is being executed. This is what closures are all about! A closure is not only the function, but also the environment in which the function was created. In this case, the childFN()
is a closure object that consists of the child()
function and the environment variables that existed when the closure was created, including the message
variable.
Closures are very important in asynchronous programming because JavaScript functions are first-class objects that can be passed as arguments to other functions. This means that you can create a callback
function and pass it as an argument to an event handler. When the event will be emitted, the function will be invoked, and it will be able to manipulate any variable that existed when the callback
function was created even if its parent function was already executed. This means that using the closure pattern will help you utilize event-driven programming without the need to pass the scope state to the event handler.
JavaScript has turned out to be a powerful language with some unique features that enable efficient yet maintainable programming. Its closure pattern and event-driven behavior have proven to be very helpful in real-life scenarios, but like all programming languages, it isn't perfect, and one of its major design flaws is the sharing of a single global namespace.
To understand the problem, we need to go back to JavaScript's browser origins. In the browser, when you load a script into your web page, the engine will inject its code into an address space that is shared by all the other scripts. This means that when you assign a variable in one script, you can accidently overwrite another variable already defined in a previous script. While this could work with a small code base, it can easily cause conflicts in larger applications, as errors will be difficult to trace. It could have been a major threat for Node.js evolution as a platform, but luckily a solution was found in the CommonJS modules standard.
CommonJS is a project started in 2009 to standardize the way of working with JavaScript outside the browser. The project has evolved since then to support a variety of JavaScript issues, including the global namespace issue, which was solved through a simple specification of how to write and include isolated JavaScript modules.
The CommonJS standards specify the following three key components when working with modules:
require()
: This method is used to load the module into your code.exports
: This object is contained in each module and allows you to expose pieces of your code when the module is loaded.module
: This object was originally used to provide metadata information about the module. It also contains the pointer of an exports
object as a property. However, the popular implementation of the exports
object as a standalone object literally changed the use case of the module
object.In Node's CommonJS module implementation, each module is written in a single JavaScript file and has an isolated scope that holds its own variables. The author of the module can expose any functionality through the exports
object. To understand it better, let's say we created a module file named hello.js
that contains the following code snippet:
var message = 'Hello'; exports.sayHello = function(){ console.log(message); }
Also, let's say we created an application file named server.js
, which contains the following lines of code:
var hello = require('./hello'); hello.sayHello();
In the preceding example, you have the hello
module, which contains a variable named message
. The message
variable is self-contained in the hello
module, which only exposes the sayHello()
method by defining it as a property of the exports
object. Then, the application file loads the hello
module using the require()
method, which will allow it to call the sayHello()
method of the hello
module.
A different approach to creating modules is exposing a single function using the module.exports
pointer. To understand this better, let's revise the preceding example. A modified hello.js
file should look as follows:
module.exports = function() { var message = 'Hello'; console.log(message); }
Then, the module is loaded in the server.js
file as follows:
var hello = require('./hello'); hello();
In the preceding example, the application file uses the hello
module directly as a function instead of using the sayHello()
method as a property of the hello
module.
The CommonJS module standard allows the endless extension of the Node.js platform while preventing the pollution of Node's core; without it, the Node.js platform would become a mess of conflicts. However, not all modules are the same, and while developing a Node application, you will encounter several types of modules.
Core modules are modules that were compiled into the Node binary. They come prebundled with Node and are documented in great detail in its documentation. The core modules provide most of the basic functionalities of Node, including filesystem access, HTTP and HTTPS interfaces, and much more. To load a core module, you just need to use the require
method in your JavaScript file. An example code, using the fs
core module to read the content of the environment hosts file, would look like the following code snippet:
fs = require('fs');
fs.readFile('/etc/hosts', 'utf8', function (err, data) {
if (err) {
return console.log(err);
}
console.log(data);
});
When you require the fs
module, Node will find it in the core modules folder. You'll then be able to use the fs.readFile()
method to read the file's content and print it in the command-line output.
To learn more about Node's core modules, it is recommended that you visit the official documentation at http://nodejs.org/api/.
To use third-party modules, you can just require them as you would normally require a core module. Node will first look for the module in the core modules folder and then try to load the module from the module
folder inside the node_modules
folder. For instance, to use the express
module, your code should look like the following code snippet:
var express = require('express'); var app = express();
Node will then look for the express
module in the node_modules
folder and load it into your application file, where you'll be able to use it as a method to generate the express
application object.
In previous examples, you saw how Node loads modules directly from files. These examples describe a scenario where the files reside in the same folder. However, you can also place your modules inside a folder and load them by providing the folder path. Let's say you moved your hello
module to a modules
folder. The application file would have to change, so Node would look for the module in the new relative path:
var hello = require('./modules/hello');
Note that the path can also be an absolute path, as follows:
var hello = require('/home/projects/first-example/modules/hello');
Node will then look for the hello
module in that path.
Although this is not common with developers that aren't writing third-party Node modules, Node also supports the loading of folder modules. Requiring folder modules is done in the same way as file modules, as follows:
var hello = require('./modules/hello');
Now, if a folder named hello
exists, Node will go through that folder looking for a package.json
file. If Node finds a package.json
file, it will try parsing it, looking for the main
property, with a package.json
file that looks like the following code snippet:
{ "name" : "hello", "version" : "1.0.0", "main" : "./hello-module.js" }
Node will try to load the ./hello/hello-module.js
file. If the package.json
file doesn't exist or the main
property isn't defined, Node will automatically try to load the ./hello/index.js
file.
Node.js modules have been found to be a great solution to write complex JavaScript applications. They have helped developers organize their code better, while NPM and its third-party modules registry helped them to find and install one of the many third-party modules created by the community. Ryan Dahl's dream of building a better web framework ended up as a platform that supports a huge variety of solutions. But the dream was not abandoned; it was just implemented as a third-party module named express
.
Node.js is a platform that supports various types of applications, but the most popular kind is the development of web applications. Node's style of coding depends on the community to extend the platform through third-party modules; these modules are then built upon to create new modules, and so on. Companies and single developers around the globe are participating in this process by creating modules that wrap the basic Node APIs and deliver a better starting point for application development.
There are many modules to support web application development but none as popular as the Connect module. The Connect module delivers a set of wrappers around the Node.js low-level APIs to enable the development of rich web application frameworks. To understand what Connect is all about, let's begin with a basic example of a basic Node web server. In your working folder, create a file named server.js
, which contains the following code snippet:
var http = require('http'); http.createServer(function(req, res) { res.writeHead(200, { 'Content-Type': 'text/plain' }); res.end('Hello World'); }).listen(3000); console.log('Server running at http://localhost:3000/');
To start your web server, use your command-line tool, and navigate to your working folder. Then, run the node CLI tool and run the server.js
file as follows:
$ node server
Now open http://localhost:3000
in your browser, and you'll see the Hello World response.
So how does this work? In this example, the http
module is used to create a small web server listening to the 3000
port. You begin by requiring the http
module and use the createServer()
method to return a new server
object. The listen()
method is then used to listen to the 3000
port. Notice the callback
function that is passed as an argument to the createServer()
method.
The callback
function gets called whenever there's an HTTP request sent to the web server. The server
object will then pass the req
and res
arguments, which contain the information and functionality needed to send back an HTTP response. The callback
function will then do the following two steps:
writeHead()
method of the response
object. This method is used to set the response HTTP headers. In this example, it will set the Content-Type
header value to text/plain
. For instance, when responding with HTML, you just need to replace text/plain
with text/html
.end()
method of the response object. This method is used to finalize the response. The end()
method takes a single string argument that it will use as the HTTP response body. Another common way of writing this is to add a write()
method before the end()
method and then call the end()
method, as follows:res.write('Hello World'); res.end();
This simple application illustrates the Node coding style where low-level APIs are used to simply achieve certain functionality. While this is a nice example, running a full web application using the low-level APIs will require you to write a lot of supplementary code to support common requirements. Fortunately, a company called Sencha has already created this scaffolding code for you in the form of a Node module called Connect.
Connect is a module built to support interception of requests in a more modular approach. In the first web server example, you learned how to build a simple web server using the http
module. If you wish to extend this example, you'd have to write code that manages the different HTTP requests sent to your server, handles them properly, and responds to each request with the correct response.
Connect creates an API exactly for that purpose. It uses a modular component called middleware, which allows you to simply register your application logic to predefined HTTP request scenarios. Connect middleware are basically callback
functions, which get executed when an HTTP request occurs. The middleware can then perform some logic, return a response, or call the next registered middleware. While you will mostly write custom middleware to support your application needs, Connect also includes some common middleware to support logging, static file serving, and more.
The way a Connect application works is by using an object called dispatcher
. The dispatcher
object handles each HTTP request received by the server and then decides, in a cascading way, the order of middleware execution. To understand Connect better, take a look at the following diagram:
The preceding diagram illustrates two calls made to the Connect application: the first one should be handled by a custom middleware and the second is handled by the static files middleware. Connect's dispatcher initiates the process, moving on to the next handler using the next()
method, until it gets to middleware responding with the res.end()
method, which will end the request handling.
In the next Lesson, you'll create your first Express application, but Express is based on Connect's approach, so in order to understand how Express works, we'll begin with creating a Connect application.
In your working folder, create a file named server.js
that contains the following code snippet:
var connect = require('connect'); var app = connect(); app.listen(3000); console.log('Server running at http://localhost:3000/');
As you can see, your application file is using the connect
module to create a new web server. However, Connect isn't a core module, so you'll have to install it using NPM. As you already know, there are several ways of installing third-party modules. The easiest one is to install it directly using the npm install
command. To do so, use your command-line tool, and navigate to your working folder. Then execute the following command:
$ npm install connect
NPM will install the connect
module inside a node_modules
folder, which will enable you to require it in your application file. To run your Connect web server, just use Node's CLI and execute the following command:
$ node server
Node will run your application, reporting the server status using the console.log()
method. You can try reaching your application in the browser by visiting http://localhost:3000
. However, you should get a response similar to what is shown in the following screenshot:
What this response means is that there isn't any middleware registered to handle the GET HTTP
request. This means two things:
Connect middleware is just JavaScript function with a unique signature. Each middleware function is defined with the following three arguments:
When you have a middleware defined, you'll just have to register it with the Connect application using the app.use()
method. Let's revise the previous example to include your first middleware. Change your server.js
file to look like the following code snippet:
var connect = require('connect'); var app = connect(); var helloWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Hello World'); }; app.use(helloWorld); app.listen(3000); console.log('Server running at http://localhost:3000/');
Then, start your connect server again by issuing the following command in your command-line tool:
$ node server
Try visiting http://localhost:3000
again. You will now get a response similar to that in the following screenshot:
Congratulations, you've just created your first Connect middleware!
Let's recap. First, you added a middleware function named helloWorld()
, which has three arguments: req
, res
, and next
. In your middleware, you used the res.setHeader()
method to set the response Content-Type
header and the res.end()
method to set the response text. Finally, you used the app.use()
method to register your middleware with the Connect application.
One of Connect's greatest features is the ability to register as many middleware functions as you want. Using the app.use()
method, you'll be able to set a series of middleware functions that will be executed in a row to achieve maximum flexibility when writing your application. Connect will then pass the next middleware function to the currently executing middleware function using the next
argument. In each middleware function, you can decide whether to call the next middleware function or stop at the current one. Notice that each Connect middleware function will be executed in first-in-first-out (FIFO) order using the next
arguments until there are no more middleware functions to execute or the next middleware function is not called.
To understand this better, we will go back to the previous example and add a logger
function that will log all the requests made to the server in the command line. To do so, go back to the server.js
file and update it to look like the following code snippet:
var connect = require('connect'); var app = connect(); var logger = function(req, res, next) { console.log(req.method, req.url); next(); }; var helloWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Hello World'); }; app.use(logger); app.use(helloWorld); app.listen(3000); console.log('Server running at http://localhost:3000/');
In the preceding example, you added another middleware called logger()
. The logger()
middleware uses the console.log()
method to simply log the request information to the console. Notice how the logger()
middleware is registered before the helloWorld()
middleware. This is important as it determines the order in which each middleware is executed. Another thing to notice is the next()
call in the logger()
middleware, which is responsible for calling the helloWorld()
middleware. Removing the next()
call would stop the execution of middleware function at the logger()
middleware, which means that the request would hang forever as the response is never ended by calling the res.end()
method.
To test your changes, start your connect server again by issuing the following command in your command-line tool:
$ node server
Then, visit http://localhost:3000
in your browser and notice the console output in your command-line tool.
As you may have noticed, the middleware you registered responds to any request regardless of the request path. This does not comply with modern web application development because responding to different paths is an integral part of all web applications. Fortunately, Connect middleware supports a feature called mounting, which enables you to determine which request path is required for the middleware function to get executed. Mounting is done by adding the path argument to the app.use()
method. To understand this better, let's revisit our previous example. Modify your server.js
file to look like the following code snippet:
var connect = require('connect'); var app = connect(); var logger = function(req, res, next) { console.log(req.method, req.url); next(); }; var helloWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Hello World'); }; var goodbyeWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Goodbye World'); }; app.use(logger); app.use('/hello', helloWorld); app.use('/goodbye', goodbyeWorld); app.listen(3000); console.log('Server running at http://localhost:3000/');
A few things have been changed in the previous example. First, you mounted the helloWorld()
middleware to respond only to requests made to the /hello
path. Then, you added another (a bit morbid) middleware called goodbyeWorld()
that will respond to requests made to the /goodbye
path. Notice how, as a logger should do, we left the logger()
middleware to respond to all the requests made to the server. Another thing you should be aware of is that any requests made to the base path will not be responded by any middleware because we mounted the helloWorld()
middleware to a specific path.
Connect is a great module that supports various features of common web applications. Connect middleware is super simple as it is built with a JavaScript style in mind. It allows the endless extension of your application logic without breaking the nimble philosophy of the Node platform. While Connect is a great improvement over writing your web application infrastructure, it deliberately lacks some basic features you're used to having in other web frameworks. The reason lies in one of the basic principles of the Node community: create your modules lean and let other developers build their modules on top of the module you created. The community is supposed to extend Connect with its own modules and create its own web infrastructures. In fact, one very energetic developer named TJ Holowaychuk, did it better than most when he released a Connect-based web framework known as Express.
3.238.82.77