Node.js is a relatively new server platform that is built on JavaScript. One of the main features of Node is that it is a non-blocking server. This means that resource-intensive tasks will not tie up the server. Node.js can then handle many concurrent connections. It can also handle real-time communications more easily than a blocking server.
One of the main uses of Node.js is as a web server. This is a perfect task as serving web pages usually involves reading files and connecting to a database. It is able to serve more clients while either of these two actions are executing. This is very different compared to a blocking server. A blocking server would have to wait for all the resources to return before it could respond to more requests.
The most popular frameworks are useful and should be utilized whenever possible, but our focus will only be on functions and modules that are included with Node.js. There will be no coverage of Express (arguably, the most used Node.js framework). Most importantly, we will cover the building blocks of Node.js that Express and its dependencies are built on.
A great book that covers frameworks and more is Building Scalable Apps with Redis and Node.js by Packt Publishing.
In this chapter, the following groups of references will be described, accompanied with examples:
We will start our overview of Node.js with the basics. This will include loading modules, managing processes, handling files and paths, and REPL (Read Eval Print Loop). These are things that virtually any Node.js project will need.
Node.js is a modular system. Much of the functionality of Node.js is contained in modules that must be loaded at runtime. This makes the knowledge of loading and building modules a core requirement to using Node. Here are the references you can use for your Node modules:
require()
modules.export
This loads the module at the given path:
require(path)
The return value will be an object. The properties of this object will vary, depending on what is loaded. We will cover module.exports
next, which is what module designers and even you can use to set the value of this return value.
The require()
function is used to load the module at a path, where the path can be a core module (a module packaged with Node.js), directory, file, or module (a project that you or someone else has built). As a result, it is very versatile. The require()
function will try to resolve the passed-in path in this order: the core module, the directory or file if the path begins with "./", "/", "../", and then the module, take a look at the following description for more information. You then need to check the require function in different locations for all of this to work:
.js
, .json
, or .node
. This means that the require (./data/models)
function and require (./data/models,js)
will load the same file. If require does not find a file, then it will try to load the path as a directory. This means it will look for either an index.js
file or the main property inside package.json
. Require will then either use index.js
or package.json
to load as a file.node_modules
folder. The first directory will be the current
directory. After that, each directory will be a parent directory, all the way up to the root
directory. In addition to this, require will also search the paths in the NODE_PATH
environment variable.As a quick summary, here is the order that require will use:
node_modules
module (no relative path)Here are some examples of how the require function is used to load core modules, files, or modules:
var http = require('http'); //loads a core module
As you can see from the comment, this example will load the HTTP core module:
var data = require('./data'); //loads a directory
This example shows you how to load a directory:
var dataModels = require('./data/models'); //loads the file models.js
This one will load a file:
var redis = require('redis'); //loads a module
Finally, this one will load a module.
Every Node.js project will use require many times, so knowing how and where it will load is important.
Here is a simple example that demonstrates require and how it works with module.exports
.
Here is the file that will be loaded with require, requireTest.js
, and it will be loaded from a relative path:
module.exports = 'This value is from requireTest';
Here is how to load this value in a Node.js script:
var requireTest = require('./requireTest'); console.log(requireTest);
The
module.exports
property is a convenience property used in a module to expose the functionality of the module outside the scope of the current file.
The module.exports
property is the return value from the module. This can seem misleading though. We will not return module.exports
. It is the object or properties set at module.exports
that are available from this module.
The module.exports
property is the link from the current file into the scope of the calling file. Any variables defined in the current file will not be in the scope of file that calls it. To make the variables available, you must either set them as the value of module.exports
or as properties of module.exports
. This functionality is the mechanism that Node.js uses with require. Exports can be set to any type: string, integer, function, object, and many others. Functions allow us to either pass in values or create a constructor()
function. Objects can be created and then used to overwrite the exports object or can be added to the exports object as properties. We have all of JavaScript's function- and object-creation tricks at our disposal.
Examples of the constructor()
function being used are shown here:
module.exports = function Foo(bar) {do something with bar};
module.exports = function FooConstructor() {};
foo
property will be used in the following two examples:module.exports = {foo: foo()}; module.exports.foo = foo();
Here are the most important functions and properties of the
os
module. The os
module allows you to get information from the operating system. This is important as we may need to know the hostname or how many CPUs are currently available.
All the functions that we will look at now assume that var os = require('os')
is already in the file.
This will return the number of CPUs of the current device:
.cpus()
We will get an array of objects that map to each CPU. The objects will have the model, speed, and times:
user
, nice
, sys
, idle
, and irq
.This information can be used, most of the time, to find out how many CPUs the machine has. It is good to know this information, as Node.js is single threaded and we can launch multiple processes in a cluster for each CPU.
Here is an example of getting the number of CPUs on a computer:
var cpus = os.cpus().length;
This gets a list of network interfaces:
.networkInterfaces()
Process is an object that allows us to get access to events on the process. We can also get access to stdin
, stdout
, and stderr
. These are global objects that will be available in any file or context.
The stdout
object is a writable stream.
stdout
This is the stream that will connect to stdout
. Note that streams will be covered later in this chapter under Utilites. If we are running Node.js as a console application, this is where we could write to stdout
if needed. Most streams in Node are non-blocking, but writes to stdout and stderr are blocking.
We also can use stdout
to find out whether Node.js is running in TTY. We can access process.stdout.isTTY
. This is a Boolean value.
The example here will show us how to write a message to the stdout
stream. By default, this will be sent to the process.stdout.write
console. This will write to stdout
.
The stderr
object is a writable stream.
stderr
This is similar to
stdout
, with the key difference being it writes to stderr
. This is very useful for console applications as we can write to the console what is happening.
As an example, this writes to the stderr
stream. By default this will write to the console.
process.stderr.write("Something seems to have gone wrong.");
This is a readable stream that maps to stdin
.
stdin
This maps to the standard stream, stdin
, in the same way stdout
and stderr
map to the stderr
streams. The difference is that the stdin
object is readable, while the others are writable. Anything that is piped into this process can be read out using process.stdin
. To read from a readable stream, we will have to listen for two possible events that let us know how we can retrieve data. The first is readable, and the other is data. We will cover this more in depth when we get to the stream section.
For instance, this example takes the data that is sent in through stdin
and sends it to stdout
using the readable event:
process.stdin.on('readable', function() { var data = process.stdin.read(); if(data !== null) process.stdout.write(data); });
This is all the command-line arguments passed in:
argv
The argv
commands will be an array of all the arguments passed into this script. Arguments are split by spaces. This means that debug=true
(which has no spaces) will be one argument, and debug = true
(a space between each word) will be three. This is an important distinction to keep in mind if you want to pass values in arguments. Arguments 0
and 1
will always be the node and the path and filename of the current script.
Here is an example:
process.argv.forEach(function(val){ if(val === 'debug'){ debug(); } });
Signal events allow you to listen for the standard POSIX signals:
process.on({signal, callback});
All of the POSIX
signals, except SIGKILL
and SIGSTOP
, can be listened for. In addition to these signals, you can also listen for some Windows signals. Here is the list of signals:
SIGUSR1
: This is a user-defined signal. In Node.js, it is usually used to start the debugger.SIGTERM
: This is the signal to terminate.SIGINT
: This is the signal to interrupt. It is usually sent by pressing Ctrl + C.SIGPIPE
: This lets a process know when it is trying to pipe to a nonexistent process.SIGHUP
: This is the signal that tells a process that the terminal it was running in has hung up.SIGBREAK
: This is non-POSIX and is used by Windows. It should be used in a manner similar to SIGINT
.SIGWINCH
: This is the signal to tell the process that the window has changed size.Here is an example of listening for SIGINT
, which recognizes that Ctrl + C was pressed:
process.on('SIGINT', function(){ //ctrl+c was pressed });
This object contains the environment variables:
process.env
The process.env
object is a simple JavaScript object that contains all the environment variables. Each property will be a string.
A great place to put configuration settings is in the environment when building an extensible application. The process.env
object can then be used to check whether the current environment is in production or even to just store the configuration settings.
Here is an example of checking for environment and setting a value from the environment:
if(process.env.NODE_ENV !== 'production') //do test stuff var redisHost = process.env.REDIS_HOST;
This will send a signal event to a process:
process.kill(pid, [signal])
This will send any of the signal events that are defined in the signal events section, which that we covered earlier. This is essentially running the POSIX
kill command.
We can use this kill command to send the current process a signal, using process.pid
, or to send another process that we have a reference to the kill signal.
Here is an example of killing the specific process ID 4634
:
process.kill(4634, 'SIGTERM');
process.pid
This section is not a true module, like the previous section on process. This section will focus on any of the functions or modules that allow us to read, write, or find files and directories.
This returns a string of the current filename:
__filename
This is a string of the current directory:
__dirname
We will now look at a list of key functions in the file module. All of these functions just run the underlying POSIX
commands.
Each of the commands we cover will have an asynchronous and synchronous version of each function. The synchronous function will be the same name, with Sync appended, for example, open
and openSync
. In a way, the asynchronous versions are more like Node.js, in that, they let the event loop process without holding up execution.
Each asynchronous function will need a callback function defined as the last parameter. This function will be in the form of function(err, result)
. We will cover this when we get to handling errors. The quick takeaway is that the first parameter, err
, will be null
if no error occurred, or it will have the error.
The synchronous functions will not process anything in the event loop until the function returns. While this seems against the Node.js non-blocking paradigm, it makes sense in certain cases, for example, when a specific file must be opened before anything else can process. This means that the classic try/catch
is needed to handle any exceptions or errors.
In almost every circumstance, the asynchronous version is the one that should be used.
Each example is under the assumption that the fs
variable already exists. Here is the code that sets the fs
variable to the fs
module:
var fs = require('fs');
This returns
fs.Stats
, which is an object with information about the queried file or directory:
fs.statstat(path, callback) fs.statSync(path)
The stat
variable gets the the data that would be returned after running stat()
on the operating system. The
stat
object is an instance of fs.Stats
. This object gives us useful functions such as isFile()
and isDirectory()
. In addition to this, fs.Stats
has many properties. We can see the modified time, access time, and file size.
There are more functions and properties, but these are the most likely used.
The fs.lstat()
and fs.fstat()
functions will both return an fs.Stats
object when pointed to a file. The difference is that lstat
will run against a link instead of following the link to the file or directory. The fstat
runs against a file descriptor instead of a path.
Here is an example that loads a file named test.txt
in the same directory as the code file and logs the fs.Stats
object:
fs.stat(__dirname + ' est.txt', function(err, stats){ console.log(stats); });
This returns a file descriptor of the path passed in:
fs.open(path, flags, [mode], callback) fs.openSync(path, flags, [mode])
This will open a file in the path passed in. There are many flags that can be passed. They are described here:
r
: Read only, errors when file does not existr+
: Read and write, errors when file does not existrs
: Read synchronous, here, "synchronous" only refers to the filesystem caching data and not synchronous such as openSync
rs+
: Read and write synchronousw
: Writewx
: Write, errors when file existsw+
: Read and write, file is created or truncateda
: Append, file is createdax
: Append, errors when file existsa+
: Read and append, file is createdax+
: Read and append, errors when file existsThe flags allow you to open, read, and write a file, whether it exists or not. This means that most of the time, a call to fs.exists
is not required as one of the flags will give you the answer you need.
The mode parameter is the permissions that are used if a file is created. It defaults to 0666
.
Finally, the callback returns a file descriptor. With this, data can be read or written using fs.read
or fs.write
.
This is an example of opening the file test.txt
:
fs.open(__dirname + 'file.txt', 'r', function(err, fd){ //fd is available to be read from });
This reads from a file descriptor:
fs.read(fd, buffer, offset, length, position, callback) fs.readSync(fd, buffer, offset, length, position)
The read()
function takes a lot of parameters to run. We need a file descriptor, which we get from open, a buffer, and the size of the file. In the example, we used fs.stat
to get the file size.
Here is a full example of opening a file and reading from it:
fs.stat(__dirname + '/test.txt', function(error, stats) {
fs.open(__dirname + '/test.txt', 'r', function(err, fd){
var buffer = new Buffer(stats.size);
fs.read(fd, buffer, 0, stats.size, null, function(err, bytesRead, buffer){
console.log(buffer.toString('utf8'));
});
});
});
This is a simplified version for reading a file:
fs.readFile(filename, [options], callback) fs.readFileSync(filename, [options])
The readFile()
function greatly simplifies the process of reading a file in Node.js. In the previous function, fs.read
, we needed to set everything up before we tried to read the file. The readFile()
function allows us to just pass a filename and get a buffer back of what is in the file.
The optional options
object takes a flag and an encoding property. The flag property is the same as the flag from fs.open
, so any of them can be used. The encoding is used for the buffer that is returned in the callback.
Here is an example that demonstrates how much easier it is to load a file with readFile
. The example is also using the optional options
parameter:
var filename = __dirname + '/test.txt'; fs.readFile(filename, {flag: 'r', encoding: 'utf8'}, function(err, data){ console.log(data.toString('utf8')); });
This writes to a file descriptor:
fs.write(fd, buffer, offset, length, position, callback) fs.writeSync(fd, buffer, offset, length, position)
The write()
function takes a file descriptor and buffer to write to a file. Much like read, we have to pass in quite a few parameters to make this work.
In the following example, we are reading a file into a buffer and then writing that buffer back out to another file. The offset and length are both integers that we need to pass in. The position can be an integer, but it can also be null
. Null
will start writing at the current position in the file.
Just like read, write can be complex. Here is a full example of reading from one file and writing to another:
var fs = require('fs'); var filename = __dirname + '/test.txt'; var writeFile = __dirname + '/test2.txt'; fs.stat(filename, function(error, stats) { fs.open(filename, 'r', function(err, fd) { var buffer = new Buffer(stats.size); fs.read(fd, buffer, 0, stats.size, null, function(err, bytesRead, buffer) { fs.open(writeFile, 'w', function(err, writeFD) { //will create a file named test2.txt with the contents from test.txt fs.write(writeFD, buffer, 0, buffer.length, null, function(err, bytesWritten, writeBuffer) { console.log(writeBuffer.toString('utf8')); }); }); }); }); });
This is a simplified function to write to a file:
fs.writeFile(filename, data, [options], callback) fs.writeFileSync(filename, data, [options])
Exactly like the difference between read and readFile
, writeFile
is a simplified version of write. We just pass in a filename and a string or buffer, and it will write the file.
The callback only returns an error when it occurs.
readFile
and writeFile
seem like the best choices in most cases, and in fact, this is most likely true. The main thing that you give up using these functions is control. You can only read an entire file or write an entire file.
Here is an example of writeFile
:
var filename = __dirname + '/write.txt'; var buffer = new Buffer('Write this to a file.', 'utf8'); fs.writeFile(filename, buffer, {encoding: 'utf8', flag: 'w'}, function(err){ if(null !== null){ //do something } });
This function allows us to append a file:
fs.appendFile(filename, data, [options], callback) fs.appendFileSync(filename, data, [options])
appendFile
exists because writeFile
will only write an entire file. So, we need another function to add to a file. This is where giving up some control for ease comes in. If we were using a file descriptor and write, then we could choose to start writing at the end of the file. This is effectively appending the file:
Here is an example of appendFile
:
var filename = __dirname + '/write.txt'; var buffer = new Buffer('Append this to a file.', 'utf8'); fs.appendFile(filename, buffer, {encoding: 'utf8', flag: 'w'}, function(err){ if(null !== null){ //do something } });
The path
module is separate from the file module. Path is concerned with fixing paths, whereas file is concerned with working with files and directories. Many times, these modules will be used together.
The functions covered are the most used and most useful. You will most likely need to locate paths relative to your current project, and this is what the path module is for:
Just like the other modules, the example assumes that the path module has been loaded:
var path = require('path');
This returns a string of the path with any oddities fixed:
path.normalize(pathString)
Normalize will fix many problems associated with reading paths. A great example of this is the difference between Windows and Unix paths. Unix uses forward slashes, and Windows uses backslashes. Normalize will return the correct slashes based on the system it is executed on.
In addition to this, normalize will also remove the current directory and parent directory shortcuts, (.
and ..
respectively). It will also remove double slashes from a directory. It normalizes any paths passed in.
Here is an example of using Unix slashes on a Windows system. This example will change all the slashes to backslashes:
var pathString = "/unix/style/path/separators"; console.log(path.normalize(pathString));
This returns a string of all the paths joined together:
path.join([pathString1],[…])
Join makes it easy to create a path from partial paths. This seems like something that can be done with concatenation, but it is easy to forget about path separators. Join will make sure that the path separators will all be in the correct spots.
Here is an example of
join
. The example will take all the parameters and join them together with the correct slash for your system:
console.log(path.join('path', 'separators', 'added'));
This returns the string of the path based on the parameters:
path.resolve([pathString], […])
This can be viewed as the cd command executed for each parameter. The paths passed in can be relative or full paths. A relative path will add to the returned path, and a full path will be used whole.
Here are two examples:
/home/josh/test
:console.log(path.resolve('/home/josh/node', '..', 'test'));
/home/brian/node
:console.log(path.resolve('/home/josh/node', '/home/brian/node'));
This returns the difference between the to
and from
paths:
path.relative(from, to)
The path returned can be used with cd
to change to the new directory.
Here is an example that will return ../../brian/node
, because you must go up two parent folders and over to brian/node
to get from one to the other:
var from = '/home/josh/node'; var to = '/home/brian/node'; console.log(path.relative(from, to));
This returns a string of the directory name:
path.dirname(pathString)
This example will return the directory that the current file is in. If the file was in the directory /home/users/jjohanan/node
, then the example will return /home/users/jjohanan/node
:
console.log(path.dirname(__filename));
This returns a string of the final part of a path:
path.basename(pathString, [ext])
This will either return the filename or directory depending on what path is passed in. The optional ext
parameter will remove that portion from the return value if it exists.
Here are two examples, one involving a file and the other a directory:
test
:console.log(path.basename('/home/josh/test.js', '.js'));
josh
:console.log(path.basename('/home/josh'));
REPL stands for Read Eval Print Loop. What this means is that it is an interactive console. We can enter in a command (Read). The command will be executed (Eval). The output of the command will will be printed to the console (Print). Finally, we can do this as many times as we want (Loop). REPL is great to run a few lines of code to see what they will do.
This starts Node.js in the REPL mode:
node
The main way REPL will be used is by calling it directly. This can be done by executing Node.js without a file to serve, by running just node
. Once node
has returned with a prompt, we can add commands and see what happens when we run them. This is perfect to test a few lines of code.
Here is a quick example of logging in to the console; first run node
to get a prompt and then run the command:
node //wait for > console.log('hey this REPL!');
Errors are a part of any development project. We must be able to handle errors in a graceful way. If we do not, then we are creating bad experiences for our users. Even worse, we could be opening up a vector for attack. A stack trace that gets sent to an end user can give out many details.
This will be a special section that deals more with design patterns than with actual code reference. Node.js is an asynchronous event-driven platform. For the major part, most people have not worked on a platform like this and can make mistakes when handling errors. We feel that handling errors is very important.
The core of this information comes from Joyent, one the major forces behind Node.js
today. You can find more information on Joyent at https://www.joyent.com/developers/node/design/errors.
Errors can be split into two types: operational and programmer errors. Operational errors are errors that occur during the operation of an application. A database server not being accessible is an example. These errors can be planned for and handled gracefully.
Next is programmer errors, which are errors in the literal sense. For example, a piece of code is malformed or an unexpected condition has come up. These are very hard to plan for (if we had planned for them, then they wouldn't be errors!). These will almost always break the server, so we can come back through the logs to find what went wrong.
Now that we know the two different types of errors, let's look at the three ways of alerting an application that has an error. The three ways are throwing the error, asynchronous callback, and emitting an error event:
try/catch
block. Here is an example with JSON.parse
, which runs synchronously and throws an error when a non-JSON string is passed to it:try{ JSON.parse(jsonObject); } catch (ex) { //do something with this error }
function(error, result)
. The first parameter will either have an error or be null or undefined. We can implement this ourselves whenever we write an asynchronous function. If there is an error, return it in the callback as the first parameter.A good example of this is asynchronous and synchronous filesystem
module calls. For example, read takes a callback, and readSync
should be wrapped in a try/catch
block.
Here is an example callback and check for error:
fs.read(path, function (err, data) { if(err !== null) //handle error })
Finally, we can emit an error event. This is used for asynchronous functions as well. Whether we implement a callback or event is a personal choice, but it should be clear which one is being used. It is also a best practice to just implement one. Many times, an event is used when there is a long running asynchronous process. Reading data from a network socket is an example. A socket does not always give the data in one simple pass, so events are set up. One of those events is an error event. To handle this, we just need to listen for that event. Here is an example of listening for the error event of a socket:
socket.on('error', function(error){ //handle error here })
3.16.79.65