4
Using Events, Listeners, Timers, and Callbacks in Node.js
Node.js provides scalability and performance through its powerful event-driven model. This chapter focuses on understanding the model and how it differs from traditional threading models used by most webservers. Understanding the event model is critical because it may force you to change the design thinking for your applications. However, the changes will be well worth the improvement in speed that you get using Node.js.
This chapter also covers the different methods you use to add work to the Node.js event queue. You can add work using event listeners or timers, or you can schedule work directly. You also learn how to implement events in your own custom modules and objects.
Understanding the Node.js Event Model
Node.js applications are run in a single-threaded event-driven model. Although Node.js implements a thread pool in the background to do work, the application itself doesn’t have any concept of multiple threads. “Wait, what about performance and scale?” you might ask. At first it may seem counterintuitive, but once you understand the logic behind the Node.js event model it all makes perfect sense.
Comparing Event Callbacks and Threaded Models
In the traditional threaded web model, a request comes in to the webserver and is assigned to an available thread. Then the handling of work for that request continues on that thread until the request is complete and a response is sent.
Figure 4.1 illustrates the threaded model processing two requests, GetFile and GetData. The GetFile request first opens the file, reads the contents, and then sends the data back in a response. All this occurs in order on the same thread. The GetData request connects to the DB, queries the necessary data, and then sends the data in the response.
Figure 4.1 Processing two requests on individual threads using the threaded model
The Node.js event model does things differently. Instead of executing all the work for each request on individual threads, work is added to an event queue and then picked up by a single thread running an event loop. The event loop grabs the top item in the event queue, executes it, and then grabs the next item. When executing code that is no longer live or has blocking I/O, instead of calling the function directly, the function is added to the event queue along with a callback that is executed after the function completes. When all events on the Node.js event queue have been executed, the Node application terminates.
Figure 4.2 illustrates the way Node.js handles the GetFile and GetData requests. The GetFile and GetData requests are added to the event queue. Node.js first picks up the GetFile request, executes it, and then completes by adding the Open() callback function to the event queue. Next, it picks up the GetData request, executes it, and completes by adding the Connect() callback function to the event queue. This continues until there are no callback functions to be executed. Notice in Figure 4.2 that the events for each thread do not necessarily follow a direct interleaved order. For example, the Connect request takes longer to complete than the Read request, so Send(file) is called before Query(db).
Figure 4.2 Processing two requests on a single event-driven thread using the Node.js event model
Blocking I/O in Node.js
The Node.js event model of using the event callbacks is great until you run into the problem of functions that block waiting for I/O. Blocking I/O stops the execution of the current thread and waits for a response before continuing. Some examples of blocking I/O are
Reading a file
Querying a database
Socket request
Accessing a remote service
The reason Node.js uses event callbacks is not to have to wait for blocking I/O. Therefore, any requests that perform blocking I/O are performed on a different thread in the background. Node.js implements a thread pool in the background. When an event that requires blocking I/O is retrieved from the event queue, Node.js retrieves a thread from the thread pool and executes the function there instead of on the main event loop thread. This prevents the blocking I/O from holding up the rest of the events in the event queue.
The function executed on the blocking thread can still add events back to the event queue to be processed. For example, a database query call is typically passed a callback function that parses the results and may schedule additional work on the event queue before sending a response.
Figure 4.3 illustrates the full Node.js event model including the event queue, event loop, and the thread pool. Notice that the event loop either executes the function on the event loop thread itself or, for blocking I/O, it executes the function on a separate thread.
The Conversation Example
To help you understand how events work in Node.js versus traditional threaded webservers, consider the example of having different conversations with a large group of people at a party. You are acting the part of the webserver, and the conversations represent the work necessary to process different types of web requests. Your conversations are broken up into several segments with different individuals. You end up talking to one person and then another. Then you go back to the first person and then to a third person, back to the second, and so on.
This example has many similarities to webserver processing. Some conversations end quickly, like a simple request for a piece of data in memory. Others are broken up into several segments as you go back and forth between individuals, similar to a more complex server-side conversation. Still others have long breaks when you are waiting for the other person to respond, similar to blocking I/O requests to the file system, database, or remote service.
Using the traditional webserver threading model in the conversation example sounds great at first because each thread acts like you. The threads/clones can talk back and forth with each person, and it almost seems as though you can have multiple conversations simultaneously. There are two problems with this model.
Figure 4.3 In the Node.js event model, work is added as a function with callback to the event queue, and then picked up on the event loop thread. The function is then executed on the event loop thread in the case of non-blocking, or on a separate thread in the case of blocking
First, you are limited by the number of clones. What if you only have five clones? To talk with a sixth person, one clone must completely finish its conversation. The second problem is the limited number of CPUs (or “brains”) that the threads (“clones”) must share. This means that clones sharing the same brain have to stop talking/listening while other clones are using the brain. You can see that there really isn’t a benefit to having clones when they freeze while the other clones are using the brain.
The Node.js event model acts more like real life when compared to the conversation example. First, Node.js applications run on a single thread, which means there is only one of you, no clones. Each time a person asks you a question, you respond as soon as you can. Your interactions are completely event driven, and you move naturally from one person to the next. Therefore, you can have as many conversations going on at the same time as you want by bouncing between individuals. Second, your brain is always focused on the person you are talking to since you aren’t sharing it with clones.
So how does Node.js handle blocking I/O requests? That is where the background thread pool comes into play. Node.js hands blocking requests over to a thread in the thread pool so that it has minimal impact on the application processing events. Think about when someone asks you a question that you have to think about. You can still interact with others at the party while trying to process that question in the back of your mind. That processing may impact how fast you interact with others, but you are still able to communicate with several people while processing the longer-lived thought.
Adding Work to the Event Queue
As you create your Node.js applications, keep in mind the event model described in the previous section and apply it to the way you design your code. To leverage the scalability and performance of the event model, make sure that you break work up into chunks that can be performed as a series of callbacks.
Once you have designed your code correctly, you can then use the event model to schedule work on the event queue. In Node.js applications, work is scheduled on the event queue by passing a callback function using one of these methods:
Make a call to one of the blocking I/O library calls such as writing to a file or connecting to a database.
Add a built-in event listener to a built-in event such as an http.request or server.connection.
Create your own event emitters and add custom listeners to them.
Use the process.nextTick option to schedule work to be picked up on the next cycle of the event loop.
Use timers to schedule work to be done after a particular amount of time or at periodic intervals.
The following sections discuss implementing timers, nextTick, and custom events. They give you an idea of how the event mechanism works. The blocking I/O calls and built-in events are covered in subsequent chapters.
Implementing Timers
A useful feature of Node.js and JavaScript is the ability to delay execution of code for a period of time. This can be useful for cleanup or refresh work that you do not want to always be running. There are three types of timers you can implement in Node.js: timeout, interval, and immediate. The following sections describe each of these timers and how to implement them in your code.
Delaying Work with Timeouts
Timeout timers are used to delay work for a specific amount of time. When that time expires, the callback function is executed and the timer goes away. Use timeouts for work that only needs to be performed once.
Timeout timers are created using the setTimeout(callback, delayMilliSeconds, [args]) method built into Node.js. When you call setTimeout(), the callback function is executed after delayMilliSeconds expires. For example, the following executes myFunc() after 1 second:
setTimeout(myFunc, 1000);
The setTimeout() function returns a timer object ID. You can pass this ID to clearTimeout(timeoutId) at any time before the delayMilliSeconds expires to cancel the timeout function. For example:
myTimeout = setTimeout(myFunc, 100000); … clearTimeout(myTimeout);
Listing 4.1 implements a series of simple timeouts that call the simpleTimeout() function, which outputs the number of milliseconds since the timeout was scheduled. Notice that it doesn’t matter which order setTimeout() is called; the results, shown in Listing 4.1 Output, are in the order that the delay expires.
Listing 4.1 simple_timer.js: Implementing a series of timeouts at various intervals
01 function simpleTimeout(consoleTimer){
02 console.timeEnd(consoleTimer);
03 }
04 console.time("twoSecond");
05 setTimeout(simpleTimeout, 2000, "twoSecond");
06 console.time("oneSecond");
07 setTimeout(simpleTimeout, 1000, "oneSecond");
08 console.time("fiveSecond");
09 setTimeout(simpleTimeout, 5000, "fiveSecond");
10 console.time("50MilliSecond");
11 setTimeout(simpleTimeout, 50, "50MilliSecond");<Listing First>
Listing 4.1 Output simple_timer.js: Timeout functions executed at different delay amounts
C:ooks odech04> node simple_timer.js 50MilliSecond: 50.489ms oneSecond: 1000.688ms twoSecond: 2000.665ms fiveSecond: 5000.186ms
Performing Periodic Work with Intervals
Interval timers are used to perform work on a regular delayed interval. When the delay time expires, the callback function is executed and is then rescheduled for the delay interval again. Use intervals for work that needs to be performed on a regular basis.
Interval timers are created using the setInterval(callback, delayMilliSeconds, [args]) method built into Node.js. When you call setInterval(), the callback function is executed every interval after delayMilliSeconds has expired. For example, the following executes myFunc() every second:
setInterval(myFunc, 1000);
The setInterval() function returns a timer object ID. You can pass this ID to clearInterval(intervalId) at any time before the delayMilliSeconds expires to cancel the timeout function. For example:
myInterval = setInterval(myFunc, 100000); … clearInterval(myInterval);
Listing 4.2 implements a series of simple interval callbacks that update the values of the variables x, y, and z at different intervals. Notice that the values of x, y, and z are changed differently because the interval amounts are different, with x incrementing twice as fast as y, which increments twice as fast as z, as shown in Listing 4.2 Output.
Listing 4.2 simple_interval.js: Implementing a series of update callbacks at various intervals
01 var x=0, y=0, z=0;
02 function displayValues(){
03 console.log("X=%d; Y=%d; Z=%d", x, y, z);
04 }
05 function updateX(){
06 x += 1;
07 }
08 function updateY(){
09 y += 1;
10 }
11 function updateZ(){
12 z += 1;
13 displayValues();
14 }
15 setInterval(updateX, 500);
16 setInterval(updateY, 1000);
17 setInterval(updateZ, 2000);
Listing 4.2 Output simple_interval.js: Interval functions executed at different delay amounts
C:ooks odech04> node simple_interval.js x=3; y=1; z=1 x=7; y=3; z=2 x=11; y=5; z=3 x=15; y=7; z=4 x=19; y=9; z=5 x=23; y=11; z=6
Performing Immediate Work with an Immediate Timer
Immediate timers are used to perform work on a function as soon as the I/O event callbacks are executed, but before any timeout or interval events are executed. This allows you to schedule work to be done after the current events in the event queue are completed. Use immediate timers to yield long-running execution segments to other callbacks to prevent starving the I/O events.
Immediate timers are created using the setImmediate(callback,[args]) method built into Node.js. When you call setImmediate(), the callback function is placed on the event queue and popped off once for each iteration through the event queue loop after I/O events have a chance to be called. For example, the following schedules myFunc() to execute on the next cycle through the event queue:
setImmediate(myFunc(), 1000);
The setImmediate() function returns a timer object ID. You can pass this ID to clearImmediate(immediateId) at any time before it is picked up off the event queue. For example:
myImmediate =setImmediate(myFunc); …clearImmediate(myImmediate);
Dereferencing Timers from the Event Loop
Often you do not want timer event callbacks to continue to be scheduled when they are the only events left in the event queue. Node.js provides a useful utility to handle this case. The unref() function available in the object returned by setInterval and setTimeout allows you to notify the event loop to not continue when these are the only events on the queue.
For example, the following dereferences the myInterval interval timer:
myInterval = setInterval(myFunc); myInterval.unref();
If for some reason you later do not want the program to terminate if the interval function is the only event left on the queue, you can use the ref() function to re-reference it:
myInterval.ref();
Warning
When using unref() with setTimout timers, a separate timer is used to wake up the event loop. Creating a lot of these can cause an adverse performance impact on your code, so use them sparingly.
Using nextTick to Schedule Work
A useful method of scheduling work on the event queue is the process.nextTick(callback) function. This function schedules work to be run on the next cycle of the event loop. Unlike the setImmediate() method, nextTick() executes before the I/O events are fired. This can result in starvation of the I/O events, so Node.js limits the number of nextTick() events that can be executed each cycle through the event queue by the value of process.maxTickDepth, which defaults to 1000.
Listing 4.3 illustrates the order of events when using a blocking I/O call, timers, and nextTick(). Notice that the blocking call fs.stat() is executed first, then two setImmediate() calls, and then two nextTick() calls. Listing 4.3 Output shows that both nextTick() calls are executed before any of the others. Then the first setImmediate() call is executed followed by the fs.stat(), and then on the next iteration through the loop, the second setImmediate() call is executed.
Listing 4.3 nexttick.js: Implementing a series of blocking fs calls, immediate timers, and nextTick() calls to show the order in which they get executed
01 var fs = require("fs");
02 fs.stat("nexttick.js", function(){
03 console.log("nexttick.js Exists");
04 });
05 setImmediate(function(){
06 console.log("Immediate Timer 1 Executed");
07 });
08 setImmediate(function(){
09 console.log("Immediate Timer 2 Executed");
10 });
11 process.nextTick(function(){
12 console.log("Next Tick 1 Executed");
13 });
14 process.nextTick(function(){
15 console.log("Next Tick 2 Executed");
16 });
Listing 4.3 Output nexttick.js: Executing the nextTick() calls first
c:ooks odech04>node nexttick.js Next Tick 1 Executed Next Tick 2 Executed Immediate Timer 1 Executed Immediate Timer 2 Executed nexttick.js Exists
Implementing Event Emitters and Listeners
In the following chapters you get a chance to implement many of the events built in to the various Node.js modules. This section focuses on creating your own custom events as well as implementing listener callbacks that get implemented when an event is emitted.
Adding Custom Events to Your JavaScript Objects
Events are emitted using an EventEmitter object. This object is included in the events module. The emit(eventName, [args]) function triggers the eventName event and includes any arguments provided. The following code snippet shows how to implement a simple event emitter:
var events = require('events');
var emitter = new events.EventEmitter();
emitter.emit("simpleEvent");
Occasionally you want to add events directly to your JavaScript objects. To do that you need to inherit the EventEmitter functionality in your object by calling events.EventEmitter.call(this) in your object instantiation as well as adding the events.EventEmitter. prototype to your object’s prototyping. For example:
Function MyObj(){
Events.EventEmitter.call(this);
}
MyObj.prototype.__proto__ = events.EventEmitter.prototype;
You then can emit events directly from instances of your object. For example:
var myObj = new MyObj();
myObj.emit("someEvent");
Adding Event Listeners to Objects
Once you have an instance of an object that can emit events, you can add listeners for the events that you care about. Listeners are added to an EventEmitter object using one of the following functions:
.addListener(eventName, callback): Attaches the callback function to the object’s listeners. Every time the eventName event is triggered, the callback function is placed in the event queue to be executed.
.on(eventName, callback): Same as .addListener().
.once(eventName, callback): Only the first time the eventName event is triggered, the callback function is placed in the event queue to be executed.
For example, to add a listener to an instance of the MyObject EventEmitter class defined in the previous section you would use the following:
function myCallback(){
…
}
var myObject = new MyObj();
myObject.on("someEvent", myCallback);
Removing Listeners from Objects
Listeners are useful and vital parts of Node.js programming. However, they do cause overhead, and you should use them only when necessary. Node.js provides server helper functions on the EventEmitter object that allow you to manage the listeners that are included. These include
.listeners(eventName): Returns an array of listener functions attached to the eventName event.
.setMaxListeners(n): Triggers a warning if more than n listeners are added to an EventEmitter object. The default is 10.
.removeListener(eventName, callback): Removes the callback function from the eventName event of the EventEmitter object.
Implementing Event Listeners and Event Emitters
Listing 4.4 demonstrates the process of implementing listeners and custom event emitters in Node.js. The Account object is extended to inherit from the EventEmitter class and provides two methods to deposit and withdraw that both emit the balanceChanged event. Then in lines 15–31, three callback functions are implemented that are attached to the Account object instance balanceChanged event and display various forms of data.
Notice that the checkGoal(acc, goal) callback is implemented a bit differently than the others. This was done to illustrate how to pass variables into an event listener function when the event is triggered. The results of executing the code are shown in Listing 4.4 Output.
Listing 4.4 emitter_listener.js: Creating a custom EventEmitter object and implementing three listeners that are triggered when the balancedChanged event is triggered
01 var events = require('events');
02 function Account() {
03 this.balance = 0;
04 events.EventEmitter.call(this);
05 this.deposit = function(amount){
06 this.balance += amount;
07 this.emit('balanceChanged');
08 };
09 this.withdraw = function(amount){
10 this.balance -= amount;
11 this.emit('balanceChanged');
12 };
13 }
14 Account.prototype.__proto__ = events.EventEmitter.prototype;
15 function displayBalance(){
16 console.log("Account balance: $%d", this.balance);
17 }
18 function checkOverdraw(){
19 if (this.balance < 0){
20 console.log("Account overdrawn!!!");
21 }
22 }
23 function checkGoal(acc, goal){
24 if (acc.balance > goal){
25 console.log("Goal Achieved!!!");
26 }
27 }
28 var account = new Account();
29 account.on("balanceChanged", displayBalance);
30 account.on("balanceChanged", checkOverdraw);
31 account.on("balanceChanged", function(){
32 checkGoal(this, 1000);
33 });
34 account.deposit(220);
35 account.deposit(320);
36 account.deposit(600);
37 account.withdraw(1200);
Listing 4.4 Output emitter_listener.js: The account statements output by the listener callback functions
C:ooks odech04>node emmiter_listener.js Account balance: $220 Account balance: $540 Account balance: $1140 Goal Achieved!!! Account balance: $-60 Account overdrawn!!!
Implementing Callbacks
As you have seen in previous sections, the Node.js event-driven model relies heavily on callback functions. Callback functions can be a bit difficult to understand at first, especially if you want to depart from implementing a basic anonymous function. This section deals with three specific implementations of callbacks: passing parameters to a callback function, handling callback function parameters inside a loop, and nesting callbacks.
Passing Additional Parameters to Callbacks
Most callbacks have automatic parameters passed to them, such as an error or result buffer. A common question when working with callbacks is how to pass additional parameters to them from the calling function. You do this by implementing the parameter in an anonymous function and then call the actual callback with parameters from the anonymous function.
Listing 4.5 illustrates implementing callback parameters. There are two sawCar event handlers. Note that the sawCar event only emits the make parameter. Notice that the emitter.emit() function also can accept additional parameters; in this case, make is added as shown in line 5. The first event handler on line 16 implements the logCar(make) callback handler. To add a color for logColorCar(), an anonymous function is used in the event handler defined in lines 17–21. A randomly selected color is passed to the call logColorCar(make, color). You can see the output in Listing 4.5 Output.
Listing 4.5 callback_parameter.js: Creating an anonymous function to add additional parameters not emitted by the event
01 var events = require('events');
02 function CarShow() {
03 events.EventEmitter.call(this);
04 this.seeCar = function(make){
05 this.emit('sawCar', make);
06 };
07 }
08 CarShow.prototype.__proto__ = events.EventEmitter.prototype;
09 var show = new CarShow();
10 function logCar(make){
11 console.log("Saw a " + make);
12 }
13 function logColorCar(make, color){
14 console.log("Saw a %s %s", color, make);
15 }
16 show.on("sawCar", logCar);
17 show.on("sawCar", function(make){
18 var colors = ['red', 'blue', 'black'];
19 var color = colors[Math.floor(Math.random()*3)];
20 logColorCar(make, color);
21 });
22 show.seeCar("Ferrari");
23 show.seeCar("Porsche");
24 show.seeCar("Bugatti");
25 show.seeCar("Lamborghini");
26 show.seeCar("Aston Martin");
Listing 4.5 Output callback_parameter.js: The results of adding a color parameter to the callback
C:ooks odech04>node callback_parameter.js Saw a Ferrari Saw a blue Ferrari Saw a Porsche Saw a black Porsche Saw a Bugatti Saw a red Bugatti Saw a Lamborghini Saw a black Lamborghini Saw a Aston Martin Saw a black Aston Martin
Implementing Closure in Callbacks
An interesting problem that asynchronous callbacks have is that of closure. Closure is a JavaScript term that indicates that variables are bound to a function’s scope and not the parent function’s scope. When you execute an asynchronous callback, the parent function’s scope may have changed; for example, when iterating through a list and altering values in each iteration.
If your callback needs access to variables in the parent function’s scope, then you need to provide closure so that those values are available when the callback is pulled off the event queue. A basic way of doing that is by encapsulating the asynchronous call inside a function block and passing in the variables that are needed.
Listing 4.6 implements a wrapper function that provides closure to the logCar() asynchronous function. Notice that the loop in lines 7–12 implements a basic callback. However, Listing 4.6 Output shows that the car name is always the last item read because the value of message changes each time through the loop.
The loop in lines 13–20 implements a wrapper function that is passed message as the msg parameter and that msg value sticks with the callback. Thus the closure shown in Output 4.6 displays the correct message. To make the callback truly asynchronous, the process.nextTick() method is used to schedule the callback.
Listing 4.6 callback_closure.js: Creating a wrapper function to provide closure for variables needed in the asynchronous callback
01 function logCar(logMsg, callback){
02 process.nextTick(function() {
03 callback(logMsg);
04 });
05 }
06 var cars = ["Ferrari", "Porsche", "Bugatti"];
07 for (var idx in cars){
08 var message = "Saw a " + cars[idx];
09 logCar(message, function(){
10 console.log("Normal Callback: " + message);
11 });
12 }
13 for (var idx in cars){
14 var message = "Saw a " + cars[idx];
15 (function(msg){
16 logCar(msg, function(){
17 console.log("Closure Callback: " + msg);
18 });
19 })(message);
20 }
Listing 4.6 Output callback_closure.js: Adding a closure wrapper function allows the asynchronous callback to access necessary variables
C:ooks odech04>node callback_closure.js Normal Callback: Saw a Bugatti Normal Callback: Saw a Bugatti Normal Callback: Saw a Bugatti Closure Callback: Saw a Ferrari Closure Callback: Saw a Porsche Closure Callback: Saw a Bugatti
Chaining Callbacks
With asynchronous functions you are not guaranteed the order that they will run if two are placed on the event queue. The best way to resolve that is to implement callback chaining by having the callback from the asynchronous function call the function again until there is no more work to do. That way the asynchronous function is never on the event queue more than once.
Listing 4.7 implements a basic example of callback chaining. A list of items is passed into the function logCars(), the asynchronous function logCar() is called, and then the logCars() function is used as the callback when logCar() completes. Thus only one version of logCar() is on the event queue at the same time. The output of iterating through the list is shown in Listing 4.7 Output.
Listing 4.7 callback_chain.js: Implementing a callback chain where the callback from an anonymous function calls back into the initial function to iterate through a list
01 function logCar(car, callback){
02 console.log("Saw a %s", car);
03 if(cars.length){
04 process.nextTick(function(){
05 callback();
06 });
07 }
08 }
09 function logCars(cars){
10 var car = cars.pop();
11 logCar(car, function(){
12 logCars(cars);
13 });
14 }
15 var cars = ["Ferrari", "Porsche", "Bugatti",
16 "Lamborghini", "Aston Martin"];
17 logCars(cars);
Listing 4.7 Output callback_chain.js: Using an asynchronous callback chain to iterate through a list
C:ooks odech04>node callback_chain.js Saw a Aston Martin Saw a Lamborghini Saw a Bugatti Saw a Porsche Saw a Ferrari
Summary
The event-driven model that Node.js uses provides scalability and performance. You learned the difference between the event-driven model and the traditional threaded model for webservers. You learned that events can be added to the event queue when blocking I/O is called. And you learned that listeners can be triggered by events or timers or called directly using the nextTick() method.
This chapter discussed the three types of timer events: timeout, interval, and immediate. Each of these can be used to delay the execution of work for a period of time. You also saw how to implement your own custom event emitters and add listener functions to them.