Before we start diving into this new challenge, let's implement a small demo server that we will use as a reference to measure the impact of the various techniques we are going to implement.

Let's consider a web server that manages the sales of an e-commerce company, in particular, we want to query our server for the sum of all the transactions of a particular type of merchandise. For this purpose, we are going to use LevelUP again for its simplicity and flexibility. The data model that we are going to use is a simple list of transactions stored in the sales sublevel (a section of the database), which is organized in the following format:

transactionId {amount, item} 

The key is represented by transactionId and the value is a JSON object that contains the amount of the sale (amount) and the item type.

The data to process is really basic, so let's implement the API immediately in a file named totalSales.js, which will be as follows:

const level = require('level'); 
const sublevel = require('level-sublevel'); 
const db = sublevel(level('example-db', {valueEncoding: 'json'})); 
const salesDb = db.sublevel('sales'); 
 
module.exports = function totalSales(item, callback) { 
  console.log('totalSales() invoked'); 
  let sum = 0; 
  salesDb.createValueStream()            // [1] 
    .on('data', data => { 
      if(!item || data.item === item) {  // [2] 
        sum += data.amount; 
      } 
    }) 
    .on('end', () => { 
      callback(null, sum);               // [3] 
    }); 
}; 

The core of the module is the totalSales function, which is also the only exported API; this is how it works:

The simple query that we built is definitely not the best in terms of performance. Ideally, in a real-world application, we would have used an index to query the transactions by the item type, or even better, an incremental map/reduce to calculate the sum in real time; however, for our example, a slow query is actually better as it will highlight the advantages of the patterns we are going to analyze.

To finalize the total sales application, we only need to expose the totalSales API from an HTTP server; so, the next step is to build one (the app.js file):

const http = require('http'); 
const url = require('url'); 
const totalSales = require('./totalSales'); 
 
http.createServer((req, res) => { 
  const query = url.parse(req.url, true).query; 
  totalSales(query.item, (err, sum) => { 
    res.writeHead(200); 
    res.end(`Total sales for item ${query.item} is ${sum}`); 
  }); 
}).listen(8000, () => console.log('Started')); 

The server we created is very minimalistic; we only need it to expose the totalSales API.

Before we start the server for the first time, we need to populate the database with some sample data; we can do this with the populate_db.js script that is found in the code samples dedicated to this section. The script will create 100 K random sales transactions in the database.

Okay! Now, everything is ready. In order to start the server, as usual, we execute the following command:

node app

To query the server, simply navigate with a browser to the following URL:

    http://localhost:8000?item=book

However, to have a better idea of the performance of our server, we will need more than one request; so, we will use a small script named loadTest.js, which sends requests at intervals of 200 ms. The script can be found in the code samples of the book and it's already configured to connect to the URL of the server, so, to run it, just execute the following command:

node loadTest

We will see that the 20 requests will take a while to complete; take note of the total execution time of the test, because we are now going to apply our optimizations and measure how much time we can save.

When dealing with asynchronous operations, the most basic level of caching can be achieved by batching together a set of invocations to the same API. The idea is very simple: if we are invoking an asynchronous function while there is still another one pending, we can attach callback to the already running operation, instead of creating a brand new request. Take a look at the following figure:

The previous image shows two clients (they can be two different objects, or two different web requests) invoking the same asynchronous operation with exactly the same input. Of course, the natural way to picture this situation is with the two clients starting two separate operations that will complete at two different moments, as shown by the preceding image. Now, consider the next scenario, depicted in the following figure:

This second image shows us how the two requests—which invoke the same API with the same input—can be batched, or in other words, appended to the same running operation. By doing this, when the operation completes, both the clients will be notified. This represents a simple, yet extremely powerful, way to optimize the load of an application while not having to deal with more complex caching mechanisms, which usually require an adequate memory management and invalidation strategy.

const totalSales = require('./totalSales'); 
 
const queues = {}; 
module.exports = function totalSalesBatch(item, callback) { 
  if(queues[item]) {                                   // [1] 
    console.log('Batching operation'); 
    return queues[item].push(callback); 
  } 
 
  queues[item] = [callback];                          // [2] 
  totalSales(item, (err, res) => { 
    const queue = queues[item];                      // [3] 
    queues[item] = null; 
    queue.forEach(cb => cb(err, res)); 
  }); 
}; 

//const totalSales = require('./totalSales'); 
const totalSales = require('./totalSalesBatch'); 
 
http.createServer(function(req, res) { 
// ... 

One of the problems with the request-batching pattern is that the faster the API, the fewer batched requests we get. One can argue that if an API is already fast, there is no point in trying to optimize it; however, it still represents a factor in the resource load of an application that, when summed up, can still have a substantial impact. Also, sometimes we can safely assume that the result of an API invocation will not change so often; therefore, a simple request batching will not provide the best performance. In all these circumstances, the best candidate to reduce the load of an application and increase its responsiveness is definitely a more aggressive caching pattern.

The idea is simple: as soon as a request completes, we store its result in the cache, which can be a variable, an entry in the database, or in a specialized caching server. Hence, the next time the API is invoked, the result can be retrieved immediately from the cache, instead of spawning another request.

The idea of caching should not be new to an experienced developer, but what makes this pattern different in asynchronous programming is that it should be combined with the request batching, to be optimal. The reason is because multiple requests might run concurrently while the cache is not set, and when those requests complete, the cache will be set multiple times. 

Based on these assumptions, the final structure of the asynchronous request-caching pattern is shown in the following figure:

The preceding figure shows us the two phases of an optimal asynchronous caching algorithm:

Another crucial detail to consider is the unleashing Zalgo anti-pattern (we have seen it in action in Chapter 2, Node.js Essential Patterns). As we are dealing with asynchronous APIs, we must be sure to always return the cached value asynchronously, even if accessing the cache involves only a synchronous operation.

const totalSales = require('./totalSales'); 
 
const queues = {}; 
const cache = {}; 
 
module.exports = function totalSalesBatch(item, callback) { 
  const cached = cache[item]; 
  if (cached) { 
    console.log('Cache hit'); 
    return process.nextTick(callback.bind(null, null, cached)); 
  } 
 
  if (queues[item]) { 
    console.log('Batching operation'); 
    return queues[item].push(callback); 
  } 
 
  queues[item] = [callback]; 
  totalSales(item, (err, res) => { 
    if (!err) { 
      cache[item] = res; 
      setTimeout(() => { 
        delete cache[item]; 
      }, 30 * 1000); //30 seconds expiry 
    } 
 
    const queue = queues[item]; 
    queues[item] = null; 
    queue.forEach(cb => cb(err, res)); 
  }); 
}; 

//const totalSales = require('./totalSales'); 
//const totalSales = require('./totalSalesBatch'); 
const totalSales = require('./totalSalesCache'); 
 
http.createServer(function(req, res) { 
  // ... 

In Chapter 4, Asynchronous Control Flow Patterns with ES2015 and Beyond, we saw how promises can greatly simplify our asynchronous code, but they offer an even more interesting application when dealing with batching and caching. If we recall what we said about promises, there are two properties that can be exploited to our advantage in this circumstance:

In short, the first property is exactly what we need for batching requests, while the second means that a promise is already a cache for the resolved value and offers a natural mechanism for returning a cached value in a consistent asynchronous way. In other words, this means that batching and caching are extremely simple and concise with promises.

To demonstrate this, we can try to create a wrapper for the totalSales() API, using promises, and see what it takes to add a batching and caching layer. Let's see then what this looks like. Let's create a new module named totalSalesPromises.js:

const pify = require('pify');                    // [1] 
const totalSales = pify(require('./totalSales')); 
 
const cache = {}; 
module.exports = function totalSalesPromises(item) { 
  if (cache[item]) {                            // [2] 
    return cache[item]; 
  } 
 
  cache[item] = totalSales(item)                // [3] 
    .then(res => {                              // [4] 
      setTimeout(() => {delete cache[item]}, 30 * 1000); //30s expiry 
      return res; 
    }) 
    .catch(err => {                             // [5] 
      delete cache[item]; 
      throw err; 
    }); 
  return cache[item];                           // [6] 
}; 

The first thing that strikes us is the simplicity and elegance of the solution we implemented in the preceding code. Promises are indeed a great tool, but for this particular application they offer a huge, out-of-the-box advantage. This is what happens in the preceding code:

Very simple and intuitive, and more importantly, we were able to achieve both batching and caching.

If we now want to try the totalSalesPromise() function, we will have to slightly adapt the app.js module as well, because now, the API is using promises instead of callbacks. Let's do it by creating a modified version of the app module named appPromises.js:

const http = require('http'); 
const url = require('url'); 
const totalSales = require('./totalSalesPromises'); 
 
http.createServer(function(req, res) { 
  const query = url.parse(req.url, true).query; 
  totalSales(query.item).then(function(sum) { 
    res.writeHead(200); 
    res.end(`Total sales for item ${query.item} is ${sum}`); 
  }); 
}).listen(8000, () => console.log('Started')); 

Its implementation is almost identical to the original app module, with the difference that now we use the promise-based version of the batching/caching wrapper; therefore, the way we invoke it is also slightly different.

That's it! We are now ready to try this new version of the server by running the following command:

node appPromises

Using the loadTest script, we can verify that the new implementation is working as expected. The execution time should be the same as when we tested the server using the totalSalesCache() API.