Of JavaScript Runtimes and Building (Mostly) Servers

Ryan Dahl – that cat has some talent, I tell ya!

Ryan is the creator of a fantastic piece of software called Node.js (or just plain Node, as it is often written, and as I’ll write it from here on out). Ryan first presented Node at the European JSConf in 2009, and it was quickly recognized as a potential game-changer, as evidenced by the standing ovation his presentation received (I presume Ryan is an excellent presenter generally as well).

Node is a platform for running primarily, though not exclusively, server-side code that has high performance and is capable of handling large request loads with ease. It is based on the most widely used language on the planet today: JavaScript. It’s straightforward to get started with and understand, yet it puts tremendous power in the hands of developers, in large part thanks to its asynchronous and event-driven model of programming. In Node, almost everything you do is nonblocking, meaning code won’t hold up the processing of other request threads. Most types of I/O, which is where blocking comes into play most, are asynchronous in Node, whether it’s network calls or file system calls or database calls. This, plus the fact that to execute code, Node uses Google’s popular and highly tuned V8 JavaScript engine, the same engine that powers its Chrome browser, makes it very high performance and able to handle a large request load (assuming that you as the developer don’t botch things of course!).

It’s also worth noting that, as weird as it may sound, Node is single-threaded. It at first seems like this would be a performance bottleneck, but in fact, it’s a net benefit because it avoids context-switching. However, this is a little bit of a misnomer in that it’s more correct to say that Node is event-driven and single-threaded with background workers. When you fire off some type of I/O request, Node will generally spawn a new thread for that. But, while it’s doing its work, that single event-driven thread continues executing your code. All of this is managed with an event queue mechanism so that the callbacks for those I/O operations are fired, back on that single thread, when the responses come back. All of this means that there is no (or at least minimal) context-switching between threads but also that the single thread is never sitting idle (unless there is literally no work to do of course), so you wind up with that net positive benefit I mentioned.

None of these technical details are especially important to use as a Node developer, but the performance it yields is what makes it no wonder that so many significant players and sites have adopted Node to one degree or another. These aren’t minor outfits we’re talking about, we’re talking names you doubtless know, including DuckDuckGo, eBay, LinkedIn, Microsoft, Netflix, PayPal, Walmart, and Yahoo, to name just a few examples. These are large businesses that require top-tier performance, and Node can deliver on that promise (again, with the caveat that you as the developer don’t mess things up, because that’s always possible).

Node is a first-class runtime environment, meaning that you can do such things as interacting with the local file system, access relational databases, call remote systems, and much more. In the past, you’d have to use a “proper” runtime, such as Java or .Net to do all this; JavaScript wasn’t a player in that space. With Node, this is no longer true. It can compete not only on performance but also in terms of what capabilities it provides to developers. If you can think of it, chances are you can do it with Node, and that wasn’t always the case with JavaScript.

To be clear, Node isn’t in and of itself a server. You can’t just start up Node and make HTTP requests to it from a web browser. It won’t do anything in response to your requests by default. No, to use Node as a server, you must write some (straightforward and concise, as you’ll see) code that then runs on the Node “runtime.” Yes, you effectively write your own web server and app server, if you want to split hairs (or potentially FTP, Telnet, or any other type of server you might wish to). That’s a very odd thing to do as a developer – we usually apply the “don’t reinvent the wheel” mantra for stuff like that and pull one of the hundreds of existing options off the shelf. Plus, writing such servers sounds (and probably actually is) daunting to most developers, and for good reason! To be sure, it absolutely would be if you tried to write a web server from scratch in many other languages, especially if you want it to do more than just serve static content files. But not with Node!

But remember, acting as a server is just one capability that Node provides as a JavaScript runtime, and it can provide this functionality only if you, as a developer, feed it the code it needs to do so! In fact, a great many developer tools, and other types of apps, use Node as their runtime nowadays. Node really is all over the place!

Node allows you to use the same language and knowledge on both client and server, something that was difficult to accomplish before. In fact, aside from Java and some Microsoft technologies (see project Blazor, which seeks to do the same thing with C#, if you’re curious), there never has really been an opportunity to do so until Node came along. It’s a pretty compelling opportunity.

Another critical aspect of Node is a driving design goal of the project, which is keeping its core functionality to an absolute minimum and providing extended functionality by way of APIs (in the form of JavaScript modules) that you can pick and choose from as needed. Node gets out of your way as much as possible and allows you only to introduce the complexity you really need, when you need it. Node ships with an extensive library of such modules, but each must be imported into your code, and then there are literally thousands of other modules that you can bring in as needed, some of which you’ll see as we progress throughout this book.

In addition to all of this, getting, installing, and running Node are trivial exercises, regardless of your operating system preference. There are no complicated installs with all sorts of dependencies to manage, nor is there a vast set of configuration files to mess with before you can bring up a server and handle requests. It’s a five-minute exercise, depending on the speed of your Internet connection and how fast you can type! There is also no required tooling to work with Node. In fact, a simple text editor is enough, in simplest terms (though that isn’t to say you won’t want a robust IDE with Node support later, but at least for this book I won’t be assuming anything other than Notepad or some equivalent text editor).

All of this makes working with Node so much more straightforward than many competing options while providing you with top-notch performance and load handling capabilities. Moreover, it does so with a consistent technological underpinning as that which you develop your client applications.

That’s Node in a nutshell!

Next, let’s see about getting it onto your machine so that you can start playing with some code together and we can look at Node in a little more depth.

First Baby Steps with Node: Installation

To get started, there’s only one address to remember:

http://nodejs.org

That’s your one-stop shop for all things Node, beginning, right from the front page, with downloading it, as you can see in Figure 1-1.

Usually, I would tell you to install the latest version available, but in this case, it might be better to choose a long-term support (LTS) version, because they tend to be more stable. However, it shouldn’t (he said, with fingers crossed) matter which you choose, for the purposes of this book. For the record, however, I developed all the code using version 10.16.3, so if you encounter any problems, I would suggest choosing that version, which you can get from the Other Downloads link and then the Previous Releases link (you’ll be able to download any past version you like from there).

The download will install in whatever fashion is appropriate for your system, and I leave this as an exercise for the reader. For example, on Windows, Node provides a perfectly ordinary and straightforward installer that will walk you through the necessary (and extremely simple) steps. On macOS X, a typical install wizard will do the same.

Once the install completes, you will be ready to play with Node. The installer should have added the Node directory to your path. So, as a first simple test, go to a command prompt or console prompt, type node, and press Enter. You should be greeted with a > prompt. Node is now listening for your commands in interactive mode. To confirm, type the following:

console.log("Hello, you ugly bad of mostly water!");

Press Enter, and you should be greeted with something like what you see in Figure 1-2 (platform differences excepted, I’m a Windows guy myself, unashamedly, so that’s where the screenshots throughout this book will be from, perhaps with a few exceptions later).

If you find that this doesn’t work, please take a moment and ensure that Node is indeed in your path. It will make things a lot easier going forward.

More Useful: Executing JavaScript Source Files

Interacting with Node in CLI (Command-Line Interface) mode like this is fine and dandy, but it’s limited. What you really want to do is execute a saved JavaScript file using Node. As it happens, that’s easy to do. Create a text file named test.js (it could be anything, but that’s a pretty good choice at this point), and type the following code into it (and, of course, save it):

let a = 5;

let b = 3;

let c = a * b;

console.log(`${a} * ${b} = ${c}`);

To execute this file, assuming you are in the directory in which the file is located, you simply must type this:

node test.js

Press Enter after that, and you should be greeted with an execution, such as the one you see in Figure 1-3.

Clearly, this little bit of code is unexceptional, but it does demonstrate that Node can execute plain old JavaScript just fine. It demonstrates that we’re dealing with at least the ES2015 specification as well, being more specific, thanks to the use of let and template literals (or string interpolation if you prefer). You can experiment a bit if you like, and you will see that Node should run any basic JavaScript that you care to throw at it like this.

Node’s Partner in Crime: NPM

You can visit it through a web browser and look through all the packages available, which makes finding exactly what you need easy.

Here, npm is the CLI program that is NPM itself, and install is one command you can issue to it. Then, express is an argument to that command, and this is the general form that most of your interactions with NPM will take.

If you execute that, you’ll find that a directory called node-modules has been created, and inside it will be a lot of…well, a lot of stuff you typically don’t need to worry about too much! In short, though, it’s all the code that makes up the Express module (which doesn’t matter right now, but is a JavaScript module, or package if you prefer, which we’ll be using in the MailBag app a few chapters hence… but we’ll get to that app in due time, we’ve got a fair bit of ground to cover before then, so for now suffice it to say it’s one of the two apps we’re going to be building with the technologies discussed over the first six chapters), plus whatever modules Express itself depends on (and whatever they might depend on, and so on). NPM takes care of fetching all those dependencies for you. You’ll also notice a file named package-lock.json has been created, and for our purposes here, you don’t need to worry about that except to know not to delete it as NPM needs it to do its job.

Now, Express will be installed in a location outside the current directory and will be shared by all Node projects (or, more precisely, it will be available to all projects, because, of course, a project won’t use a globally installed module unless you tell it to). Most usually, you will want to install dependencies in the project cache so that different projects can use different version of a given module than other projects (there is always a single version of a given module in the global cache, if any are present at all).

Initializing a New NPM/Node Project

Now, in all of this, I did skip one step that clearly is optional but is, in fact, typical, and that’s initializing a new project. With most Node/NPM projects, you’ll also have a file named package.json in the root directory of the project. This file is the project manifest file, and it provides metadata information to NPM (and Node, at least indirectly) about your project that it needs to do certain things. It will tell NPM what modules to install if they haven’t been installed yet for one thing (which makes giving a project to another developer very easy!). It will also contain information like the name and version of the project, its main entry point, and lots of other information (most of which is optional, but we’ll look at that a bit more in the next chapter).

While you can write this file by hand or even go entirely without it, it’s a good idea to have it, and it’s a good idea to let NPM create it for you, which you can do by executing this command:

npm init

If you are following along, please make sure the directory you run this from is empty (delete node_modules and package-lock.json if present, both of which will be described later). This will trigger an interactive process that walks you through the creation of the package.json file, something like you see in Figure 1-4.

This will walk you through an interactive, step-by-step process wherein you can enter whichever information is relevant to your project, if any. You can just hit Enter on each option to use the default (or a blank value, whichever is applicable), or you can enter the values that are appropriate to you. For our purposes here though, you indeed can and should simply hit Enter on each prompt in the process.

Opening the generated package.json file should look something like this:

{

  "name": "temp",

  "version": "1.0.0",

  "description": "Init'ing a project",

  "main": "test.js",

  "scripts": {

    "test": "echo "Error: no test specified" && exit 1"

  },

  "author": "Frank W. Zammetti",

  "license": "ISC"

}

Fisher Price’s “My First Node Web Server”

Now that you know a bit about Node and NPM, the very basics at least, let’s write some actual code, beyond the simple example shown earlier, that is, and run it with Node.

What this does is it effectively defines a custom command for NPM. The start command is one that already exists, but it’s one that does nothing until you add this entry in package.json, so it may as well not exist! Once you add that entry though, NPM will look for that start key, take its value, and execute whatever the command is that you provide in it. The benefit of doing this is that every project you create with Node and NPM will be startable the same way. Without this, a developer would need to figure out what file is the main entry point to launch it with Node (and note that the main key in package.json may not be enough to tell someone this, as is the case here, since the default value of index.js would be wrong for this project).

Yep, that’s it! NPM knows what to do now and will launch Node and tell it to execute server.js.

Now fire up your favorite web browser and visit http://127.0.01. You’ll be greeted with the text “Hello from my first Node Web server”. Note, however, that if anything else on your system is already listening on port 80 then the app won’t actually start, you’ll get an error instead. In that case, simply change the listen(80) call to a free port and you’ll be good to go (and, naturally, add the port to the end of the URL in that case too).

If that isn’t a little bit amazing to you, then you’ve probably seen the Flying Spaghetti Monster (FSM) travel one too many times around your neighborhood and have been totally desensitized to the amazing! (FSM – yeah, uhh, I’m not gonna even try and explain what the FSM is; here’s a link: www.venganza.org).

Obviously, this is a simplistic example, but it should get the basic idea across well enough. But what exactly is going on in that simple example at a code level? As it happens, quite a bit, and most of it is key to how Node works.

The first concept is the idea of importing modules. In the example code, http is a module. This is one of the core Node modules that Node comes with out of the box, and, as such, it is compiled directly into the Node binary. Therefore, you won’t find a separate JavaScript file for it in the Node installation directory for it. This is true of all the Node core modules, all of which you can find in the Node documentation on the Node site. To import any of them, you just require() them by name.

You can create your own modules too just by adding other .js files to your project and require()-ing them. This gets a little more involved, with discussions of things like scope and exports, and we’ll get to all of that in time. But for now, I wanted to mention it at least in case you really are entirely new to Node so that you can find the appropriate section in the Node docs to describe this if you want to jump ahead.

The require() function returns an object that is essentially the API provided by the module. This object can include methods, attributes, or whatever you want. In fact, it could conceivably be just a variable with some data in an array. More times than not, though, it will be an object with some methods and attributes. In the case of http in this example, one of the methods the object returned is createServer(). This method creates a web server instance and returns a reference to it.

The argument you pass to this method is a function that serves as a request listener; that is, the function executed any time a request is made to the server.

You can then perform some branching logic on any or all of these, perhaps access a database or other durable storage mechanism, and return an appropriate and fully dynamic response for the specific request.

Creating a web server alone won’t actually do anything. It won’t respond to requests until you do a little more work. As mentioned, the createServer() method returns a reference to the web server instance, which itself contains the method listen(). That method accepts a port number on which the server should listen and, optionally, the hostname/IP address on which to listen. In the example, the standard HTTP port 80 is specified, and by default, the local machine loopback address 127.0.0.1 is used if no IP address is specified, as is the case here. Once you call this method, the server will begin listening for requests (i.e., assuming nothing else is already using that port on your system!), and for each request that comes in, it will call the anonymous function passed to createServer().

This callback function (callback functions being one mechanism by which Node provides nonblocking functionality, the others being Promises and async/await) receives two arguments, inRequest and inResponse, which are objects representing the HTTP request and response, respectively. In this simple example, all this callback function does is call the end() method on the response object, passing the response you want to send back. By default, an HTTP 200 response code header will be added automatically, so this completes the handling of a given request.

With just this little bit of code, you, in fact, know the basics of what you would require for writing a server for the MailBag app later! But, when we get to that app, we’ll use something a little more robust (Express, as mentioned earlier), but this gives you a fundamental idea of what it takes.

Summary

In this chapter, we looked at Node and NPM and discussed the very basics of their usage. But, because Node and NPM are, conceptually, pretty simple things, these basics are, by and large, all you need to write real applications. You now know how to execute JavaScript code, how to create an NPM project, and how to add dependencies. You understand the difference between the global cache and the local (project) cache, and you even know how to write a basic web server!

In the next chapter, we’ll continue looking at these two tools in a bit more detail, getting to some slightly more advanced stuff with them, to expand the foundation from which we’ll build our two apps later in the book.