Chapter 1: Introducing Node-RED and Flow-Based Programming

This chapter will help you grow from being a reader to being a Node-RED user. First, you'll learn about the history of Flow-based programming (FBP) tools, not just Node-RED. You will then gain a broad understanding of the entirety of Node-RED as a useful tool for building web applications and the Internet of Things (IoT) data handling, before learning what IoT and Node.js are in terms of Node-RED.

Providing technical content will help accelerate your software application development, but if you take a look at the history of the Node-RED tool itself, it will help you better understand why you need a FBP tool such as Node-RED. That is what we will be doing in this chapter.

More specifically, we'll be covering the following topics:

• What is FBP?
• What is Node-RED?
• Node-RED benefits
• Node-RED and IoT

Let's get started!

What is FBP?

So, what is FBP in the first place? It's the workflows you use in your work that you can easily imagine. Let's recall those workflows.

Workflows

In a normal workflow, boxes and wires indicate the process flow. It may be just one business design. Boxes represent processes. Box processing is defined by who, when, where, what, and how much. Sometimes, it's like explicitly writing out the flow of processing, such as by using swim lanes or placing writing definitions inside boxes. In any case, looking at the box should reveal what will be done.

On the other hand, let's try to summarize this business process as a document. Don't you think it will be complicated? Who will do what as they read it, even if they use some paragraphs well to put it together? When will you do it? It could be confusing:

Figure 1.1 – Workflow example

Now, let's get back to software programming. FBP is a kind of concept for software programming that defines an application with a data flow. Each part of the process is there as a black box. They communicate data between connected black boxes that have been predefined. FBP is said to be component-oriented because these black-box processes can be connected repeatedly to form several applications without needing to be modified internally. Let's explore FBP in more detail.

Flow-based programming (FBP)

I think FBP is a good blend of workflow and dataflow. FBP uses a data factory metaphor to define an application. It sees an application as a network of asynchronous processes that start at some point and do a single sequential process that does one operation at a time until it ends, rather than communicating by using a stream of structured chunks of data. This is called an information packet (IP). This view focuses on the data and its transformation process to produce the output that is needed. Networks are usually defined outside a process as a list of connections that is interpreted by a piece of software called a scheduler.

Processes communicate via fixed capacity connections. Connections are connected to processes using ports. The port has a specific name that is agreed on by the network definition and the process code. At this point, it is possible to execute the same code by using multiple processes. A particular IP is usually only owned by a single process or transferred between two processes. The port can be either a normal type or an array type.

FBP applications typically run faster than traditional programs, since FBP processes can continue to run as long as there is room to put in data and output to process. It does not require any special programming and makes optimal use of all the processors on the machine.

FBP has a high-level, functional style so that the behavior of the system can be easily defined; for example, in a distributed multi-party protocol such as a distributed data flow model, for accurately analyzing the criteria for determining whether a variable or statement behaves correctly:

Figure 1.2 – Simple FBP design example

Now that you have a solid understanding of FBP, let's learn how Node-RED can be implemented in this way.

What is Node-RED?

Node-RED is one of the FBP tools that we have described so far. Developed by IBM's Emerging Technology Services team, Node-RED is now under the OpenJS Foundation.

Overview

FBP was invented by J. Paul Morrison in the 1970s. As we mentioned earlier, FBP describes the behavior of the application as a black box network, which in Node-RED is described as a "node." Processing is defined in each node; data is given to it, processing is performed using that data, and that data is passed to the next node. The network plays the role of allowing data to flow between the nodes.

This kind of programming method is very easy to use to make a model visually and makes it easy to access for several layer users. Anybody can understand what the flow is doing if a problem is broken down into each step. That's why you don't need to the code inside the nodes:

Figure 1.3 – Node-RED Flow Editor as an FBP tool

Flow editor and runtime

Node-RED is not only a programming tool but also an execution platform that wraps up the Node.js runtime for applications that are built using Node-RED.

We need to use the flow editor to make Node-RED applications for IoT, web services, and more. The flow editor is also a Node.js web application. We will tell you how to use flow editor clearly in Chapter 3, Understanding Node-RED Characteristics by Creating Basic Flows.

The flow editor, which is the core function of Node-RED, is actually a web application made with Node.js. It works with the Node.js runtime. This flow editor operates within the browser. You must select the node you want to use from the various nodes in the palette and drag it to the workspace. Wiring is the process of connecting the nodes to each other, which creates an application. The user (developer) can deploy the application to the target runtime with just one click.

The palette that contains various nodes can easily be expanded as you can install new nodes created by developers, meaning you can easily share the flow you created as a JSON file to the world. Before we explore the benefits of Node-RED, let's look at the brief history behind its creation.

History and origin of Node-RED

In early 2013, Nick-O'Leary and Dave Conway-Jones from IBM UK's Emerging Technology Services Team created Node-RED.

Originally, it was a just proof of concept (PoC) to help visualize and understand the mapping between Message Queue Telemetry Transport (MQTT) topics, but soon, it became a very popular tool that could be easily extended to various uses.

Node-RED became open source in September 2013 and remains to be developed as open source now. It became one of the founding projects of the JS Foundation in October 2016, which has since merged with the Node.js Foundation to create the OpenJS Foundation, doing so in March 2019.

The OpenJS Foundation supports the growth of JavaScript and web technologies as a neutral organization to lead and keep any projects and fund activities jointly, which is beneficial to the whole of the ecosystem. The OpenJS Foundation currently hosts over 30 open source JavaScript projects, including Appium, Dojo, jQuery, Node.js, and webpack.

Node-RED has been made available under the Apache 2 license, which makes it favorable to use in a wide range of settings, both personal and commercial:

Figure 1.4 – Dave Conway-Jones and Nick O'Leary

Why is it Called Node-RED?

The official documentation (https://nodered.org/about/ states that the name was an easy play on words that sounded like "Code Red." This was a dead end, and Node-RED was a big improvement on what it was called in its first few days of conception. The "Node" part reflects both the flow/node programming model, as well as the underlying Node.js runtime.

Nick and Dave never did come to a conclusion on what the "RED" part stands for. "Rapid Event Developer" was one suggestion, but it's never been compelled to formalize anything. And so, the name "Node-RED" came to life.

Node-RED benefits

Let's think a little here. Why do you use cars? I think the answer is very simple and clear. First of all, we can come up with the answer that they are used as a means of transportation in a broad sense. There are other options for transportation, such as walking, bicycle, train, and bus. Then, we have the reasons for choosing a car from among these other options, as follows:

• You do not get exhausted.
• You can reach your destination quickly.
• You can move at your own pace.
• You can keep your personal space.

Of course, there are some disadvantages, but I think these are the main reasons for using a car. Although other means of transportation can also serve the same purpose, the important thing is to consider the advantages and disadvantages of each, and use the car as a transportation tool for the reason that you feel is the most suitable to you.

We can see the same situation in software. As an example, why do you use Word, Excel, and PowerPoint? You'll probably use Word because it's the most efficient way to write a document. However, you could use a word processor separately or handwrite anything. Similarly, instead of Excel, you can use any other means to make spreadsheets. There are also other means if you want to make presentation materials and make them look effective, besides PowerPoint. However, you are likely to choose the optimum tool for your situation.

Let's recall what Node-RED is for. It is a FBP tool, suitable for making data control applications for web applications and IoT. Its development environment and execution environment are browser-based applications made with Node.js, which makes their development as easy as possible.

So, what is the reason for using Node-RED, which provides these kinds of features? Do you want to avoid heavy coding? Do you not have coding skills? Yes, of course, these are also reasons to use the program.

Let's recall the example of a car. In a broad sense, our dilemma (transportation) is replaced here by developing (creating) a Node.js application for describing software tools. The transport options, such as cars, bicycles, trains, buses, ships, planes, and so on, are options, and with software development, we also have numerous options, such as using Node.js scratch, or using various frameworks of Node.js and using Node-RED. As for reasons to choose Node-RED, let's take a look at some essential points.

Simplification

When programming with Node-RED, you'll notice its simplicity. As the name no-code/low-code indicates, coding is eliminated and programming is intuitively completed with a minimal number of operations needing to be used.

Efficiency

The FBP typified by Node-RED can be completed with almost only GUI operations. Node-RED flow editor takes care of building the application execution environment, library synchronization, the integrated development environment (IDE), and editor preparation so that you can concentrate on development.

Common

As represented by object-oriented development, making the source code a common component is one of the most important ideas in development. In normal coding-based development, each common component exists in functions and classes, but in Node-RED, they exist as an easy-to-understand node (just a box). If you don't have a node as a common component you want to use, anyone can create one immediately and publish it to the world.

High quality

High quality is the true value of flow-based and visual programming. Each node provided as a component is a complete module that has been unit tested. As a result, app authors can focus on checking the operation at the join level without worrying about the contents of node. This is a big factor that eliminates human error at the single level and ensures high quality.

Open source

Node-RED is an open source piece of software. Therefore, it can be used flexibly under the Apache2 license. Some are developing their own services based on Node-RED, while others are changing to their own UI and deploying it as built-in. As we mentioned previously, we have also established a platform where we can publish our own developed node so that anyone can use it.

Node-RED library

The library indexes all Node-RED modules published to the public npm repository (https://www.npmjs.com/), assuming they follow the proper packaging guidelines.

This is the area in which we've seen the most community contribution, with well over 2,000 nodes available – which means there's something for everyone:

Figure 1.5 – Node-RED library

Various platforms

Node-RED can be used on various platforms. That's because Node-RED itself is a Node.js application, as we mentioned previously. If you have a runtime environment for Node.js, you can run it. It is mostly used on Edge devices, cloud services, and in embedded formats.

You can get a sense of this by understanding the relationship between Node-RED and IoT and the architecture of IoT, which will be explained in the next section.

Node-RED and IoT

Again, Node-RED is a virtual environment that combines hardware devices, APIs, and online services in a revolutionary way on a browser. It provides the following features:

• Browser-based UI.
• Works with Node.js and is lightweight.
• Encapsulates function and can be used as a node (meaning functions are locked in an abstract capsule) .
• You can create and add your own nodes.
• Easy access to IBM Cloud services.

In other words, it can be said that this tool is suitable for building IoT-related services, such as data control on devices, and linking edge devices and cloud services. Originally, the development concept of Node-RED was for IoT, so this makes sense.

Now, let's look at the basic structure of IoT so that those who are only vaguely aware of IoT can understand it. It can be said that IoT is basically composed of six layers, as shown in the following diagram:

Figure 1.6 – IoT six layers

Let's take a look at these in more detail.

Device

The device is a so-called edge device. IoT has various sensors and handles the data that's acquired from them. Since it doesn't make sense to have the data only on the edge device, we need to send that data through the gateway to the network.

Network

This is the network that's required to send the data that's been obtained from the device to a server on the internet. It usually refers to the internet. In addition to the internet, there is also a P2P connection via Bluetooth or serial.

Platform

The party that receives and uses the data is the platform. We may also have a database for activating and authenticating things, managing communications, and persisting received data.

Analytics

This is a layer that analyzes the received data. Broadly speaking, it may be classified as an application. This is the part that prepares the data so that it can be processed into a meaningful form.

Application

An application provides a specific service based on data analysis results. It can be a web or mobile application, or it can be a hardware-specific embedded application. It can be said to be the layer that's used by the end user of the IoT solution.

Now that we have an understanding of IoT, we will examine why Node-RED should be used for it.

Node-RED and IoT

While explaining IoT so far, we've made it clear why Node-RED is suitable for IoT. For example, you can understand why FBP tools that have been developed for IoT survive when used with Node-RED. In particular, the following three points should be taken into account:

• Since it can be run on edge devices (pre-installed on specific versions of Raspberry Pi OS), it is ideal for data handling at the device layer.
• Since it can be run on the cloud (provided as a default service in IBM Cloud), it is easy to link with storage and analysis middleware.
• Since MQTT and HTTP protocols can be covered, it is very easy to exchange data between the edge device and the server processing cloud.

In this way, Node-RED, which largely covers the elements required for IoT, is now used for a wide range of applications, such as web services and chart display, as well as programming for IoT. Also, as of June 2020, if you look at Google Trends for Node-RED, you can see that the number of users is gradually increasing. As such, Node-RED is a very attractive FBP tool:

Figure 1.7 – Google Trends for "Node-RED"

A typical edge device that can use Node-RED is Raspberry Pi. Of course, it is possible to use Node-RED on other platforms, but it goes well with Raspberry Pi, which also has a pre-installed version of the OS.

Raspberry Pi OS Supports Node-RED

Node-RED has also been packaged for the Raspberry Pi OS repositories and appears in their list of recommended software. This allows it to be installed using apt-get install Node-RED and includes the Raspberry Pi OS-packaged version of Node.js, but does not include npm. More information can be found at https://nodered.org/docs/getting-started/raspberrypi.

IBM Cloud is a typical cloud platform that can use Node-RED. Of course, you can use Node-RED on other clouds, but IBM Cloud provides a service that anyone can easily start.

Important Note

Node-RED is available on the IBM Cloud platform as one of its Starter Kits applications in their catalog. It is very easy to start using the flow editor as a web application on IBM Cloud (https://nodered.org/docs/getting-started/ibmcloud).

Summary

In this chapter, you learned what FBP and Node-RED are. Due to this, you now understand why Node-RED is currently loved and used by lots of people as an FBP tool. At this point, you may want to build an application using Node-RED. In the next chapter, we'll install Node-RED in our environment and take a look at it in more depth.