Chapter 6: Implementing Node-RED in the Cloud

In this chapter, we will learn how to utilize Node-RED, which can be used standalone on a cloud platform (mainly Platform as a Service). Platform as a Service (PaaS) provides an instance that acts as the execution environment for an application, and the application developers only focus on executing the application created by themselves without using their power to build the environment. Node-RED is actually a Node.js application, so you can run it wherever you have a runtime environment for Node.js.

There are various major mega clouds such as Azure, AWS, and GCP, but Node-RED is prepared as a Starter App (a web application that can be launched on IBM Cloud is called a Starter App) by default in IBM Cloud, so we will use it in this chapter.

In this chapter, we'll cover the following topics:

• Running Node-RED on the cloud
• What is the specific situation for using Node-RED in the cloud?
• IoT case study spot on the server side
• Making a sample flow

By the end of this chapter, you will have mastered how to build a flow for handling sensor data on the cloud.

Technical requirements

The code that will be used in this chapter can be found in the Chapter06 folder at https://github.com/PacktPublishing/-Practical-Node-RED-Programming.

Running Node-RED on the cloud

This time, we will use IBM Cloud. The reason for this is that IBM Cloud has Node-RED Starter Kit on it. This is a kind of software boilerplate that includes services needed for Node-RED on the cloud, such as a database, CI/CD tools, and more.

If you have not used IBM Cloud yet, don't worry – IBM provides a free IBM Cloud account (Lite account) with no credit card registration needed. You can register for an IBM Cloud Lite account at http://ibm.biz/packt-nodered.

Before using Node-RED on IBM Cloud, you need to finish the registration process for your IBM Cloud Lite account.

Important Note

In this book, we strongly recommend that you select a Lite account when using IBM Cloud. You can upgrade from a Lite account to a standard account (PAYG/Pay as you go) at your own will. This means you can automatically upgrade to PAYG by registering your credit card.

Please note that services that can be used free of charge with a Lite account may be charged for with PAYG.

Now, let's launch Node-RED on IBM Cloud by following these steps:

Important Note

The instructions/screenshots provided here are correct at the time of writing. The UI of IBM Cloud changes so often that it might be different from the current UI.

1. Log in to IBM Cloud (https://cloud.ibm.com) with the account you created previously:
   

   Figure 6.1 – Logging in via your Lite account

2. After logging into IBM Cloud, you will see your own dashboard on your screen. If this is your first time using IBM Cloud, no resources will be shown on the dashboard:
   

   Figure 6.2 – IBM Cloud dashboard

   Next, we will create Node-RED on this cloud platform.

3. We will create Node-RED as a service on this cloud. Click App Development from the menu at the top left and click the Get a Starter Kit button. This lets you create a new application service:
   

   Figure 6.3 – Get a Starter Kit button

4. You can find Node-RED if you type Node-RED into the search text box. Once you've found it, click on the Node-RED panel:
   

   Figure 6.4 – Node-RED Starter Kit

5. After clicking on the Node-RED panel, we need to set some items.

   You can freely change each item by providing your own values, but in this chapter, the values that have been set here will be used for explanation purposes.

   See Figure 6.5 for the settings and values to configure. Please note that once they are set, these items cannot be changed later.

6. After setting all the items, click the Create button:
   

   Figure 6.5 – Create Node-RED as a Node.js application

   You have now created the framework for the applications that make up Node-RED. After this, you will be redirected to the App Details screen automatically, where you will be able to see that the Cloudant instance of the linked service has also been provisioned.

   However, only the application source code and the instance of the cooperation service are created, and they haven't been deployed to the Node.js execution environment on IBM Cloud yet. The actual deployment will be done when the CI/CD toolchain is enabled.

7. When everything is ready, click on the Deploy your app button in the center of the screen to enable it:
   

   Figure 6.6 – Deploying your Node-RED application

8. After clicking the Deploy your app button, move to the application settings window.
9. You will be asked to create an IBM Cloud API Key. Don't worry about this, as one will be generated automatically. Click the New button to open a new popup window, and then the OK button on the popup window. Once you do this, an IBM Cloud API Key will be generated:

   IBM Cloud API Key

   The IBM Cloud API Key is used to control your IBM Cloud account and various services (for example, it's Cloud Foundry in this tutorial). You can use this to issue a token for external access to services on IBM Cloud, for example. You can find out more about the IBM Cloud API Key here: https://cloud.ibm.com/docs/account?topic=account-manapikey.

   

   Figure 6.7 – Generating an IBM Cloud API Key

10. Select the resource spec on the window.

    This time, we are using IBM Cloud with a Lite account, so we have only 256 MB of memory available for all our services on IBM Cloud. So, if we use 256 MB for the Cloud Foundry Node.js service, we won't be able to use more memory for other services. But Node-RED needs 256 MB to run on IBM Cloud, so please use 256 MB here. It is already allocated 256 MB for the instance by default, so click the Next button, with no parameters changed:

    

    Figure 6.8 – Node.js runtime instance details

    Once you've done this, a DevOps toolchain setting screen will be displayed.

11. Click the Create button, with the default values filled in.

    You can change the DevOps toolchain name to any name you like. This is the name that identifies the toolchain you've created in IBM Cloud:

    

    Figure 6.9 – Configure the DevOps toolchain window

    Now, you are ready to use the environment (Node.js runtime and DevOps toolchain) to run the Node-RED application you created in the previous step. The Node-RED application you created is automatically deployed on the Node.js runtime through the toolchain.

12. Confirm that the Status that's displayed in the Delivery Pipelines (pipeline for executing each tool in the DevOps toolchain) area is Success, and click the toolchain's name (Node-REDforPackt, in this case) above it:
    

    Figure 6.10 – Checking the status of Node-RED and moving to the Pipeline tool

    In Delivery Pipelines, check that the statuses of both the BUILD and DEPLOY panels are green and displaying STAGE PASSED.

13. Click on View console under LAST EXECUTION RESULT on the DEPLOY panel:
    

    Figure 6.11 – Checking the status of each stage and moving to App Console

14. On the console screen of the Node-RED application, confirm that the status is Running, and then click View App URL:
    

    Figure 6.12 – Checking that Node-RED is running and opening Flow Editor

    Great work! You opened the Node-RED flow editor on IBM Cloud. Next, we will start to use the Node-RED flow editor you just opened.

    If you got any errors while performing these steps, it would be best for you to delete Cloud Foundry App, Cloudant, and DevOps toolchain and recreate them by following the same steps mentioned previously.

15. Set up a Username and Password to access your flow editor on IBM Cloud.

    After clicking on Visit App URL, you will be redirected to the initial setup dialog so that you can use Node-RED flow editor on IBM Cloud.

    You can proceed through this dialog by clicking each Next button, though please note that you should select Secure your editor so only authorised users can access it with Username and Password in order to log in to your own flow editor. This is because this flow editor is on IBM Cloud as a public web application. This means that anybody can access your flow editor if the URL is known. So, I strongly recommend that you select this option and set your own Username and Password values:

    

    Figure 6.13 – Setting a username and password to access flow editor

    We're almost done!

16. Click on the Go to your Node-RED flow editor button and then log in with the Username and Password details that you set in the previous step:
    

    Figure 6.14 – Logging into your Node-RED flow editor

    Next, we will check Node-RED flow editor on IBM Cloud and see if it is available.

17. Click the inject node and check the result:

Figure 6.15 – Default sample flow

When you click the inject node, you will see the resulting value on the debug tab:

Figure 6.16 – Checking the result

Now, you can create a flow in Node-RED on IBM Cloud. The Node-RED flow editor is always running as a Node.js application on IBM Cloud. This means that the Node.js runtime service (instance) is enabled on IBM Cloud. In other words, unlike Node-RED running on Raspberry Pi, this version of Node-RED accesses the flow editor via the internet.

In the next section, I will explain a little about situations where Node-RED is used on such a cloud.

What is the specific situation for using Node-RED in the cloud?

Let's revisit the situation where Node-RED is used in the cloud.

As we mentioned in the previous chapter, Node-RED is both a tool and an execution environment for creating Node.js applications written in Node.js. As a reason to build an application with Node-RED, I explained that by black boxing individual units of data processing, the role of each process becomes very clear, and it is easy to build and maintain.

This is the same reason not only on the edge device, but also on the server side (cloud side), for persisting, analyzing, and visualizing the data that's collected by the edge device.

The biggest feature of Node-RED is that it connects the processing of Node.js in a sequential manner or in parallel with input/output data chunks in the form of messages. It can be said that this is very suitable for IoT data handling.

Again, as we discussed in the previous chapter, the standard architecture for IoT solutions is built on edge devices and cloud platforms. It sends the sensor data acquired by the edge device to the cloud, makes it persistent, and processes it for the desired processing chain.

This chapter will focus on that part of the cloud.

The edge device and the cloud don't actually connect yet. Assuming that the data has been passed to the cloud, let's make the data persistent in the database and visualize it.

We're going to use a dashboard node that is popular with all developers for Node-RED on IBM Cloud.

Before you use Node-RED on IBM Cloud, please install a new node; that is, node-red-dashboard.

Node-RED provides the palette manager, which is easy to install and is used to install extra nodes directly. This is very helpful when you're using lots of nodes. However, it might have issues due to the limited memory of the Node-RED application of an IBM Cloud Lite Account.

So, here, we need to get the node-red-dashboard node in order to edit the application's package.json file and redeploy the Node-RED application on IBM Cloud.

You can read about this node at https://flows.nodered.org/node/node-red-dashboard.

Follow these steps to make changes in the package.json file:

1. On the Node-RED App details page of IBM Cloud, click source. This will redirect you to a Git repository where you can edit the Node-RED application source code:
   

   Figure 6.17 – Accessing your application source

2. Click on package.json on the file list. This file defines the module dependencies of your application:
   

   Figure 6.18 – Selecting package.json

3. Click the Edit button and add the following entry to the dependencies section:

   "node-red-dashboard": "2.x",

4. Add any commit message and click on the Commit changes button:

Figure 6.19 – Editing package.json and adding node-red-dashboard

After this, the Continuous Delivery Pipeline will automatically start to build and deploy the Node-RED application. You can check the status on the Delivery Pipeline at any time, just like you did while creating Node-RED Starter App:

Figure 6.20 – Rebuilding and redeploying your application automatically

When you get Deploy Stage failed with the memory limit error for lite account, please stop your Node-RED service on your IBM Cloud dashboard and after that run the Deploy Stage. You can stop your Node-RED service by accessing your IBM Cloud dashboard and clicking on Cloud Foundry apps under Resource summary:

Figure 6.21 Selecting Cloud Foundry apps

After that, click on the stop option on the Node-RED record under the Cloud Foundry apps.

Figure 6.22 Clicking the Stop option

That's all. You can confirm that the dashboard node has been added by closing the Palette management screen and scrolling down the left-hand side of the flow editor, as shown in the following screenshot:

Figure 6.23 – Checking that the dashboard node has been installed

There's one more thing: we need to use a database, but IBM Cloud's version of Node-RED has a Cloudant database by default. We will use Cloudant for the case study in the next section.

Now, you can use Node-RED on IBM Cloud for IoT server-side situations.

IoT case study spot on the server side

Now, let's consider a server-side case study for IoT.

It does not depend on the case of each edge device. It primarily serves to process data and store it in a database for visualization.

In this chapter, we'll consider the use case of IoT; that is, assuming that the sensor data that's received using the sensor module is received on the server side, and the subsequent processing part.

The difference from the previous chapter is that in this server-side processing tutorial, the content of the data doesn't make much sense. The main purpose is to save the received data and visualize it as needed, so I would like to define the following two use cases.

Use case 1 – Storing data

The first case is to store data. Let's create an application (flow) that stores data you receive from devices. In this section, we don't use real data from devices; we just use the data generated by the inject node instead:

Figure 6.24 – Use case 1 overview

Now, let's look at the second use case.

Use case 2 – Temperature/humidity sensor

The second case is to show data as graphs or charts. Let's create an application (flow) that publishes data you received from devices, on the dashboard. We won't be using real data from any devices, just the data that's been generated by the inject node:

Figure 6.25 – Use case 2 overview

As we mentioned earlier, we will use Cloudant for the database for case 1 and the dashboard for the graph display for case 2. These have already been prepared.

Making a sample flow

Now, let's create these two server-side case flows on the Node-RED flow editor.

Please check again that the Cloudant node and the Dashboard node have already been installed on your flow editor. If you don't have them, please install these nodes by following the steps mentioned in the What is the specific situation for using Node-RED on the cloud? section of this chapter.

Now, you need to prepare a specific database for this tutorial on Cloudant. Follow these steps:

1. Access your IBM Cloud dashboard and click View all from the Resource summary area:
   

   Figure 6.26 – IBM Cloud dashboard view

2. You will find the Cloudant service that you created using Node-RED. Please click the service's name:
   

   Figure 6.27 – Selecting the Cloudant service from the Resource list

3. Click the Launch Dashboard button at the top left of IBM Cloud:
   

   Figure 6.28 – Launching the Cloudant dashboard

4. After launching the Cloudant dashboard, please click Create Database and enter a name for your database. You can name it whatever you want; here, we have used packt_db. After that, click the Create button:

Figure 6.29 – Creating a new database on Cloudant

Now that you have created the database for this tutorial, you can use it at any time!

Making a flow for use case 1 – storing data

With IoT, server-side processing starts from the point where it is received from the edge device. However, as we mentioned earlier, we will focus on storing the data in the database, so we will be using dummy data that will be generated by the inject node. The chunk of data that's received as a message is persisted in the Cloudant database on Node-RED.

We can make the flow by following these steps:

1. Place an inject node and a cloudant out node from the palette on the left-hand side of the flow editor by dragging and dropping them into the workspace:
   

   Figure 6.30 – Placing the Inject node and cloudant out node

   The inject node generates dummy data, while the cloudant out node stores the input value as-is in the Cloudant database.

2. After that, we will also create a flow to retrieve data from Cloudant, but first, let's just create the flow for saving data. Wire these nodes:
   

   Figure 6.31 – Wiring these two nodes

3. Next, modify the settings of the inject node. Double-click the node to open the Settings panel.
4. Select JSON for the first parameter; that is, msg.payload, and click the right-hand side […] button to open the JSON editor:
   

   Figure 6.32 – JSON on the first parameter of the inject node

   You can use both types of editor here; that is, the Text editor or the Visual editor. You can add any values to the JSON style, but here's what we have used for the JSON data:

   {"temp":"29.18", "humi":"55.72"}

   You can switch between the Text editor and the Visual editor using tabs. Please refer to the following image:

   

   Figure 6.33 – Two types of JSON editor are available

   There's no need to edit msg.topic.

5. After setting the JSON data, click the Done button in the top-right corner to close the Settings panel.
6. Then, edit the settings for the cloudant out node. This is simple: just enter packt_db as the database name. This name is the database you named on the Cloudant dashboard.

   The first parameter, Service, is set automatically; it is your Cloudant service on IBM Cloud. The third parameter, Operation, does not need to be changed from its default value.

7. After setting the database name, click the Done button in the top-right corner to close the Settings panel:
   

   Figure 6.34 – Setting the database name on the cloudant out node

8. That's it! Don't forget to click the Deploy button.
9. Click the button on the inject node to run the flow. The data will be stored on the Cloudant database when the button has been clicked.

At this point, we can't check the data on Cloudant via the Node-RED flow editor; we can only check it on the Cloudant dashboard:

Figure 6.35 – Result on the Cloudant dashboard

Now, let's make a flow that gets the data from Cloudant by following these steps:

1. Place an inject node, a cloudant in node, and a debug node from the palette on the left-hand side of the flow editor by dragging and dropping them into the workspace from the previous flow.

   The inject node just executes this flow as a trigger, so there's no need to change the parameters in it. The cloudant in node gets the data from your Cloudant database. The debug node outputs a log on the debug tab.

2. Wire these nodes:
   

   Figure 6.36 – Placing new three nodes and wiring them to get data

3. Next, modify the settings of the cloudant in node by double-clicking the node to open its Settings panel.
4. Just like the cloudant out node, enter packt_db as the database's name and select all documents for the third parameter' that is, Search by.

   The first parameter, Service, is set automatically; it is your Cloudant service on IBM Cloud.

5. After setting the database name and search target, click the Done button in the top-right corner to close the Settings panel:
   

   Figure 6.37 – Setting the database name and searching for a target on cloudant in the node

6. That's it! Don't forget to click the Deploy button.
7. Click the button on the inject node to run the flow. You will get the data from the Cloudant database when you do so.

   You will see that the result was output to the debug window:

Figure 6.38 – Result of getting the data from Cloudant

Congratulations! With this, we have successfully created a basic flow (application) that stores sensor data on a database with Node-RED.

You can also download this flow definition file here: https://github.com/PacktPublishing/-Practical-Node-RED-Programming/blob/master/Chapter06/cloudant-flows.json

Important Note

This flow has no values for the Cloudant service name in cloudant in/out flows. Please check if your service name is set on that automatically once this flow definition has been imported.

You now understand how to handle data on Node-RED. We'll visualize that data in the next section.

Making a flow for use case 2 – visualizing data

The first use case was for storing sensor data in a database, while the second one was for visualizing sensor data on Node-RED. In IoT, after acquiring sensor data, we must visualize it in some form. The focus here is on retrieving and visualizing the data stored in use case 1. We will do this by following these steps:

1. Place an inject node, a function node, and a chart node from the palette on the left-hand side of the flow editor by dragging and dropping them into the workspace. Then, wire these nodes:
   

   Figure 6.39 – Placing the nodes and wiring them to show data

   The Inject node just executes this flow as a trigger, so there's no need to change the parameters in it. The function node generates numeric data to be shown on the Node-RED as a chart. Finally, the chart node makes it possible for the data to appear on the chart.

2. Code in the function node to generate numeric data that can be passed to the chart node.
3. Double-click the node to open the settings panel. Then, add the following code to the function node you placed:

   // Set min and max for random number

   var min = -10 ;

   var max = 10 ;

   // Generate random number and return it

   msg.payload = Math.floor( Math.random() * (max + 1 - min) ) + min ;

   return msg;

   This is what it looks like:

   

   Figure 6.40 – Code for generating a random number

4. After coding this script, click the Done button in the top-right corner to close the Settings panel.
5. Then, edit the settings for the chart node. When the Settings panel opens, click the pencil button to the right of the Group parameter. The dashboard group settings screen will open. You can use the default name if you wish, but we named it Packt Chart here.
6. After entering a name, click the Add button in the top right to return to the chart node's settings panel; make sure the Group parameter is Packt Chart. Now, click the Done button at the top right:
   

   Figure 6.41 – Setting a parameter on the chart node

7. That's it! Don't forget to click the Deploy button.
8. Click the left button on the inject node to run the flow. The data that is generated by the function node will be sent to the chart node when the button is clicked.

   You can check the result on the dashboard window.

9. Click the Dashboard button at the top right of the flow editor and click the Open window button. These two buttons are icons, so please refer to the following screenshot to see which buttons you must click:
   

   Figure 6.42 – Clicking the dashboard icon button and opening the window icon button

10. The line chart will be empty in the new window. Please click the switch of the inject node a few times. After that, you will see the line chart filled in with values:

Figure 6.43 – Line chart with values

Congratulations! With this, we have successfully created a basic flow (application) that shows sensor data as a chart with Node-RED.

You can also download this flow definition file here: https://github.com/PacktPublishing/-Practical-Node-RED-Programming/blob/master/Chapter06/dashboard-flows.json.

Summary

In this chapter, you learned how to create a server-side sample flow (application) by following a real IoT use case. These were simple tutorials, but I am sure it will be beneficial for you so that you understand how to make flows for IoT server-side applications.

The flow steps we created here will help you create different flows for other server-side applications in the future.

In the next chapter, we will use the same IoT use case we used in this chapter, but we will create a practical sample flow (application) that will call a web API.