© Christopher Harrold 2020
C. HarroldPractical Smart Device Design and Constructionhttps://doi.org/10.1007/978-1-4842-5614-5_10

10. Smart System Building Lab

Christopher Harrold1 
(1)
Denver, CO, USA
 

So now it is time for the big end-cap assembly project, the smart system! This will be a collector, sender, and receiver that will show you all the workings of a complete smart system, and how to assemble it. You will be working with technologies we have mentioned earlier in the book as well as the various hardware techniques for building this smart system prototype.

Before you begin the build, let’s look at the system you are going to build, and the parts that are involved in a little more detail. You are going to construct a system that consists of three main parts (and lots of little components):
  • The collector – This is the smart device itself really, which consists of the sensor(s) and the MCU to collect the data itself.

  • The network that allows you to send messages and data from point to point using a messaging software.

  • The central hub – This is just the name I am using, not a formal convention so you can call it the hub, or the central system, or whatever you like, but its function is to take the data coming in from the collector(s) and to translate it into something human readable. (We will use Node-RED for this.)

The software in this case is going to take on a much more important role. While I could just build a collector and use LEDs to alert me to changes in conditions, that is not nearly as useful in all applications as being able to collect and store historical data or displaying it in a more visually pleasing manner. Dashboards are the way that a lot of smart devices communicate with users, and they provide an effective visual representation of what’s going on, so they make sense in any application where you want to present data in a simple and meaningful way. Node-RED, in addition to being able to collect and process inputs from both hardware and software sources, also has a really nice and simple dashboard component which if you are not great at web sites (like me) provides a “click button, get graphs” way of creating dashboards.

We will also introduce the use of a messaging software for allowing data packages to be sent and received in a standard format. Mosquitto is a free software implementation of the popular MQTT protocol for messaging and allows you to do a lot of really cool things as you expand your smart device horizons. It is built on the concept of publishers (things that send messages) and subscribers (things that read messages). In the case of this smart system, the Collector is the publisher, and the Hub is the subscriber. The messages themselves can be further classified and categorized through the use of queues, called topics (which look like directories in the file system) and allow you to provide some levels of routing and separation of messages should you need it.

The messages in this case will be sent using the JSON format. JSON stands for JavaScript Object Notation, and what it is, is a series of key:value pairs that provides for sending data in very small packages. Unlike the olden days of having to use a big SQL query or other really “heavy” process, JSON objects can be very small, but contain a lot of information. The messages your collector will send in this exercise are on the order of a few kilobytes in size, but even a very large message would be very small. It is worth noting that there is a maximum message size for MQTT and Node-RED both (you will not experience it here in this lab). I tell you this because as you expand, you may want to send data from, say, a handful of sensors. That would require maybe a very long string of JSON keys and values. You could exceed the message size, but you can surmount this easily by breaking it up and sending more than one message, another very nice feature of the technologies we are using!

LET’S BUILD IT! SMART SYSTEM BUILD
This lab will require you to have both MCU options to complete it “as built” in these steps, but you absolutely can use your personal computer as a replacement for the Raspberry Pi in this lab, and it will function more or less the same. The purpose of using the Raspberry Pi for the hub in this smart system is twofold:
  • It is a very cheap option as a home automation hub, as it is capable of running multiple tools for simplifying your own automation projects

  • It removes the need for you to use your own computer/laptop as the hub. You can 100% do that, but that also means that if you shut off your laptop or leave the house with it, your system stops working.

Note You can run Mosquitto and Node-RED on all major OS platforms and the code provided will work for any implementation with essentially the same networking changes that are called out in the lab. Also, the instructions in the lab will generally work for any OS that you might be running, but on Windows you may have some minor changes to make – the reason I do not have those 100% documented is the Windows machine I used for the Arduino IDE is woefully underpowered, and I could not get Mosquitto and Node-Red to run reliably enough to test it all out. There are great install guides out there on the Internet and can be found in an easy search for either Node-RED or Mosquitto for Windows.

Depending on what you use for the Hub, Raspberry Pi or your own system, your finished smart system will look like one of the following in Figure 10-1.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig1_HTML.png
Figure 10-1

Architecture diagram for options 1 and 2

No matter what you choose, you will need to use a web browser to access Node-RED during the lab, which I recommend doing from your computer/laptop.

The first thing you are going to build is the Collector. You will be wiring up the sensors and lights on a breadboard as you can see in Figure 10-2.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig2_HTML.jpg
Figure 10-2

The complete smart system set of hardware

Note This lab will also include a LOT more software than you have used before now. I have provided you “ready to run” code mostly. The mostly here is that you will be using a network now, and because of that you will need to change some things in the code to match your environment. I have called them out and show you where they are, but if you have issues later on, it is highly likely that you have an error in one of those changes to a setting that is causing the network environment to be off.

Part 1 – The Collector Hardware

First, you are going to wire up a few sensors and lights to the collection unit. You will use the ESP32 NodeMCU for this part of the lab and the Raspberry Pi (or your own computer) for the central unit for data collection and display. For the collector you will be creating a monitoring device that uses three sensors:
  • Temperature/humidity sensor – DHT11 style sensor with a digital output (it has real digital data, not just on and off!)

  • Light sensor – Standard photo-resistant diode that will record the light level and works using a voltage divider circuit!

  • Soil moisture sensor – I used the Gikfun Capacitive style sensor for a lot of reasons, but the main one is they are just better. If you want to use this for any length of time, I recommend them over the cheap “two metal prongs” versions that come with a lot of starter kits. Those kinds degrade very quickly and are not as reliable, BUT they do work just fine with the appropriate wiring changes during the build. Also, this is a straight up analogue sensor, so you will be reading a raw voltage and making it make sense with software!

These three sensors are what make the collection unit “fit for purpose” and determine the type of smart system this will be. In this case, it is a way to monitor conditions for a plant to grow in and track that information over time. The latter part is done through the dashboard feature of Node-RED, which will be configured in a later part of the lab. For now, let’s focus on building our collector hardware, and wiring up the sensors to the NodeMCU.

BEFORE YOU BEGIN – If you have not prepped your computer, Raspberry Pi, and cloned the GitHub repository companion for the book, go back to the beginning of Chapter 9 and do so. It will make things so much easier for you later on, as the code is going to be a major factor in this build.

In order to build the collector, you will need the parts seen in Figure 10-3.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig3_HTML.jpg
Figure 10-3

Complete parts for the collector

  • NodeMCU

  • Breadboard

  • Three 68 Ohm Resistor Minimum – I have used a 100 Ohm Resistor here

  • Three LEDs – Any color is fine, but I used Red, Green, and Blue.

  • Three M-F wires – The color doesn’t matter; I do recommend long ones here if you have them

  • 10 MM Wires – Color is not an issue, but this is one place I will say that a variety of lengths will make things nicer for you as you build.

  • One photoresistor

  • One 10k Ohm resistor for the voltage divider!

  • A DHT11 temp/humidity sensor

  • Again, I used the capacitive soil sensor, but the other “metal legs” one will work too.

  • Small glass of water (or a plant) – Optional but makes for interesting readings of the sensor, without it you will get roughly the same reading all the time.

To assemble the hardware circuit, follow these steps:

  1. 1)

    Start by putting the Node MCU onto the breadboard (if it wasn’t there already from previous labs). You can see clearly in Figure 10-4 how it should be oriented if you did not already do so.

     
  1. 2)

    Next take the DHT11 sensor module, and place it up against the IC ravine like you did with the sound sensor in the previous lab and shown in Figure 10-5.

     
  1. 3)

    You are going to wire up your breadboard with a 3v and a 5v power rail – one on either side. You are going to choose which is which now, by putting the photoresistor with one leg in the “+” rail and one leg in the breadboard field anywhere that isn’t occupied as in Figure 10-6.

     
  1. 4)

    Now we will put the LEDs into the breadboard. Again, as before, these will be controlled by a GPIO pin so they do not share the power rail. Place them across multiple columns, and not in the same column as anything else, as in Figure 10-7. Note that in the image, my anode “leg” is to the right on all of them. This will help me remember where the wire from the GPIO plugs into.

     
  1. 5)

    Now we can also plug in the three 68 Ohm (I used 100 Ohm) resistors for each of the three LEDs. Place one leg into the breadboard with the cathode leg, and the other into the “–” rail of the breadboard on the 3v side as in Figure 10-8.Note I rotated the board 180 degrees for the image for clarity, so the change in orientation doesn’t throw you off!

     
  1. 6)

    Now you will plug in the 10k Ohm resistor for the voltage divider circuit on the photoresistor. This will go from the same column as the photoresistor to the “–” rail on the 5v side of the breadboard as in Figure 10-9 (again – rotated the breadboard back to its original orientation for clarity in the image).

     
  1. 7)

    Now you will connect the MF wires to the soil moisture sensor. If you did not use the same sensor I have, but the other type you will need to do this as well but the connection is slightly different. Just pay attention to the labels on the small PCB for the sensor and the rest should be the same. Plug the Female end of the three wires into the socket on the sensor body. (It comes with a wire, but this is a more convenient arrangement for the breadboard and doesn’t impact functionality at all). See Figure 10-10 for the end result.

     
  1. 8)

    Now you will make all the remaining wire connections to get the system ready to work. You will start with the power and ground rails. Plug in one end of a MM wire to the “–” rail on each side of the breadboard as in Figure 10-11. I chose the spot I did as it is relatively close to a GND pin on the MCU.

     
  1. 9)

    Next plug those wires into the GND pin that you have chosen. See Figure 10-12 for the end result.

     
  1. 10)

    Now you will connect the 5V “+” rail – we won’t need a 3V rail, as only one sensor (the soil moisture) uses that voltage. You can save a wire this way which in a complex prototype is sometimes a nice thing! As before, start with one end of the MM into the “+” rail on the 5V side as in Figure 10-13.

     
  1. 11)

    Then plug the other end into the 5v pin on the Node32 as in Figure 10-14. Notice that I chose this location for proximity to that pin.

     
  1. 12)

    Now you can plug in the DHT11 to the power and ground rails using MM wires. The DHT11 pinout can vary depending on your sensor, so check yours carefully. The one used in the lab is shown in Figure 10-15 and goes “Signal, Voltage, GND” from left to right as you look at the front.

     
  1. 13)

    Start with two MM wire, one in each column with the middle and rightmost pin on the DHT11 as in Figure 10-16.

     
  1. 14)

    Take the other end of the corresponding wire and plug it into the “–” and “+” rail as shown in Figure 10-17. Middle wire to the “+” and right pin to the “–” rail.

     
  1. 15)

    Now plug in a MM wire to the column that corresponds to the left pin of the DHT 21 as in Figure 10-18.

     
  1. 16)

    The other end will go to pin 17 on the ESP32. How do you know that? It is determined in the code as the “DHT” pin number (17). If you change this, you need to update the code, but if not, it will work as is as you can see in Figure 10-19. Things are starting to get complex with wires crossing over, but it will all come together!

     
  1. 17)

    You can also finish off the voltage divider circuit for the photoresistor by connecting a MM wire in the same column as the photoresistor and one of the legs of the 10k Ohm resistor as in Figure 10-20.

     
  1. 18)

    Then connect the other end into the “light” pin which is pin 36 – again, this is determined by the code so if you change this, you will need to change the code as well. Figure 10-21 shows it plugged in to the right pin on the breadboard.

     
  1. 19)

    Now you can finalize the LEDs for the indication of Power, Execution, and Network connectivity. (Green, Red, and Blue respectively. A MM Wire in the same column as the anode (long) leg of the LED to start as in Figure 10-22. Again, I’ve rotated the board for clarity in the image.

     
  1. 20)
    Now you will plug them in to the appropriate pin for the program, again, determined by the settings in the code – they are plugged in in Figure 10-23:
    • Green LED wire goes to pin 22

    • Red LED wire goes to pin 21

    • Blue LED wire goes to pin 16

     
  1. 21)

    Last part to connect is the moisture probe. You will need to connect the power, ground, and signal wire to the right pins. Start with power and ground by connecting the GND wire to the “–” rail on the 3v side of the breadboard, and the VCC wire directly to the 3v pin on the ESP32 as in Figure 10-24.

     
  1. 22)

    The last connection that completes the connector is to plug the signal wire from the probe into the pin as determined by the code. As with the other GPIO pins, if you change this you have to change the code. In this case pin 34 is the pin for the sensor to use and it will look like Figure 10-25 when connected.

     
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig4_HTML.png
Figure 10-4

MCU on Breadboard

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig5_HTML.jpg
Figure 10-5

DHT11 on the board

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig6_HTML.jpg
Figure 10-6

Photoresistor in its power rail and breadboard column

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig7_HTML.jpg
Figure 10-7

LEDs on the breadboard

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig8_HTML.jpg
Figure 10-8

Resistors for the LEDs plugged in

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig9_HTML.jpg
Figure 10-9

10K resistor on the breadboard for the voltage divider

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig10_HTML.jpg
Figure 10-10

Wires in the soil moisture sensor

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig11_HTML.png
Figure 10-11

Both ground rails with MM wires in

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig12_HTML.png
Figure 10-12

Ground wires connected to ground pins

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig13_HTML.jpg
Figure 10-13

MM wire in 5v power rail

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig14_HTML.jpg
Figure 10-14

5v pin connected to power rail

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig15_HTML.jpg
Figure 10-15

DHT11 pinout

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig16_HTML.jpg
Figure 10-16

DHT11 wires in place

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig17_HTML.jpg
Figure 10-17

Power and ground from DHT11 plugged in

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig18_HTML.jpg
Figure 10-18

Signal wire in the breadboard

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig19_HTML.jpg
Figure 10-19

DHT11 plugged into pin 17 on MCU

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig20_HTML.jpg
Figure 10-20

Voltage divider circuit wire in breadboard

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig21_HTML.jpg
Figure 10-21

Analogue signal wire from the voltage divider circuit to the MCU

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig22_HTML.jpg
Figure 10-22

MM wires for the LEDs in the breadboard

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig23_HTML.jpg
Figure 10-23

LED wires in the corresponding pins on the MCU

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig24_HTML.jpg
Figure 10-24

Moisture probe power and ground in place on board

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig25_HTML.jpg
Figure 10-25

Moisture probe sensor in place on the MCU

That’s it for the collector – it is now ready to run and collect readings. However, there is something missing from our smart system; someplace to send the data to! This collector is predicated on the central hub existing and without it, it will just keep pinging away at trying to connect forever. Now you will need to move on to make the required software changes to make the collector and hub talk and to finalize the software that supports the hardware you just created.

There are two parts to this change – the collector end of things, where your sensors are, and the hub end where your dashboard and display waits to collect the data and show you what it says. You will start by updating the settings on the collector and making sure it can connect to your wireless network, and then update the hub to ensure that it is ready and waiting to receive data and display it out. You will be working with some C++ code on the collector, and with Node-Red that I mentioned before for the hub and dashboard.

The collector code is written in C++ and is significantly more complex than anything you will have worked with to this point. I will dig a little deeper into the dissection of the key parts, because they are things you will trip over as you grow your knowledge, and they are important parts of building more complex devices. You will also need to install a couple of libraries that are not default in the Arduino IDE to support the networking, MQTT, and the DHT11 sensor, and I will walk you through that part first.

Part 2 – The Collector Code

This part will take place in the Arduino IDE, and when complete you will have done the following:
  • Installed the supporting libraries for the DHT11, Networking, and MQTT

  • Changed the network settings to match your network

A quick networking side note for you. Wireless networking equipment comes in many sizes, shapes, and flavors. The current standard is starting to be more commonly in the 5GHz spectrum, which is faster and more reliable. Unfortunately, none of the stuff you are using for these labs works with 5GHz; it uses the older 2.4 GHz standard. MOST wireless routers broadcast both frequencies, but if yours does not you are going to have a hard time working with any of these MCUs as they do not support the 5GHz standard.

You will want to browse to the smartdevicebook folder, and find the plant_monitor directory. Inside is the plant_monitor.ino file, which is the standard file extension for Arduino files. This file has the code in it that will drive the collector and is meant to be opened in the Arduino IDE. I made sure to really clearly comment the part you care about in the code for the collector, right at the top under the libraries that are required. You can see it here in Figure 10-26.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig26_HTML.png
Figure 10-26

The network settings section of the collector code

As you can see, you need to change the YOURXXXXX to match your home network setup in the file and save it BEFORE you upload it to the collector MCU. If you upload it without changing those settings, it will try to connect, literally, to the YOUR_WIFI_NETWORK_NAME wireless network, which, unless that is your network (that’s totally hilarious, btw) nothing will happen.

Once you have made the changes, it is a good opportunity to take a browse through the rest of the code too. It will not compile in its current state because we have not installed the requisite libraries into the IDE, but you can look at it and see what it is going to do. I have put in a lot of comments to guide you to what is happening but let me highlight a couple of the more interesting parts here for you as well.

Libraries, as I said, are the backbone of smart devices, allowing you to interact with sensors and devices in a snap. Figure 10-27 shows the libraries in this application from the top of the code. Time you know from previous builds, but the new ones here are for the DHT11 sensor, and the networking/MQTT elements. (WiFi.h and PubSubClient.h) I will walk you through installing these in the Arduino IDE momentarily, as the code will not compile without them!
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig27_HTML.png
Figure 10-27

The libraries required for the collector

Figure 10-28 is the time between readings, and the comment pretty much says it all, but it is worth pointing out. Every 10 seconds is excessive for checking on a plant, but I have left it like that for the lab because otherwise you wait for hours for a reading, and that’s not as exciting. The timer_value controls the time in between readings. For my outdoor garden I checked at 4-hour intervals (Denver’s climate is hot and dry in the summer and it was not abnormal for water levels to fluctuate by 50% or more during a day). Indoor plants an 8-hour reading is OK. Obviously, this is a “you choose” setting, so I only offer guidance for you based on my experience; there is no right or wrong here!
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig28_HTML.png
Figure 10-28

Time Between readings setting and comments

Just because this is new to you, the commands you see in Figure 10-29 use the libraries you imported to establish a new Wi-Fi connection and the connection to the MQTT server. You will use the “client” to make publishing requests later and control the connection to MQTT. “espClient” is the wireless connection and is how you will use the wireless network.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig29_HTML.png
Figure 10-29

The Wi-Fi and MQTT settings in the application

Analogue readings are finicky things sometimes, and this code bit in Figure 10-30 controls the accuracy of the analogue readings on the analogue inputs. It is not required, but always good to have, and these settings are the recommended for best results with the sensors in use.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig30_HTML.png
Figure 10-30

Analogue tuning on the ESP32 analogue inputs

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig31_HTML.png
Figure 10-31

MQTT Publishing command with JSON String Formatting

This is the actual MQTT publishing command (there are a few in the code, I have chosen this one toward the end of the code). This little blob here does three things:
  • Formats up a nice JSON string to send off to the hub using the C++ inbuilt function “sprintf” which lets you have a bit more control over strings. I find C++ is frankly unwieldly for working with strings of text, but sprintf takes a bit of the load off. The JSON is important, because that’s what Node-RED expects and we will look at that shortly as well.

  • Next it lets the console know it is going to publish (Serial.println again).

  • Then it calls the client.publish method to push the string present in the data0 array off to the MQTT server with the message topic of “garden1” (this could be anything really, and if you had, say, multiple gardens, you might want to change that “garden1” value). In our simple system, the MQTT broker on the other end doesn’t care about having different topics, so anything here will work.

Obviously, there is a heck of a lot more going on in there, but those are some of the more interesting bits. In order for this to run though, you will need the right libraries in the Arduino IDE. Installing them is pretty simple, just a search and button click effort really thanks to the nice library manager built into the IDE. You will be installing three libraries:
  • DHT11 Sensor library

  • ESP32 Wireless library

  • MQTT Client library

Follow these steps to get them installed so you can compile the code:

  1. 1)

    In the Arduino IDE, select the Tools ➤ Manage Libraries option as in Figure 10-32.

     
  1. 2)

    In the library manager, click into the “Search” field on the upper right as in figure and type in “DHT-11” as in Figure 10-33 and hit Enter.

     
  1. 3)

    The library we want is by the fine folks at Adafruit, and should be the second one in the list, as highlighted in Figure 10-34.

     
  1. 4)

    Click the install button next to the version box as in Figure 10-35. You will likely be prompted to also install the Adafruit unified sensor library, so click the “Install All” button on the prompt in Figure 10-36 and wait for the install to complete. (It should only take a few seconds.)

     
  1. 5)

    It will now say “Installed” next to the library as in Figure 10-37.

     
  1. 6)

    Now in the search field type “esp32 wifi” and hit Enter.

     
  2. 7)

    Scroll down the list and select the ConfigManager library from the list and install it, as you did with the other library by clicking the Install button, as shown in Figure 10-38.

     
  1. 8)

    Last, replace “esp32 wifi” in the search bar with “pub sub” as in Figure 10-39.

     
  1. 9)

    Scroll down and find the PubSubClient by Nick O’Leary in the list that you can see in Figure 10-40,

     
  2. 10)

    and again, install by clicking the Install button.

     
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig32_HTML.jpg
Figure 10-32

Manage libraries menu

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig33_HTML.jpg
Figure 10-33

DHT11 search

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig34_HTML.jpg
Figure 10-34

DHT11 Library to install

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig35_HTML.jpg
Figure 10-35

Install button to start installation

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig36_HTML.jpg
Figure 10-36

Install supporting libraries prompt

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig37_HTML.jpg
Figure 10-37

Library installed

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig38_HTML.jpg
Figure 10-38

Install this library for Wi-Fi

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig39_HTML.jpg
Figure 10-39

Search for “pub-sub” library

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig40_HTML.jpg
Figure 10-40

Install this library

Now you have the libraries you need to make the IDE compile and test the code for deployment onto your collector. You can validate this by closing the library manager and clicking the Checkmark in the upper left of the IDE. This is the “validate” button, and it will test the code and ensure there are no errors. You can see the button highlighted in Figure 10-41 and the result when all the code is right in Figure 10-42.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig41_HTML.jpg
Figure 10-41

The “Validate Code” button

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig42_HTML.jpg
Figure 10-42

Code validated success!

At this point you can flash the code to your ESP32 as you did with the other applications in the other builds, and it will run, but it will not see the result of this, because you have to get the hub ready to receive and display the information. If you are watching the serial monitor, you will see the application attempting to connect to the hub, and retrying this behaviour in a loop forever. This is expected behavior for the collector at this point, but I do not want you to be disappointed you’ve done something wrong. This is now part of a smart system, and in order for it to work, all the parts need to be up and running, and if one is not, then the system doesn’t work the way it’s meant to. I will walk you through getting the hub ready in the next part, so you can get everything up and running.

Part 3 – The Hub

Time for the most crash course ever in Node-RED on the Raspberry Pi! Node-RED is a phenomenally accessible language, built on top of Java and pretty much designed from the ground up to make working with smart devices WAY simpler. It uses a drag-and-drop model of programming that takes all the formatting and syntax and structure away and replaces it with “nodes” that get dropped onto a canvas and “wired” together to form a “flow.” The flow is essentially an application that starts and ends based on inputs.

As with the other code in the book I have taken the liberty of building out the Node-RED flow for you for the lab. If you did not follow the instructions in Chapter 8 around the configuration of the Raspberry Pi (or are using your local machine), you will need to install Node-RED and Mosquitto before you can proceed to the directions for configuring the Hub. You can run these on Mac, Windows, and Raspberry Pi, so if you do not have the Raspberry Pi for this and are using your own computer to act as the hub, I won’t spell out the instructions here, as there are lots and lots of resources online to guide you on the installation of both.

There is also the possibility that I could have used some hardware on the hub, the Pi does have a GPIO header, and Node-RED will interact with it to control the GPIO outputs which can turn on lights and read sensors and all of that. For the smart system, the strength of the Raspberry Pi lies in the fact that it is a computer and can act as the central hub for our smart system. The hardware access it has is a bonus then that you could explore to expand its capability beyond “just the hub.” Also, because I do not want to exclude anyone who does not have a Raspberry Pi to work with, hardware would have meant missing part of the function of the lab if you have to use your own computer as the hub.

So why does that matter? It only matters because in this exercise the hub is 100% software and from here on out, everything is done in a web browser on your own computer or laptop, as Node-RED is all driven out of a web-based UI. I will be taking screen shots from Chrome on a Mac, and connecting to a Raspberry Pi, but it will look identical on any platform and browser (minus the browser menus which you will not use). You do need the IP address of the Hub (your computer/laptop or the Raspberry Pi) for this, as that is how you will connect to both the UI and connect Node-RED to your MQTT server.

Now let’s get your Node-RED Flow ready to run, and you can finally see your smart system in action!

  1. 1)
    In order for this part to work, you need at a minimum the following:
    1. a.

      Your Hub (Raspberry Pi or your own Computer/Laptop) needs to be running and on your network.

       
    2. b.

      You need Mosquitto and Node-RED installed on your Raspberry Pi as we covered in the setup section earlier, OR on your own computer laptop if you do not have a Raspberry Pi.

       
     
  2. 2)

    Regardless whether you have used the Raspberry Pi, or your own Computer/Laptop, in your regular web browser on your laptop or computer, type in the IP address of your hub (Raspberry Pi or computer) and add :1880 to the end of the IP address (the “:” matters). This will open the Node-RED canvas to the default flow as you see in Figure 10-43.

     
  1. 3)
    This is the pure, default canvas for building Node-RED flows. The flow that you will import from the code repository requires some extra add-in nodes, which are like libraries for Python or C++. They are very easy to install, with these steps.
    1. a.

      First click on the “hamburger” menu on the far right of the UI (the three lines stacked horizontally) and you will see the menu as in Figure 10-44.

       
    2. b.

      From the menu click the “Manage palette” option as in Figure 10-45.

       
    3. c.

      The Palette Menu will pop up. Click on the “Install” tab at the top as in Figure 10-46.

       
    4. d.

      Type in “dash” and the search results will fill in. You are looking for the second one in the list as in Figure 10-47. (Highlight added for clarity – it will not be highlighted in the list.)

       
    5. e.

      Click the Install button to the right of the package name in the list as in Figure 10-48.

       
    6. f.

      A popup will appear as in Figure 10-49, and you may click the Install button on the lower right.

       
    7. g.

      The install will begin and the package will change to the progress bar seen in Figure 10-50.

       
    8. h.

      When it finishes, you will see the button changes to say “Installed” and is grayed out as in Figure 10-51.

       
    9. i.

      You need to install two other sets of nodes by repeating the same steps. First type “moment” into the search bar, and choose the highlighted package as in Figure 10-52 by clicking “Install” – follow the previous steps for any prompts.

       
    10. j.

      Then change the search to “message count” and install the package highlighted (it should be the only one in the list) as in Figure 10-53.

       
    11. k.

      You can now click “Close” in the upper right, and the needed packages will be ready for use.

       
     
  1. 4)

    Next step is to import the flow that was created for you and is in the repository for the book you cloned/downloaded from GitHub. In your web browser, click on the “hamburger menu” in the upper right of the UI, and select the “Import” menu as shown in Figure 10-54.

     
  1. 5)

    The “Import Nodes” dialog box will open as you see in Figure 10-55.

     
  1. 6)

    Click the “Import from File” button at the top of the box that you can see in Figure 10-56.

     
  1. 7)

    A directory browser will open, and you can browse to the file that you cloned/downloaded in the /smartdevicebook/plant_monitor directory. It is called plant_monitor.json and you can see it in Figure 10-57 (Mac screenshot).

     
  1. 8)

    Click the Open button when you have highlighted the file, and you will see that there is now some JSON code in the import screen as in Figure 10-58.

     
  1. 9)

    Click the “Import” button at the lower right of the dialog, and a confirmation will appear at the top of the screen as in Figure 10-59.

     
  1. 10)

    You will also notice a new tab has appeared behind the “Flow 1” tab in the canvas area as in Figure 10-60. This is the flow you will work with so you can click on that tab and it will move to the foreground.

     
  1. 11)

    The flow should look like the image in Figure 10-61 – if you have any errors about missing nodes, or nodes that are not right looking (dashed border line, or orange text), make sure you installed all three needed libraries from the preceding steps!

     
  1. 12)

    There are a couple of settings that you will need to change to make the flow work, but they are very easy. Start by double-clicking the node at the far upper left that is highlighted orange, called “All Sensors” as you see in Figure 10-62 – this is the first node of the flow, and it is an MQTT input node type. It reads the MQTT message queue and if a new message appears it will process it based on the rules in the flow.

     
  1. 13)
    The node configuration dialog will pop in from the right side of the UI as in Figure 10-63. This is how nodes are configured to do what you want them. The MQTT Node requires five settings:
    1. a.

      The network to connect to (called “Home Network” in this flow) – This is what you will change in a second.

       
    2. b.

      The message topic to “listen” for. In this case, by using the “#” topic you are saying “listen to everything”. If you decide to add more flows or functions, this can be changed to just listen to the messages that come from a specific sensor collector.

       
    3. c.

      QOS and Output are defaults but could be changed based on your needs – They are not usually required though.

       
    4. d.

      And lastly the Name – This is literally what appears in the flow on the node itself. In this case I named it “All Sensors”, but you can call it what makes sense for you if you want to change it.

       
     
  2. 14)

    Click on the little pencil icon next to the Server field where it says “Home Network” in Figure 10-64. This will open the server configuration node.

     
  1. 15)

    There is only one field you must change in this config, and that is the IP of the MQTT server. This should match the IP of your Raspberry Pi or computer depending on what you are using as the hub, as you can see in Figure 10-65 mine matches the IP I connected to for the Node-RED UI.

     
  1. 16)

    Once you’ve entered it in, click the “Update” button at the upper right to update that configuration as in Figure 10-66.

     
  1. 17)

    Then you can click “Done” in the node configuration screen as in Figure 10-67.

     
  1. 18)

    Back in the canvas, notice the small blue pip on the tops of all your nodes (same as in Figure 10-68)? That means that while the node is present, it is not actually running. Node-RED requires you to Deploy the flow before it executes (like flashing your ESP32 or executing the code on the Pi from before).

     
  1. 19)

    To deploy the flow, click on the big red “Deploy” button on the upper left, as in Figure 10-69.

     
  1. 20)

    You will see a confirmation pop in from the top of the browser, confirming the deployment as in Figure 10-70. The unused configuration nodes notice is an erratum from my prepping for the book and you can ignore it.

     
  1. 21)

    The Dashboard nodes that you installed earlier are what drives the UI for the sensor data. On the upper right of the UI, you will see a little button that looks like a bar graph and if you hover over it, it will say “Dashboard” as in Figure 10-71. Click the button to bring up the dashboard configuration options.

     
  1. 22)

    Another strange erratum of my prep work appears when you do this, showing what would appear to be three duplicate dashboards under the configuration, but when it opens there is only one. I tried to eliminate this, but since it isn’t broken, I left it. Just do not be surprised when you see Figure 10-72, it is not broken!

     
  1. 23)

    The dashboard UI is accessible from the tiny “go to” link on the upper right side of the configuration menus just below the bar graph button as you see in Figure 10-73. Clicking this will open the dashboard for your sensors.

     
  1. 24)

    The default dashboard shows up as in Figure 10-74.

     
  1. 25)

    Now, I personally do not like the “Light” color scheme as I find it hard to read. If you want to change it, click back on the tab in your browser for the Node-RED editor, and click the “Theme” tab in the dashboard config menu, seen in Figure 10-75.

     
  1. 26)

    Click on the drop down there at the top and choose “Dark” and then click “Deploy” again as in Figure 10-76 (remember, no change you make anywhere in the editor exists until you deploy it!).

     
  1. 27)

    Now if you click back, the dashboard will be in dark mode and a lot friendlier on the eyes as in Figure 10-77. Notice also, there is not data there. That is because your sensors are still sitting cold and dark on the workbench, waiting to be fired up!

     
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig43_HTML.jpg
Figure 10-43

The default Node-RED canvas

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig44_HTML.jpg
Figure 10-44

The hamburger menu in the Node-RED

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig45_HTML.jpg
Figure 10-45

Manage palette menu option

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig46_HTML.jpg
Figure 10-46

Install tab in Palette Manager

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig47_HTML.jpg
Figure 10-47

Dashboard library to install

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig48_HTML.jpg
Figure 10-48

Install button for installing libraries

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig49_HTML.jpg
Figure 10-49

Confirmation popup

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig50_HTML.jpg
Figure 10-50

Install progress

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig51_HTML.jpg
Figure 10-51

Install complete

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig52_HTML.jpg
Figure 10-52

Moment library to be installed – highlight added for clarity

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig53_HTML.jpg
Figure 10-53

Message count library to install – highlight added for clarity

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig54_HTML.jpg
Figure 10-54

Import menu

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig55_HTML.jpg
Figure 10-55

Import nodes dialog

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig56_HTML.jpg
Figure 10-56

Import from File button

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig57_HTML.jpg
Figure 10-57

Open this JSON file from the directory you cloned

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig58_HTML.jpg
Figure 10-58

JSON filled in ready to import

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig59_HTML.jpg
Figure 10-59

Confirmation of import

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig60_HTML.jpg
Figure 10-60

New tab in the UI

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig61_HTML.jpg
Figure 10-61

The imported flow

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig62_HTML.jpg
Figure 10-62

The “all sensors” node to be edited

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig63_HTML.jpg
Figure 10-63

The MQTT settings in the node configuration screen

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig64_HTML.jpg
Figure 10-64

Modify configuration button

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig65_HTML.jpg
Figure 10-65

MQTT server configuration – change IP field to match your MQTT server IP

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig66_HTML.jpg
Figure 10-66

Update button to confirm changes

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig67_HTML.jpg
Figure 10-67

Click Done to confirm settings

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig68_HTML.jpg
Figure 10-68

Undeployed updates pip on a node

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig69_HTML.jpg
Figure 10-69

To deploy your flow to active – click the Deploy button

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig70_HTML.jpg
Figure 10-70

Confirmation of deployment popup

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig71_HTML.jpg
Figure 10-71

The “Dashboard” button to open the dashboard config

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig72_HTML.jpg
Figure 10-72

Duplicate dashboard erratum

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig73_HTML.jpg
Figure 10-73

Click this “go to” link to open the dashboard in a new tab

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig74_HTML.jpg
Figure 10-74

The default dashboard

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig75_HTML.jpg
Figure 10-75

The theme configuration tab

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig76_HTML.jpg
Figure 10-76

Deploy the Dark!

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig77_HTML.jpg
Figure 10-77

The dark themed dashboard

With the hub up and ready, the last step is to turn on the collector and watch the data flow in!

Part 4 – The Whole System

Time for the grand finale! Depending on where you stopped with the collector, you probably have a contraption sitting next to you that looks like Figure 10-78, and has been happily turning LEDs on and off (if you left it turned on, if not you can plug it in now) and sending messages to the Mosquitto server on your hub. Up until now though, you have not had a way to see what that looks like, because the Node-RED piece had been missing. Now that the hub is configured, if you start up your collector, you will start seeing the data arrive on the dashboard. It will change from the empty one you started with to one that looks more like Figure 10-79.
../images/480481_1_En_10_Chapter/480481_1_En_10_Fig78_HTML.jpg
Figure 10-78

Your sensor pack looking ready to do something!

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig79_HTML.jpg
Figure 10-79

The dashboard in Node-RED – now with actual information!

For some fun and to make the readings more interesting, there are a few little things you can do to make the graphs move around.
  • Cover the light sensor with your hand (like I did here in Figure 10-80) which should make the light reading drop considerably. Make sure to really cover it well, as it is actually quite sensitive.

  • Blow on the DHT11 sensor (you can put an ice cube in your hand and do this across the ice cube and it will really drop measurably). The DHT11 is not supersensitive, but you should see a downward spike like in Figure 10-81.

  • My personal favorite, stick the moisture sensor in a glass of water (careful not to submerge the electronics!) or in a plant! Figure 10-82 shows the result when the water level bounces up.

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig80_HTML.jpg
Figure 10-80

Covering the light sensor makes it drop its reading

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig81_HTML.jpg
Figure 10-81

DHT11 temp drop thanks to some ice cold “wind”

../images/480481_1_En_10_Chapter/480481_1_En_10_Fig82_HTML.jpg
Figure 10-82

Moisture reading in a glass of water

Smart System Recap

Your first smart system fully functioning and collecting data, sharing it visually, and otherwise improving something. That is a fully functioning smart system and you built it and that is AWESOME!

Some things to point out at this moment of triumph. You have reached the point where you are taking default, uncalibrated readings from your sensors. EVERY mass market sensor has variance and frankly some of them are a lot worse than others. You will need to do some tuning to figure out the right ranges and variances for your sensors/environment/needs. Commonly I see people complain about sensors throwing seemingly “bad” data and I wonder how much of that is trusting it to be perfect out of the box. If you look at the reading from the soil moisture sensor in Figure 10-82,it says roughly 50% – meaning 50% water present, but it is LITERALLY IN A GLASS OF WATER! That is 100% water. How do I fix that? Well I need to take wet and dry readings, and then figure out what the real range the sensor can read is and do my math from that. For example, in my own garden, I had to divide my sensors’ readings by around 920 instead of 1024, because that was about the highest raw value reading they would register, even in a glass of water. This tuning of your devices is really critical to the performance of your system, and something all builders go through.

We have also come to the big turning point in the smart device builder’s path, where you have something functional, that works and does what you intended. The thing about this is you can expand it, add more sensors, more collectors, and so on and really scale it out! BUT, all those wires, and the breadboard are fragile. Seriously, go move it around a bunch. I will wait here. Did wires come loose? Kind of a pain, right? Prototypes like this are critical to making things work, and they are fun! They just aren’t permanent, and that’s where we go next. On to making this prototype something that will last and stand the test of time (or at least being moved across the room).

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.223.159.195