Testing the System

Before you start your system, you need to ensure that every device in your solution is working properly. Raspberry Pi is your master system and you need to boot it up first to see if it works properly. In order to do so, you need to plug the power cord into the power cord of the Raspberry Pi board. Once done, you need to connect the LED screen using the HDMI port. All this is done after you have installed the Raspbian OS on the SD card of the Raspberry Pi 3 Model B+. Figure 3-2 shows the Raspberry Pi booting up.

After it has finished booting up, the Raspberry Pi desktop will show up and will look similar to what you see in Figure 3-3.

Go to the Start button at the top of your Raspberry Pi desktop, click Programming, and then click the Thonny Python IDE. This is the IDE that you will be using for your programming. Please remember that this is not a very professional IDE but a small one to help you get started on IoT projects. It does not have advanced features that an IDE like PyCharm has, but it is good enough to write code and get it executed on Python IDLE.

Run the Python code in the Thonny IDE by pressing the Run button in the toolbar and you should have the output that appears in Figure 3-4.

Successful output means that most of the common libraries like pandas, matplotlib, numpy, and Seaborn are installed and that Python IDLE is configured to give you output. If you encounter any error in this simple Python program on the Raspberry Pi, it could be due to a typo or a library not being installed on your Raspbian-based Python. In this case, you could try installing it using the pip install <library_name> command. Replace <library_name> with the name of library that is giving you the error, such as pandas, Seaborn, matplotlib, etc.

You could run another simple test on pandas by creating a test dataframe such as one given in the following code, which imports the pandas dataframe and then creates an instance of the DataFrame() object and then prints out “Pandas loaded”. The result appears in the Thonny IDE in Figure 3-5.

import pandas as pd

df=pd.DataFrame()

print("Pandas loaded")

After running the pandas load test, you are assured that the pandas Python installation is fine. Now you need to test the machine learning library scikit-learn, which is the library you will be using for machine learning implementation in the solution exercises of this book. Run the following code to test scikit-learn loading on Raspberry Pi. The resulting output is shown in Figure 3-6.

import sklearn as sk

print("Scikit learn loaded")

If the scikit-learn library is loaded properly, you should see the output given in Figure 3-6 in the shell section at the bottom. If you get any error from the shell, it means the library is not loaded in your Raspberry Pi system. To load the library, use the command pip3 install scikit-learn. If you need to troubleshoot, refer to the Stackoverflow.com discussion on the topic at https://stackoverflow.com/questions/38865708/how-can-i-run-python-scikit-learn-on-raspberry-pi.

You now have Python and its libraries tested and running. If everything has worked fine so far, the next step is to test Arduino using Python, which you will do in the next section.

Arduino Hardware Setup and Communication

You first connect using the USB serial cable that comes with the Arduino Mega 2560 with Raspberry Pi 3 Model B+, as you can see in Figure 3-7. The USB port of the Raspberry Pi is used to do this and the other end is connected to the serial port of the Arduino board.

In the next step, open up the Arduino IDE as shown in Figure 3-8 and write the program shown in Listing 3-2.

When you write a sketch program for Arduino, there are two functions that are very similar to the C language that need to be present. The first one is the void setup() function, which is primarily used for initializing things like serial bus communications, LEDs, or other devices connected to the Arduino microcontroller. The Arduino IDE shown in Figure 3-12 is what you will open on Raspberry Pi 3 B+ from the Start ➤ Programming menu from the desktop and then you’ll write the program shown in Figure 3-9.

The code is self-explanatory and is in the C style and syntax. The first function is void setup() and the comment after the symbol //, which stands for comment, explains that you can put your code here and that you need to run it only once, like initializing any devices or variable values, etc. In the setup function, the body denoted by curly braces, { }, is the area where you write your code. The next function is the loop function, which you want to run in a loop or repeatedly. You may find this confusing if you have a pure software background and may not understand why you need a function that does things repeatedly. Try to understand it this way: a machine or a microcontroller like Arduino cannot function on its own unless there is a program that tells it what to do again and again. It is like having a servant at your disposal who gets bored of staying idle. Similarly, Arduino also needs to do something repeatedly and cannot stay idle. You can ask it to monitor a device and trigger something when a certain event happens or the device is about to malfunction. This is just an example of what you can do with this loop() function ; the applications are limitless. You could turn an LED on or off based on a Modbus device’s input value such as temperature etc. This just requires that the Arduino microcontroller works continuously without stopping, and that is what this loop() function is all about.

Before you can continue, you need to know the process of compiling and running the program in the Arduino IDE. First, you write the sketch code and save it in a file with the extension ∗.ino. Once saved, you go to Tools ➤ Board and hover to open up the list of supported Arduino microcontroller boards. Select Arduino Mega 2560 from the list. You have to remember to check every time you open up the Arduino IDE so that you don’t get errors in your compilation. Figure 3-10 shows how to do this.

After choosing Arduino/Genuino Mega or Mega 2560, you are good to write your first Hello world program, as shown in Figure 3-11.

You will notice that the setup function of the program initializes the serial port with a baud rate of 9600. The loop function contains a print statement to the serial port of the Arduino. In practical terms, this type of program does not mean anything; however, you are trying to test your communication with the Arduino microcontroller. In the next step, if you are able to successfully compile the program, it means you have not made any errors. The common mistakes in this Hello world program are the capitalization of first word (in serial, the letter S has to be capitalized). Another thing to remember is the functions of object serial do not to be capitalized. The begin and println functions are not capitalized. Just like any C or C ++, program every sentence has to end with a semicolon.

Now let’s move to the next step of compiling the program in the Arduino IDE, which is shown in Figure 3-12.

Running the Sketch

After you have successfully compiled the Hello world program, you may proceed to the next step of uploading it. In this step, the Arduino IDE communicates to the Arduino Mega 2560 microcontroller through Raspberry Pi’s USB serial interface and flashes the entire program to it. Now this program will reside in the microcontroller's memory and run infinitely until you write your next program and write it on top via the same process. Figure 3-13 shows the Upload option in the Sketch menu for uploading the program onto the microcontroller board.

Now that you have compiled the program and clicked upload, you should see the program being uploaded and a progress bar which will indicate the progress of the Arduino IDE writing the program to the microcontroller board. If you have not chosen the right microcontroller board, you will see an error here. You can also see an error in this step if there is a problem with the connection between the Raspberry Pi and Arduino. The most common reason for an error at the upload stage is a loose cable or a bad serial bus cable. So check the ends on both Raspberry Pi and Arduino boards to verify your cable does not have any issue. Try changing it if you still face an error when uploading. Figure 3-14 shows the result of a successful upload of your Hello world program onto the Arduino microcontroller board. Notice the “Done Compiling” message at the bottom panel of the Arduino IDE. The box below shows messages related to the program storage space or the maximum space available for it to use. In this case, it has used 1820 bytes of the available maximum space of 253952 bytes. Similarly, it also informs you that it uses for its global variables 212 bytes (2%) of dynamic memory, leaving 7980 bytes for local variables out of a maximum of 8192 bytes. Very long programs may not fit into the small memory of the Arduino Mega 2560 board, so this information comes in handy so you can optimize the program to fit into it.

This is the last step of the process to test the Raspberry Pi to Arduino communication. Now you are ready to undertake the next step in getting IoT sensor data from the Raspberry Pi.

Connecting It All Together

Let’s now connect an IoT sensor on the Raspberry Pi 3 Model B+ board and write a Python program to get data from it. You will learn how to store the IoT sensor data in a database in the next section of this chapter.

The circuit diagram of the IoT sensor with Raspberry Pi needs to be defined first so that you can understand what you are trying to build. Figure 3-17 shows the electrical connections diagram for the project.

The IoT sensor that you are going to use is an LDR, or light-dependent resistor module. You’re not using an LDR or a photoresistor but a pre-made LDR module that has a completely functional circuit board made on its PCB with the far end hosting the photoelectric resistor marked as number 6 in Figure 3-17. This module senses light from the ambient environment and gives back data in the 0/1 format. The values it gives are in a float, and it gives back 0 for daylight and a value closer to 1 (like 0.90 to 0.99) when it is dark. To simulate darkness, you can use an object to cover the light photo resistor of the LDR. You can see the point marked as 1, which is the VCC; this is where you have to connect the wire to the GPIO pin 1 on the Raspberry Pi board. The point marked is the ground connection and must be connected to the GPIO pin 6 on the Raspberry Pi board. Point number 3 is the DO, or digital output signal wire, from the LDR module and it must be connected to GPIO pin 25 on the Raspberry Pi board. The LDR module, when fully connected, gives out two red LED signals on both sides, which you can see in Figure 3-18.

Now let’s look at the LDR module with the photoelectric resistor. At the far end, it’s covered; as a result, only one light glows, showing there is darkness around the LDR module (see Figure 3-19).

The point marked 1 in Figure 3-19 is the second LED for digital output, which is off and shows there is no signal being received from the photoelectric resistor to it. The point marked 2 in the diagram is the photoelectric resistor, which is covered by me using a pencil. Anything can be used to cover the PE resistor as long as no light goes through it. Simulation of day and night is for testing purposes; however, in the real world, this sensor works wonders when you place it in an application such as detecting day or night and switching on or off an consumer/industrial appliance based on the result. This data, combined with data from an industrial machine, also helps diagnose critical problems in applications of predictive maintenance, which you will apply in the case studies. You will be storing this LDR IoT sensor data in a database in the next section of this chapter.

The GPIO pin number 1 has to be connected to the LDR module pin marked on its PCD as VCC. Similarly, the GPIO Pin for Ground should be connected through the wire to the middle pin showing GND on the LDR module. The third pin you are using in your program is the GPIO signal pin, which will send and receive the digital signal to and from the LDR module, which is pin number 25, to the third pin of the GPIO signal pin. There are some LDR modules that have four pins; that last pin is for analog input and output in case you want to use it. But in your case, since you have a three-pin LDR module, you don’t have to worry about it. The connected LDR pins on the GPIO and the LDR module can be seen in Figure 3-20.

Programming the IoT Sensor LDR Module

The output is a trial run of the LDR Python program and it works by giving out the floating point values closer to 1, which means it is dark when this program is being run or there is darkness around the LDR sensor module (it may be covered by an object). If you were to light a torch or bulb over the photoelectric resistor of the LDR module, it would start giving a value of 0, indicating there is light around it. You can test this and improve the sensitivity by adjusting the blue square-shaped potentiometer on top of the LDR module PCB. You have come to the end of this section because you are able to achieve getting data from IoT sensors.

Please remember there are many more sensors with many uses and which can be used for simple to complex consumer and industrial applications. The data from any of these sensors would be very similar. The digital output is always on the extreme ends of 0 and 1; however, analog output can be varied and give you in-between readings, which are not in the extremities of 0 and 1. Also, one point to remember is that the accuracy of these IoT sensors deteriorates over time when they are used heavily so you may need to replace sensors that come in contact with water or soil, leading to corrosion of their sensor heads.

Configuring a SQLite3 Database

In Figure 3-21, you can see the SQLite3 database named iotsensor.db. Since this is a vanilla database, the SQLite3 database engine automatically creates it for you. This new database will be empty and will not have any tables or other structures. You will need to create them.

Once you are logged into the database, it greets you with its version number and the date and timestamp when you have logged in. It displays a prompt sqlite>, which is where you run your commands to work with the databases. There are two sets of commands that work in this prompt: one is the set of commands that start with a dot, such as .databases or .tables, and the other is the set of SQL commands such as select ∗ from <tablename>;. A common beginner mistake is to forget to put a dot before the databases or tables commands and then the SQLite prompt throws out an error. For the SQL commands, do not forget to use a semicolon (;) at the end of each statement; if you don’t, the sqlite> prompt will continue to the next line and your command will not execute.

You can now look at the database and its information by typing the .databases and .dbinfo commands at the sqlite> prompt. The code for the commands and the output are shown in Listing 3-6 and Figure 3-22, respectively.

SQLite version 3.16.2 2017-01-06 16:32:41

Enter ".help" for usage hints.

sqlite> .databases

sqlite> .dbinfo

sqlite> .tables

sqlite>

Listing 3-6

Code for Database Information for SQLite3

Creating the Database Structure

You can see the result of this table creation in Figure 3-23, which shows the output of the .tables command and that the table ldrvalues now exists.

Inserting Data into the Database

Notice the modifications to the code from Listing 3-2. You added import statements to the SQLite3 Python library; this is necessary to communicate with the SQLite3 database. You also imported the datetime library to get date and time so that you can enter it in the database when inserting the query. The while(true): loop didn’t change; you just added the database insertion code at the end of it. The conn object is used to connect to the iotsensor.db database, where you created a table named ldrvalues; refer to Figure 3-23 for this. The cursor object inside the while loop initializes the cursor with the connection to the iotsensor database. The next statement is an INSERT INTO... statement which enters data into all the three columns: date, time, and ldrvalue. The value entered into time column is taken from the datetime.now() function, which returns the date and time together. Since you want the time and date separately, you first use datetime.date() and then in the second column, you use the datetime.time() function to get only the time value out of it. You separate the date and time instead of creating a single column because when you are doing EDA it becomes convenient if your date and time are in separate columns; it’s easier to find trends with date and time. Although you can do the same operation for taking out date and time from the column during EDA, you save yourself some cumbersome functions at the time of querying it. In a practical world, however, you may rarely find these two values of date and time separate and in the same column known as a timestamp. After the query is done, you use the curr.execute(query) to get the insert into the statement executed by the SQLite database engine. This is the stage when you are likely to get an error if you have a syntax problem in your query.

Checking the Data for Sanity

Based on the error messages that you get, you should try to resolve it. I ran the program for a few minutes and it inserted the LDR module IoT sensor data into the SQLite3 iotsensor.db database. You can see the result in Figure 3-24.

You can see that the program has inserted about 1316 rows in a few minutes. The structure of the table can be seen in the execution of the pragma table_info(ldrvalues); statement . The pragma statement has a function named table_info and takes the argument of the table name of the connected database. In your case, the connected database is iotsensor.db and the table is ldrvalues. It has three columns, which you can see as an output of the pragma statement. The first one is date, the second is time, and the third is ldrvalue. Their respective datatypes in the SQLite3 table are given beside each of them. After this, you run a query of select count(∗) from ldrvalues to get the count of total rows. Next, you look at the count of all the values in ldrvalue column which are not equal to 0.0, which stands for day. Any value in this column shows darkness or night. You should understand through the demonstration of this simple program how the IoT sensor data can be stored in databases and used.

Next, I am going to show you a full-fledged Python program that uses the internal Raspberry Pi CPU to measure its temperature and store it in a database. The program is going to simulate that it is measuring data from an industrial device or an IoT sensor and storing it in a database. After this, through a separate program, it is going to create a graph to display the data as you would do in a real-world IoT application. In the next section, you will apply and build a machine learning model. It is really going to be exciting when you get this data from the internal Raspberry Pi SBC board and then store it in a database for use in a machine learning model later. Let’s get started.

Although what you see is a large chunk of code, the real purpose is to show you how a practical Python-based IoT application is built. The only simulation you are doing is instead of getting data from an actual IoT sensor, you are taking it from the internal hardware IoT sensors embedded in the Raspberry Pi. The concept of temperature and the percent values are applicable to industrial devices such as heat exchangers or boilers, which have a mix of values in temperature and percentages and some absolute values as well. The electrical devices have various values ranging from frequencies to voltages. All this makes up the data you will use in the case study examples because there the data is going to come from the actual IoT sensors connected to the Raspberry Pi and Arduino.

The code first initializes and imports the required Python libraries like CPUTemperature from the gpiozero library. This is the one that is going to get you the internal Raspberry Pi temperature. You also import libraries like datetime to get the time and date of when the data is generated, the pandas dataframe to store the data temporarily, and an imported LED to give a status of green, orange, or red through the lighting of the appropriate one. For example, if the CPU temperature is less than 55 degrees Celsius, the green LED will light up; if the CPU temperature of the Raspberry Pi is greater than 55 but less than 60 degrees Celsius, the orange LED will light up; and if the temperature is greater than 60 degrees, the red LED will light up. This is exactly how you would implement an advance warning system using IoT sensors if you were measuring outside temperature values. After this, you import pygame to ensure that there is a sound coming out of the Raspberry Pi speaker when the temperature reaches beyond 60 degrees Celsius. This is a critical system for your SBC and if something is not done, the system will hang after a while and the board or its components can also burn. Try to imagine this in an IoT-based environment where you would like to implement such a critical alert system. The library pygame is initialized using the init() function and in the next line, it loads example.mp3, a shrill alerting sound, into memory. It does not play it since you have not given the command yet; it is just loaded into the memory.

In the next section of the code, you initialize the LEDs that will light up according to the CPU temperature. The red LED is at GPIO pin number 18, the green LED is at GPIO pin number 22, and the yellow LED is at GPIO number 17. Make sure the LEDs on the Raspberry Pi have been connected to the right pin numbers given in Figure 3-15.

The advance warning system now comprises a sound for the red temperature and a visual alert using LEDs for the CPU temperature. In a real-world application, you will need these alerting systems plus others like SMS or e-mail alerts for which you may need to configure mail and SMS gateway servers on your Raspberry Pi. We will not be doing this as we are undertaking PoC-level code and this is beyond the scope of the book.

Now you need to initialize the LDR, or light sensor module, at pin number 25, which you did in an earlier section of this chapter. After this, you initialize the pandas dataframe to store data in memory. The reason you are using pandas is that it gives a structure to the data and allows to you manipulate and analyze it as per need in Python. Next is the infinite while loop in which first you get the CPU temperature using cpu.temperature and store it in a variable. The next step is to check the value of the temperature. If it is greater than 60, an alert is displayed on the screen saying “RED ALERT CPU EXCEEDING HIGH TEMPERATURE” followed by the red LED being turned on in the program using the redled.on() function. When this event happens, the other LEDs (green and yellow) should be turned off so you use the off() function for these LED objects so that there is no confusion as to the status of the CPU temperature. You do not want all of the LEDs to glow; only the red LED should glow at this time to show the critical status. The pygame is used to play an alerting sound from the Raspberry Pi speaker using the pygame.mixer.music.play() function . Similarly, there are two other conditions. The next is when the temperature value of the CPU lies between 55 and 60 degrees; the status is displayed on the screen as “YELLOW ALERT CPU NEARING HIGH-TEMPERATURE THRESHOLD” and the yellow LED is turned on using the on() function and the red and green LEDs are turned off using their respective off() functions. There is no sound played because this is not a critical situation; the sound plays only when the temperature reaches a critical limit of 60 degrees. You may wonder how I came up with this number of 60 degrees Celsius. I simply referred to the Raspberry Pi manual, which came with it, and it mentions a range of 30 to 60 degrees Celsius. The next is the green status where the CPU is safe and does not need any alerting, but for someone watching the screen, a message of “TEMPERATURE IS NORMAL” is displayed. The green LED is turned on using its on() function and the yellow and red LEDs are turned off using their respective off() functions. Once you have this status of the CPU system in the variable tempstatus, you need to store it in a pandas dataframe and this is done by storing the date and time using the datetime now() function in the df['date'] and df['time’] columns. df['temperature'] stores the temperature from the tem variable. The df['tempstatus'] column stores the value that you get after going through the if condition for the tempstatus variable of “GREEN,” “RED,” or “YELLOW.”

Now you need to prepare to write the data into the SQLite3 database, which is done in the import statement import SQLite3. Then you initialize the connection object to the machinemon.db database. A curr cursor object is created after this to help parse the table if needed. Before you can write to the dataframe, it has empty values for some of the columns like cpu_percent, memory_percent, disk_percent, and LDR value. The cpu_percent variable is used for storing the CPU percentage value, the memory_percent variable is used for storing the memory percentage value, disk_percent is used for storing the disk percentage value, and the ldrval variable is used for storing the LDR module value of day or night.

This database structure needs to be created before you can start the execution of machinemon.py because if the back-end database structure does not exist, it will throw an error and the program will fail. Here you first create a new database connection to the database named machinemon.db and connect to it with the statement SQLite3 machinemon.db. Once this is done, in the code you look at basic data by running the dot commands .databases , .tables, and .schema, which tell you about the database’s path, the tables that exist within the database, and the schema of the tables or the structure of the tables. In your case, the table is machinedata, which comprises of the following columns: date, time, temperature, tempstatus, cpupercent, diskpercent, memorypercent, outage, and ldrvalue. The uses for these columns were shown in the code execution in Listings 3-9 and 3-10.

Creating the IoT GUI-Based Monitoring Agent

This agent runs on Python 3.6 as the library used is the tkinter GUI library. To run it, you simply type on the Raspbian command line python3 mmagent.py. Please note the use of python3 and not just python. Just typing python will invoke the Python 2.7 compilers whereas typing python3 will invoke the Python 3.x compiler. To make the code in Figure 3-38 compatible with Python 2.7, you may first have to install the Tkinter GUI python library and then change the code to import Tkinter instead of the small case tkinter. The difference between the uppercase and lowercase can make it confusing for people who do not understand the significance. A StackOverflow discussion on this topic may help you understand this better; go to https://raspberrypi.stackexchange.com/questions/53899/tkinter-on-raspberry-pi-raspbian-jessie-python-3-4.

The code in Listing 3-12 creates a window of the instance of the tk.TK() win object. It gives it a title of LED Monitoring Agent Application and then in the next line sets the font for the screen to Helvetica with a size of 30 and a weight parameter of bold. After this, you initialize the three LEDs at GPIO pins on the Raspberry Pi board at pin numbers 18-Red, 22-Yellow, and 17-Green. Next, you create a function to check the LED status because this needs to be done repeatedly each time the agent calls it. The function checks the LED status value through its value property and checks if it is equal to 1. If the value of any of the LEDs is equal to 1, this means the LED is on. It prints the LED status on the screen. After this function, you have another function to help exit the program gracefully. This function has a single line which uses the win.quit() function to exit the application window, bringing down the application monitoring GUI with it. In the general section after this exitprog() function, you instantiate an ledstatus button using the tk.Button() method, which takes in arguments for command=ledstatus. The ledstatus is the name of the button function that is required to be executed when the status button is pressed. By default, the color of the button is green. Then it is set in the grid, telling it using the grid() function for the row where it has to be placed; here you use 0 because you want it to be placed at top of the screen. The sticky argument takes the argument tk.NSEW, which stands for North, South, East, and West. In a similar manner, you now fix the exit button, which allows the user to exit gracefully from the program using win.quit() in the exitprog() function. The last statement of this program is the tk.mainloop(), which is required to make the window stay on the screen until the application work is done. The resultant screen window on the Raspberry Pi after running the command from the command line $python mmagent.py is shown in Figure 3-25.

The entry in the crontab above runs the command python3 /home/pi/iot/mmagent.py each day at 10 a.m. and the LED monitoring agent checks the LED status and outputs the values on the screen and sends an alert through email if the status at that time is RED. You can modify the crontab entry to check every few minutes.

In the last section of this chapter, you’ll apply a machine learning model on the simulated sensor data that you gathered in your SQLite3 database.