Chapter 1. Setting Up Your Environment

A path lies ahead,

Brambles and thorns, then it clears.

Almost there. Patience.

You’ve likely gathered by now that the Internet of Things can be vast, unwieldy, and very difficult to tame. To plan a way forward, we’ll first want to identify a problem area to tackle and then create an architecture from which to design and build our IoT solution.

Let’s start with a few key questions to establish a baseline: What problem are you trying to solve? Where does it start and end? Why does it require an IoT ecosystem? How will all these pieces work together to solve this problem? What outcome can you expect if everything works as designed? We’ll explore each of these in detail, and along the way construct an end-to-end, integrated IoT solution that meets our needs.

What you’ll learn in this chapter

To help you really understand how an IoT system can and should be constructed, I’ll dig into some basic architectural concepts based on the questions above, and use this as the basis for each programming activity. From there, you’ll build a solution that addresses the problem layer by layer, adding more functionality as you work through each subsequent chapter.

It goes without saying, of course, that the right development tools will likely save you time and frustration, not to mention help you with testing, validation, and deployment. There are many excellent open source and commercial development tools and frameworks available to support you.

If you’ve been a developer for any length of time, I expect you have your own specific development environment preferences that best suit your programming style and approach. I certainly have mine, and while the examples I present will be based on my preferred set of tools, my goal in this chapter is not to specify those you must use, but to help you ramp up on IoT development in a way that enables you to move out quickly and eventually choose your own for future development projects.

The concepts I present will be what matter most – the programming languages, tools (and their respective versions), and methods can be changed. These concepts represent some of the fundamentals of consistent software development: system design, coding, and testing. Let’s dig into each and get moving towards building your integrated IoT system.

Designing Your System

Creating a problem statement is probably the most important part of this puzzle. Let’s try it by drafting something reasonably straight-forward, but sufficient to encompass a variety of interesting IoT challenges:

I want to understand the environment in my home and how it changes over time, and make adjustments to enhance comfort while saving money.

Seems simple enough, but this is a very broad goal. We can narrow it down by defining the key actions and objects in our problem statement. Our goal is to isolate the ‘what’, ‘why’, and ‘how’. Let’s first look at the ‘what’ and the ‘why’, and identify any action(s) that the design should consider as part of this process.

Breaking Down The Problem

The exercises in this book will focus on building an IoT solution that can help you understand your home environment and respond appropriately. The assumption is that you’ll want to know what’s going on within your house (within reason), and take some sort of action if it’s warranted (that is, if the temperature is too hot, turn on the air conditioning). Simple, right?

This part of your design approach requires three steps:

  1. Collect Data

    Let’s define this in terms of what can be sensed, like temperature, humidity, etc. This is centered on the capture and transmission of telemetry (measurement data). The action, or rather, action category, will simply be named data collection

    • Temperature

    • Relative Humidity

    • Barometric Pressure

    • System Performance (utilization metrics for CPU, memory, storage)

  2. Determine Relevant Changes

    To decide which data is relevant, and whether or not a change in value is important, we need to not only collect data, but to store, and trend, time-series data on the items we can sense (like temperature, humidity, etc, as indicated above). Let’s refer to this action category as data management.

  3. Take Action

    We’ll establish some basic rules to determine if we’ve crossed any important thresholds, which simply means we’ll send a signal to something if a threshold is crossed that requires some type of action (e.g. turn up / down a thermostat). We’ll refer to this action category as system triggers.

Defining Relevant Outcomes

Now that we know what steps we need to take, let’s explore the ‘why’ portion of our problem statement. We can summarize this using the following two points:

  • Increase Comfort

    Ideally, we’d like to maintain a consistent temperature and humidity in our living environment. Things get a bit more complicated when we consider the number of rooms, how they’re used, etc. We’ll refer to this action category as configuration management, and it goes hand-in-hand with both data management and system triggers.

  • Save Money

    This gets a bit tricky. The most obvious way to save money is to not spend it! Since we’ll likely need to allocate financial resources to heat, cool, or humidify a given area, we want to optimize - not too much (wasteful), and not too little (we could end up with frozen water pipes in the winter). Since we might have some complexity to deal with here - including utility costs, seasonal changes, etc., as well as anything related to configuration management - we’ll probably need some more advanced analytics to handle these concerns. We’ll call this action category analytics.

You’ve likely noticed that each of the steps in the ‘what’ and ‘why’ sections above have an action category name that will help with the solution design once we move onto the ‘how’. As a reminder, these categories are: data collection, data management, system triggers, configuration management, and analytics. We’ll dig further into each of these as part of our implementation approach.

Although the problem statement seems rather simple on the surface, it turns out that the things you’ll need to do to address the problem are actually quite common within many IoT systems. There’s a need to collect data at its source, store and analyze that data, and take action if some indicator suggests that would be beneficial. Once you define your IoT architecture and start building the components that implement it - even though it will be specific to this problem - you’ll see how it can be applied to many other problem areas.

Let’s take a quick look at a simple data flow that represents this decision process in Figure 1-1. You’ll notice each action category is highlighted in the data flow diagram depicted in Figure 1-1:

Simple IoT data flow
Figure 1-1. Simple IoT data flow

Most IoT systems will require at least some of the five action categories I’ve called out: data collection, data management, analytics, configuration management, and system triggers. This means we can define an architecture that maps these into a systems diagram, and then start creating software components that implement part of the system.

This is where the fun starts for us engineers, so let’s get going with an architecture definition that can support our problem statement (and will, in fact, be reusable for others).

Architecting a Solution

Organization, structure, and clarity are hallmarks of a good architecture, but too much can make for a rigid system that doesn’t scale well for future needs. And if we try to establish an architecture that will meet all our plausible needs, we’ll never finish (or perhaps even never get started)! It’s about balance, so let’s define the architecture with future flexibility in mind, but also keep things relatively well bounded. This will allow you to focus on getting to a solution quickly, while still permitting updates in the future. But first, there are a few key terms that need to be defined to help establish a baseline architectural construct to build your solution upon.

As you may recall from Figure 0-1 in the Preface, IoT systems are generally designed with at least two (sometimes three or more) architectural tiers in mind. This allows for the separation of functionality both physically and logically, which permits for flexible deployment schemes. All of this is to simply say that the cloud services running within the cloud tier can - technically speaking - be anywhere in the world, while the devices running within the edge tier must be in the same location as the physical systems that are to be measured. Just as Figure 0-1 implies, an example of this tiering may include a constrained device with sensors or actuators talking to a gateway device, which in turn talks to a cloud-based service, and vice-versa.

Since we need a place for these five categories of functionality to be implemented, it’s important to identify their location within the architecture so we can have some things running close to where the action is, and others running in the cloud where you and I can access (and even tweak) the functionality easily. Recalling the edge tier and cloud tier architecture from the Preface, let’s see how to map each of the action categories from the ‘what’ and ‘why’ into each:

  • Edge Tier (constrained devices and gateway devices): Data collection, data management, device triggers, configuration management, and analytics.

  • Cloud Tier (cloud services): Data management, configuration management and system analytics.

Why do the Edge Tier and Cloud Tier include similar functionality? First, let’s take a look at how our flow chart fits within a tiered architecture:

  Notional IoT data flows between the Edge and Cloud Tiers
Figure 1-2. - Notional IoT data flows between the Edge and Cloud Tiers

Again, notice that we have some shared responsibility where some of the action categories are implemented within both tiers. Normally, duplication of effort is a bad thing - but in this case, it can be an advantage! Simple analytics can be used to trigger a device based on some simple settings if warranted - for example, if the temperature in your home exceeds 80o F, you’ll probably want to trigger the HVAC straight away and start cooling things down to, say, 72o F. There’s no need to depend on a remote cloud-based service in the Cloud Tier to do this, although it would be useful to notify the Cloud Tier that this is happening, and perhaps store some historical data for later analysis.

Our architecture is starting to take shape. Now we just need a way to map it to a systems diagram so we can interact with the physical world (using sensors and actuators). It would also be good to structure things within our Edge Tier to avoid exposing components to the Internet unnecessarily. This can be managed using a constrained device with a specialized software application that can either run directly on the device, or on a laptop or other generic computing system with simulation logic that can emulate sensor and actuator behavior.

Since you’ll want to access the Internet eventually, your design should include a gateway to handle this and other needs. This functionality can be implemented as part of a software application that runs on a gateway device (or, again, a laptop or other generic computing system). Your gateway device and constrained device will comprise the ‘edge’ of your IoT design, which I’ll refer to as the edge tier of your architecture going forward.

We’ll also want to deploy analytics services, storage capabilities, and event managers in a way that’s secure, but accessible from our gateway device and also by human beings. There are many ways to do this, but we’ll focus on one (or more) cloud services for this functionality.

Figure 1-3 provides a new view that will provide further insight into what you’re going to build and how you can begin incorporating the five action categories I’ve already mentioned. It represents, in grey boxes, cloud services within the Cloud Tier, and two applications within the Edge Tier - CDA and GDA - that will be implemented to contain the functionality of your constrained device and gateway device, respectively.

  Notional IoT simplified logical architecture with Edge and Cloud Tiers
Figure 1-3. - Notional IoT simplified logical architecture with Edge and Cloud Tiers

Let’s dig into each a bit further:

  • Constrained Device App (CDA)

    You’ll build this software application to run within the constrained device (emulated or actual), and it will provide data collection and system triggers functionality. It will handle the interface between the device’s sensors (which read data from the environment) and actuators (which trigger actions, such as turning the thermostat on or off). It will also play a role in taking action when an actuation is needed. Eventually, it will be connected to a communications library to send messages to, and receive messages from, the Gateway Device App.

  • Gateway Device App (GDA)

    You’ll build this software application to run within the gateway device (emulated or actual), and it will provide data management, analytics, and configuration management functionality. It’s primary role is to manage data and the connections between the CDA and cloud services that exist within the Cloud Tier. It will manage data locally as appropriate, and - sometimes - take action by sending a command to the constrained device that triggers an actuation. It will also manage some of the configuration settings - that is, those that represent nominal for your environment, and perform some simple analytics when new telemetry is received.

  • Cloud Services

    All cloud services applications and functionality do the heavy data processing and storage work, as they can theoretically scale it ad infinitum. This simply means that, if designed well, you can add as many devices as you want, store as much data as you want, and do in-depth analysis of that data - trends, highs, lows, configuration values, etc., all while passing any relevant insights along to a human end user, and perhaps even generate Edge Tier actions based on any defined threshold crossing(s). Technically, cloud services within an IoT environment can handle all the action categories previously mentioned, with the exception of data collection (meaning, they don’t perform sensing or actuation actions directly). You’ll build some cloud services to handle this functionality, but mostly utilize those generic services already available from some cloud service providers.

  Notional IoT detailed logical architecture with Edge and Cloud Tiers
Figure 1-4. - Notional IoT detailed logical architecture with Edge and Cloud Tiers

Putting it all together into a detailed logical architecture, Figure 1-4, shows how each major logical component within our two architectural tiers interacts with the others. We’ll use these two diagrams (Figure 1-3 and Figure 1-4) as our baseline architecture for all the exercises in this book.

Now that we have a handle on what we’re up against, let’s get our development environment setup so we can start slinging code.

Building, Testing and Deploying Software for the IoT

Building and deploying code across different operating systems, hardware configurations, and configuration systems is no walk in the park. With typical IoT projects, we not only have to deal with different hardware components, but the myriad ways to develop and deploy across these platforms, not to mention the various Continuous Integration / Continuous Deployment (CI/CD) idiosyncrasies with the various cloud service provider environments we often work within.

With all these challenges, how do we even get started? First things first - what problem are we trying to solve? As a developer, you want to implement your IoT design in code, test it, package it into something that can be easily distributed to one or more systems, and deploy it safely. We can think of our development challenges in two build, test and deploy phases, that just so happen to map to our architectural tiering: Edge Tier Environment and Cloud Tier Environment. We’ll dig into the functionality within the Cloud Tier beginning in Chapter 10. For now, we’ll focus just on our Edge Tier.

Although our Edge Tier will eventually have specialized hardware to deal with, we can simulate some aspects of these hardware components within our local development environment. This will make deployment much easier and is perfectly fine for all the exercises in this book except for Chapter 4, which I’ll touch on later in this chapter and is optional.

There are many ways to get up and running with IoT development, but we’ll focus on three specific paths: One is purely simulated, and as I mentioned previously is sufficient for all but Chapter 4. The other two require IoT-specific hardware and will be discussed in more depth in Chapter 4.

Integrated Simulated Deployment

This approach doesn’t require any specialized device, and allows you to use your development workstation (laptop) as both gateway device and constrained device. This means you’ll run your GDA and CDA in your local computing environment. You’ll emulate your sensing and actuation hardware by building simple software simulators to capture this functionality within your CDA. All exercises, with the exception of those in Chapter 4, will work using this deployment approach.

Separated Physical Deployment

This requires a hardware device, such as a Raspberry Pi, that meets the hardware criteria listed in the below, giving you the ability to connect to and interact with real sensors and actuators. Although many off-the-shelf single-board computing (SBC) devices can be used as full-blown computing workstations, I’ll refer to this as your constrained device, and it will run your CDA directly on the device. As with the Simulated approach, you’ll run the GDA in your local computing environment.

Note

The Internet Engineering Task Force’s (IETF) Request For Comments document RFC 7228 defines various classes of constrained devices (also referred to as constrained nodes). These classes include Class 0 (very constrained), Class 1 (constrained), and Class 2 (somewhat constrained). 1 For our purposes, we’ll assume our CDA can run on Class 2 or even more powerful devices, which typically support full IP-based networking stacks, meaning the protocols we’ll deal with in this book will generally work on these types of devices. Although technically feasible to connect Class 2 devices directly to the Internet, all of our examples and exercises will interact indirectly with the Internet via the GDA.

Blended Physical Deployment

This approach is nearly identical to the Separated Deployment approach, but will run both your CDA and GDA on the SBC device (e.g. Raspberry Pi or other, similar system). This technically means you can choose to deploy each app to a different SBC, although it isn’t necessary for any of the exercises listed.

If you choose either Path B or C for your deployment, there are a wide range of inexpensive SBC’s that could work for you. Like choosing our device should have general purpose input/output (GPIO) functionality, at least one available Inter-Integrated Circuit (I2C) bus, support TCP/IP and UDP/IP networking via Wi-Fi or Ethernet, run a general purpose Linux-based operating system such as Debian (or a derivative such as Raspbian), and support Python 3 and Java 11 (or higher).

Although you may choose any SBC that works for your needs, the exercises in the book will focus on the Integrated Simulated Deployment Path, excepting Chapter 4 of course. Part II introduces the concept of integration with the physical world, and I’ll devote Chapter 4 to this type of integration using actual hardware. While the principles will apply to many SBC’s, any hardware-specific integration will focus on the Raspberry Pi platform - specifically, the Raspberry Pi 3 Model B+, Model 4 Model B, and the latest version of the Raspberry Pi Zero W.

Irrespective of the selected Deployment Path, all exercises and examples assume you’ll do your development and deployment on a single workstation. This involves a three-step process that includes preparation of your development environment, defining your testing strategy, and identifying a build and deployment automation approach. I’ll cover the basics of these steps to get you started in this chapter, but also add to each as you dig into later exercises that have additional dependencies, testing, and automation needs.

Note

As I mentioned previously in this section, the exceptions to the ‘develop on a single workstation’ assumption are the exercises in Chapter 4, where I’ll discuss implementation and deployment on real hardware. Again, this only applies if you are planning on following deployment path B and C, otherwise, all your Edge Tier code will deploy and run on your local development workstation.

Step I: Prepare Your Development Environment

Recall that your CDA will be written in Python, and your GDA will be written in Java. Let’s make sure your workstation has the right stuff installed to support these languages and their associated dependencies, by following the steps below.

  1. Install Python 3.7 or higher on your workstation (the latest version as of this writing is 3.8.2). You can check if it’s already installed, or install it if not, using the following steps:

    • Open a terminal or console window, and type:

      $ python3 –version

    • It should return something similar to the following:

      Python 3.7.6

    • If you get an error (e.g. ‘not found’), you’ll need to install Python 3.7 or higher. Follow the instructions for your operating system (Windows, Mac, Linux) at https://www.python.org/downloads/.

  2. Install pip by downloading the script located at https://bootstrap.pypa.io/get-pip.py.

    1. Use Python to execute the pip installation. Open a terminal or console window, and type:

      $ python3 get-pip.py

  3. Ensure Java 11 or higher is installed on your workstation (the latest version of OpenJDK as of this writing is Java 14). You can check if it’s already installed, or install it if not, using the following steps:

    1. Open a terminal or console window, and type:

      $ java –version

    2. It should return something like the following (make sure it’s at least Java 8)

      java 13.0.1 2019-10-15

      Java(TM) SE Runtime Environment (build 13.0.1+9)

      Java HotSpot(TM) 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)

    3. If you get an error (e.g. ‘not found’), you’ll need to install Java 11 or higher. Follow the instructions for your platform (Windows, MacOS, or Linux) here: https://openjdk.java.net/install/

  4. Install Git. Go to https://git-scm.com/book/en/v2/Getting-Started-Installing-Git and review the instructions for your specific operating system.

    Note

    A prerequisite for any of the exercises in this book, and for setting up your development environment, is a basic understanding of Git – a source code management and versioning tool. Many IDE’s (including Eclipse) come with source code management already enabled via an embedded Git client. In a previous step, you installed Git via the command line so you can run Git commands independently of your IDE. For more information on using Git from the command line, please see https://git-scm.com/docs/gittutorial.

    Tip

    You can use Git as a standalone source code management tool on your local development workstation, and also manage your source code in the cloud using a variety of free and commercial services. GitHub 2 is the service I use, and will leverage some of the features this platform provides to support CI/CD pipelines, or workflow steps, that allow you to automate the build, test, package, and deployment process.

  5. Create a working development directory, and download the source code and unit tests for this book:

    1. Open a terminal or console window, create a new working development directory, and then change to that directory. Type the following:

      1. Linux/macOS:

        mkdir $HOME/programmingtheiot

        cd $HOME/programmingtheiot

      2. Windows:

        mkdir C:programmingtheiot

        cd C:programmingtheiot

    2. Clone the following two source code repositories for this book by typing the following:

      $ git clone https://github.com/programmingtheiot/python-components.git

      $ git clone https://github.com/programmingtheiot/java-components.git

  6. Install and configure virtualenv using pip 3

    1. Open a terminal or console window. Change your directory to that of the constraineddeviceapp code you just cloned from GitHub, and install virtualenv by typing the following:

      1. Linux/macOS:

        $ pip install virtualenv

      2. Windows:

        C:programmingtheiot> pip install virtualenv

    2. Configure virtualenv for your environment. Change your directory to that of the constraineddeviceapp code you just cloned from GitHub, and type the following:

      1. Linux/macOS

        $ cd $HOME/programmingtheiot/python-components

        $ virtualenv -p python3 .venv

        $ . .venv/bin/activate

        (venv) $ pip install -r basic_imports.txt

      2. Windows

        cd C:programmingtheiotpython-components

        C:programmingtheiotpython-components> virtualenv -p C:PathToYourPythonExecutablepython.exe .venv

        C:programmingtheiotpython-components> .venvScriptsactivate.bat

        (.venv) C:programmingtheiotpython-components> pip install -r basic_imports.txt

    3. Your virtualenv environment is now set up, and your terminal is running within virtualenv. You can activate (using the activate script) and then deactivate virtualenv (using the deactivate command) from your command line easily enough:

      1. Linux/macOS

        $ . .venv/bin/activate

        (venv) $ deactivate

      2. Windows

        C:programmingtheiotpython-components> .venvScriptsactivate.bat

        (.venv) C:programmingtheiotpython-components> deactivate

At this point, your development workstation is mostly configured, so let’s dig into the next section and install some additional tools to make the development process more streamlined.

Configuring an Integrated Development Environment (IDE)

There are many excellent tools and IDE’s that help you, the developer, write, test, and deploy applications written in both Java and Python. There are tools that I’m very familiar with and work well for my development needs. My guess is you’re much the same and have your own tool preferences. It doesn’t really matter which toolset you use, provided they meet some basic requirements. For me, these include code highlighting and completion, code formatting and refactoring, debugging, compiling and packaging, unit and other testing, and source code control.

There are many fantastic IDE’s on the market - both commercial and open source, and the choice is certainly yours. I developed the examples in this book using the Eclipse IDE 4 with PyDev 5 installed, as it meets the requirements I’ve specified and provides a bunch of other convenient features that I regularly use in my development projects.

If you’re already familiar with writing, testing, and managing software applications using a different IDE, most of this section will be old hat. I do recommend you read through it, however, as this section sets the stage for the development of your GDA and CDA.

Note

I maintain detailed instructions for setting up Eclipse with PyDev for use with Programming the IoT on my website for students of my Connected Devices course. You can find more details on this books website at https://programmingtheiot.com/programming-the-iot-book.html.

Setup Your GDA Project

The first step in this process is to install the latest Eclipse IDE for Java development. You can find the latest download links for Eclipse at https://www.eclipse.org/downloads/. You’ll notice that there are many different flavors of the IDE available. For our purposes, you can simply choose ‘Eclipse IDE for Java Developers’. Then, follow the instructions for installing the IDE onto your local system.

Once installed, launch Eclipse, select ‘File -> Import’, and find ‘Git -> Projects from Git’, and click ‘Next’.

Select ‘Existing local repository’ and click ‘Next’. If you already have some Git repositories in your home path, Eclipse will probably pick them up and present them as options to import in the next dialog (not shown). To pull in the newly cloned repository, simply click ‘Add…’, which will take you to the next dialog, shown in Figure 1-5. From here, you can add your new Git repository.

On my workstation, the repository I want to import is located at E:akingprogrammingtheiotjava-components. Yours will most likely have a different name, so be sure to enter it correctly!

  Import java components from your local Git repository
Figure 1-5. - Import java-components from your local Git repository

Click ‘Finish’, and you’ll see your new repository added to the list of repositories you can import. Highlight this new repository and click ‘Next…’. Eclipse will then present you with another dialog, asking you to import the project, using one of the options as shown in Figure 1-6.

  Import java components as an existing Eclipse project
Figure 1-6. - Import java-components as an existing Eclipse project

You now have a choice to make: you can import the java components as an existing Eclipse project, using the new project wizard, or simply as a general project. Unless you want to fully customize your project environment, I’d recommend using the first option - import an existing Eclipse project. This process will look for a .project file in the working directory (which I’ve included in each of the repositories you’ve already cloned), resulting in a new Java project named ‘piot-java-components’.

Click ‘Finish’, and you’ll see your new project added to the list of projects in the Eclipse Package Explorer, which - by default - should be on the left side of your IDE screen.

Your GDA project is now set up in Eclipse, so let’s explore the files inside. Navigate to this project in Eclipse and click on the ‘>’ to expand it further, as shown in Figure 1-7.

  GDA project now setup and ready to use
Figure 1-7. - GDA project now setup and ready to use
Note

What if you don’t like the project name? Easy. You can simply right click the ‘piot-java-components’ name. select ‘Rename…’, type the new name, and click OK. Done. Just know that I’ll continue to refer to the project by the original name throughout the book :).

You’ll notice that there’s already two files in the project: One is GatewayDeviceApp in the programmingtheiot.gda.app package, and the other is at the top level called pom.xml. The GatewayDeviceApp is a placeholder to get you started, although you may replace it with your own. I’d recommend you keep the naming convention the same, however, as the pom.xml depends on this to compile, test, and package the code. If you know your way around Maven already, feel free to make any changes you’d like.

Note

For those of you new to Maven, the pom.xml is simply Maven’s primary configuration file and contains instructions for loading dependencies, their respective versions, naming conventions for your application, build instructions, and of course packaging instructions. Most of these dependencies are already included, although you may want to add your own if you find others to be useful. You’ll also notice that Maven has its own default directory structure, which I’ve kept in place for the Java repository. To learn more about these and other Maven features, I’d recommend you walk through the 5-minute Maven tutorial located at https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html.

Now, to make sure everything is in place and you can build, package, and run the GDA, do the following:

  1. Make sure your workstation is connected to the Internet.

  2. Build your project and create an executable package.

    1. Right click on the project ‘piot-java-components’ in Project Workspace, scroll down to ‘Run As’, and click ‘Maven install’.

      1. Since Maven will have to install any missing dependencies specified in the pom.xml, this may take a little bit to run, depending on your Internet connection speed and other factors.

    2. Check the output in the Console at the bottom of the Eclipse IDE screen. There should be no errors, with the last few lines similar to the following:

      [INFO] --- maven-install-plugin:2.4:install (default-install) @ gateway-device-app ---

      [INFO] Installing E:akingworkspacegdaprogrammingtheiot-java argetgateway-device-app-0.0.1.jar to C:Usersaking.m2 epositoryprogrammingtheiotgdagateway-device-app.0.1gateway-device-app-0.0.1.jar

      [INFO] Installing E:akingworkspacegdaprogrammingtheiot-javapom.xml to C:Usersaking.m2 epositoryprogrammingtheiotgdagateway-device-app.0.1gateway-device-app-0.0.1.pom

      [INFO] ------------------------------------------------------------------------

      [INFO] BUILD SUCCESS

      [INFO] ------------------------------------------------------------------------

      [INFO] Total time: 4.525 s

      [INFO] Finished at: 2020-07-04T14:31:45-04:00

      [INFO] ------------------------------------------------------------------------

  3. Run your GDA application within Eclipse.

    1. Right click on the project ‘programmingtheiot-java’ again, scroll down to ‘Run As’, and this time click ‘Java application’.

    2. Check the output in the Console at the bottom of the Eclipse IDE screen. As with your Maven build, there should be no errors, with the output similar to the following:

      Jul 04, 2020 3:10:49 PM programmingtheiot.gda.app.GatewayDeviceApp initConfig INFO: Attempting to load configuration.

      Jul 04, 2020 3:10:49 PM programmingtheiot.gda.app.GatewayDeviceApp startApp INFO: Starting GDA...

      Jul 04, 2020 3:10:49 PM programmingtheiot.gda.app.GatewayDeviceApp startApp INFO: GDA ran successfully.

At this point, you’re ready to start writing your own code for the GDA. Now let’s get your development workstation setup for the CDA.

Setup Your CDA Project

This process will mimic the GDA setup process, but requires the addition of PyDev to Eclipse. Here’s a summary of activities to get you started.

If not already running, launch the Eclipse IDE. In a separate window or screen, open your web browser and navigate to https://marketplace.eclipse.org/content/pydev-python-ide-eclipse. Drag the PyDev “Install” icon from the web page and drop it near the top of the Eclipse IDE (you’ll see a green ‘plus’ icon show up, which is the indicator you can drop it into the IDE). Eclipse will then automatically install PyDev and its dependencies for you.

Once PyDev is installed, you can switch the Python interpreter to use the virtualenv environment you created in the previous section. Select ‘Preferences -> PyDev -> Interpreters -> Python Interpreter’. Eclipse will present a dialog similar to that shown in Figure 1-8.

  Add a new Python interpreter
Figure 1-8. - Add a new Python interpreter

Then, simply add a new interpreter using the ‘Browse for python/pypy.exe’ selection and provide the relevant information in the next popup window. Once complete, select the virtualenv interpreter and click ‘Up’ until it’s at the top of the list. At this point virtualenv will be your default Python Interpreter, as Figure 1-9 indicates. Click ‘Apply and Close’.

  Virtualenv Python interpreter now set as default
Figure 1-9. - Virtualenv Python interpreter now set as default

Once these steps are complete, select ‘File -> Import’, and import the python-components Git repository you’ve already cloned from GitHub. Again, this is nearly identical to the previous steps shown in Figures 1-5, 1-6, and 1-7, except you’ll import the python-components Git repository you cloned from GitHub.

On my workstation, the repository I want to import is located at E:akingprogrammingtheiotpython-components. As with the GDA, your repository name will likely be different, so be sure to use the correct path. I’ve also included the Eclipse .project file within this repository, so you can simply import it as an Eclipse project. This one will default to Python, so will use PyDev as the project template. Again, you can import any way you’d like, but my recommendation is to use import it as you did with the GDA.

Once you complete the import process, you’ll notice a new project in your Package Explorer named ‘piot-python-components’. You now have the CDA components set up in your Eclipse IDE.

To view the files inside, navigate to ‘piot-python-components’ and click on the ‘>’ to expand it further, as shown in Figure 1-10.

  CDA project now setup and ready to use
Figure 1-10. - CDA project now setup and ready to use

You’ll notice that there’s already six files in the project: One is ConstrainedDeviceApp in the programmingtheiot.cda.app package, and the other is at the top level called pom.xml. There are also four __init__.py files, which are empty files the Python interpreter uses to determine which directories to search for Python files (you can ignore these for now). Much like the GDA example previously given (and written in Java), the ConstrainedDeviceApp is simply a placeholder to get you started.

There’s also a Maven pom.xml provided, which may seem odd for those familiar with Maven, as this is a Python project, not Java, and Maven is often associated with building, testing and packaging Java applications. So, why Maven? It’s rather flexible and doesn’t really care about the type of files it manages, so will be useful for testing and packaging Python modules. You won’t need this capability until you start working on the exercises in the next chapter, so for now, you only need to verify your CDA runs by doing the following:

Note

If you’ve worked extensively with Python, you’re likely familiar with the PYTHONPATH environment variable. Since I’ve attempted to keep the GDA and CDA packaging scheme similar, you may need to tell PyDev (and your virtualenv environment) how to navigate this directory structure to run your application. Make sure ‘python’ is set for both ‘main’ and ‘test’ in PYTHONPATH by doing the following: Right click ‘piot-python-components’, select ‘PyDev - PYTHONPATH’, then click ‘Add source folder’, as shown in Figure 1-11. Select the ‘python’ folder under ‘main’, and click ‘Apply’. Do the same for the ‘python’ folder under ‘test’. Click ‘Apply and Close’ to finish.

  Updating the PYTHONPATH environment variable within PyDev and Eclipse
Figure 1-11. - Updating the PYTHONPATH environment variable within PyDev and Eclipse
  1. Run your CDA application within Eclipse.

    1. Right click on the project ‘piot-python-components’ again, scroll down to ‘Run As’, and this time click Python Run’.

    2. Check the output in the Console at the bottom of the Eclipse IDE screen. As with your GDA test run, there should be no errors, with the output similar to the following:

      2020-07-06 17:15:39,654:INFO:Attempting to load configuration...

      2020-07-06 17:15:39,655:INFO:Starting CDA...

      2020-07-06 17:15:39,655:INFO:CDA ran successfully.

Congratulations! Both your GDA and CDA are set up and working within your IDE. At this point, you’re technically ready to start writing your own code for both applications. But before we jump into the exercises in Chapter 2, we need to get our heads wrapped around the next two steps: testing and automation.

Step II: Define Your Testing Strategy

Now that your development environment is established for your GDA and CDA, we can discuss how you’ll test the code you’re about to develop. Obviously, good testing is a critically important part of any engineering effort, and programming is no exception to this. Every application you build should be thoroughly tested, whether it works completely independently of other applications or is tightly integrated with other systems. Further, every unit of code you write should be tested to ensure it behaves as expected. What exactly is a unit? For our purposes, a unit is always going to be represented as a function or method that you want to test.

Note

What’s the difference between a function and a method? To grossly oversimplify, a function is a named grouping of code that performs a task (such as adding two numbers together) and returns a result. If the function accepts any input, it will be passed as one or more parameters. A method is almost identical to a function, but is attached to an object. In object oriented parlance, an object is simply a class that’s been instantiated, and a class is the formal definition of a component - its methods, parameters, construction, and deconstruction logic. All of the Java examples in this book will be represented in class form with methods defined as part of each class. Python can be written in script form with functions or as classes with methods, but I prefer to write Python classes with methods and will do so for each Python example shown in this book, with only a few exceptions to this.

Unit, Integration, and Performance Testing

There are many ways to test software applications and systems, and some excellent books, articles and blogs on the subject. Developing a working IoT solution requires careful attention to testing - within an application and between different applications and systems. For the purposes of the solution you’ll develop, I’ll focus on just three: Unit tests, integration tests, and performance tests.

Unit tests are code modules written to test the smallest possible unit of code that’s accessible to the test, such as a function or method. These tests are written to verify a set of inputs to a given function or method returns an expected result. Boundary conditions are often tested as well, to ensure the function or method can handle these types of conditions appropriately.

Note

A unit of code can technically be a single line, multiple lines of code, or even an entire code library. For our purposes, a unit refers to one or more lines of code, or an entire code library, that can be accessed through a single interface that is available on the local system - that is, a function or a method which encapsulates the unit’s functionality and can be called from your test application. This functionality can be, for example, a sorting algorithm, a calculation, or even an entry point to one or more additional functions or methods.

I use JUnit for unit testing Java code (included with Eclipse), and PyUnit for unit testing Python code (part of the standard Python interpreter, and available within PyDev). You don’t have to install any additional components to write and execute unit tests within your IDE if you’re using Eclipse and PyDev.

Note

In your GDA project, you’ve likely noticed that Maven creates two directory structures for your source code: one for Java source code located in ./src/main/java, and another Java unit test code located in ./src/test/java. Since this is the default convention for Maven projects, to leave it as is.

In your CDA project, the code path structure is a bit different than the GDA project: the ./src/main/java and ./src/test/java are missing. That probably seems quite sensible, since this is a Python project and not Java! All the source code is contained within ./src/programmingtheiot, while all the tests are contained within ./tests/programmingtheiot.

Although I provide a bunch of unit tests for you to use, we’ll dig into creating some of your own in Chapter 2. If you’re new to this area of development, I’ve provided a couple simple code snippets below in both Java and Python to help you become more familiar with the code structure. Here’s a simple unit test in Java using JUnit 6 that verifies the method addTwoIntegers() behaves as expected:

@Test

public int testAddTwoIntegers(int a, int b)

{

MyClass mc = new MyClass();

// baseline test

assertTrue(mc.addTwoIntegers(0, 0) == 0);

assertTrue(mc.addTwoIntegers(1, 2) == 3);

assertTrue(mc.addTwoIntegers(-1, 1) == 0);

assertTrue(mc.addTwoIntegers(-1, -2) == -3);

assertFalse(mc.addTwoIntegers(1, 2) == 4);

assertFalse(mc.addTwoIntegers(-1, -2) == -4);

}

What if you have a single test class with two individual unit tests, but you only want to run one? Simply add @Ignore before the @Test annotation, and JUnit will skip that particular test. Remove the annotation to re-enable the test.

Let’s look at the same example in Python, using Python 3’s built-in PyUnit framework and the unittest library 7:

def testAddTwoIntegers(self, a, b):

MyClass mc = MyClass()

# baseline test

self.assertTrue(mc.addTwoIntegers(0, 0) == 0)

self.assertTrue(mc.addTwoIntegers(1, 2) == 3)

self.assertTrue(mc.addTwoIntegers(-1, 1) == 0)

self.assertTrue(mc.addTwoIntegers(-1, -2) == -3)

self.assertFalse(mc.addTwoIntegers(1, 2) == 4)

self.assertFalse(mc.addTwoIntegers(-1, -2) == -4)

PyUnit, much like JUnit, allows you to disable specific tests if you wish. Simply add @unittest.skip("Put your reason here.”), or just @unittest.skip as the annotation before the method declaration, and the framework will skip over that specific test.

Integration tests are super important for the IoT, as they can be used to verify that the connections and interactions between systems and applications work as expected. Let’s say you want to test a simple sorting algorithm using a basic data set embedded within the testing class - you’ll typically write one or more unit tests, execute each one, and verify all is well. Simple, right?

What if, however, the sorting algorithm needs to pull data from a data repository accessible via your local network or even the Internet? So what, you might ask? Well, now you have another dependency just to run your sort test. You’ll need an integration test to verify that data repository connection is both available and working properly before exercising the sorting unit test.

These kinds of dependencies can make integration testing pretty challenging with any environment, and even more so with the IoT, since we sometimes have to set up servers to run specialized protocols to test our stuff. Fortunately, Maven can help with this, too, since it supports plugins which make some of this much easier. For all the code that needs to integrate only within the Edge Tier, we’ll use Maven plus some specialized Unit Tests as part of our integration testing strategy.

Finally, performance tests are useful for testing how quickly, or efficiently, a system handles a variety of conditions. They can be used with both unit and integration tests when, for instance, response time or the number of supported concurrent, or simultaneous connections needs to be measured.

Let’s say there are many different systems that need to retrieve a list of data from your data repository, and each one wants that list of data sorted before your application returns it to them. Ignoring system design and database schema optimization for a moment, a series of performance tests can be used to time the responsiveness of each system’s request (from the initial request to the response), as well as the number of concurrent systems that can access your application before it no longer responds adequately.

Another aspect of performance testing is to test the load of the system your application is running on, which can be quite useful for IoT applications. IoT devices are generally constrained in some way - memory, CPU, storage, etc - whereas cloud services can scale as much as we need them to. It stands to reason then, that our first IoT applications - coming up in Chapter 2 - will set the stage for monitoring each device’s performance individually.

Since performance testing often goes hand-in-hand with both integration and unit testing, we’ll continue to use Maven and specialized Unit Tests for this as well, along with open source tools where needed.

Note

There are many performance testing tools available, and you can also write your own. While system-to-system and communications protocol performance testing is completely optional for the purposes of this book, I’ll cover it in more detail beginning in Part III, Chapter 5, where I’ll introduce Locust 8, a tool you can use in your local environment to test scalability for your solution. Locust allows for extensions that enable you to test many different protocols and connection paradigms, which will be important for device-to-device communications testing between your GDA and CDA.

Testing Tips for the Exercises in this Book

The sample code provided for each exercise in this book includes unit tests, which you can use to test the code you’ll write. These unit tests are provided as part of the connected-devices-java and connected-devices-python repositories you’ve already pulled into your new iot-gateway and iot-device (respectively) projects are key to ensuring your implementation works properly.

Some exercises also have integration tests that you can use as-is, or modify to suit your specific needs. I’ve also included some sample performance tests you can use to test how well some of your code performs when under load.

Your implementation of each exercise should pass each provided unit test with 100% success. You’re welcome to add more unit tests if you feel they’ll be helpful to verify the functionality you develop. The integration tests and performance tests provided will be helpful, but not required as part of each exercise.

Remember, tests are your friend. And, like a friend, they shouldn’t be ignored. They can surely be time consuming to write and maintain, but any good friendship takes investment. These tests - whether unit, integration, or performance - will help you validate your design and verify your functionality is working properly.

Step III: Managing Your Workflow - Requirements, Source Code, and CI/CD

So you’ve figured out how you want to write your code and test it - great! But wouldn’t it be great if you could manage all your requirements, source code, and CI/CD pipelines. Let’s tackle this in our last step, which is all about managing your overall development process workflow. This includes requirements tracking, source code management, and CI/CD automation.

You’re probably sick of me saying that building IoT systems is hard, and that’s largely because of the nature of the Edge Tier (since we often have to deal with different types of devices, communication paradigms, operating environments, security constraints, and so on). Fortunately, there are many modern CI/CD tools that can be used to help navigate these troubled waters. Let’s look at some selection requirements for these tools, and then explore how to build out a CI/CD pipeline that will work for our needs.

Your IoT CI/CD pipeline should support secure authentication and authorization, scriptability from a Linux-like command line, integration with Git and containerization infrastructure, and the ability to run pipelines within my local environment as well as a cloud-hosted environment.

There are many online services that provide these features, some of which provide both free and paid service tiers. When you downloaded the source code for this book, you pulled it from my GitHub 9 repositories using Git’s clone feature. GitHub is an online service that supports overall developer workflow management, including source code control (using Git), CI/CD automation, and planning.

Each exercise will build, test, and deploy locally, but also assume your code is committed to an online repository using Git for source code management. You’re welcome to use the online service of your choice, of course. For this book, all examples and exercises will assume GitHub is being used. You can learn more about GitHub’s features and create a free account on their website (https://github.com/).

Note

There are lots of great resources, tools and online services available that let you manage your development work and set up automated CI/CD pipelines. Read through this section, try things out, and then as you gain more experience, choose the tools and service that works best for you.

Managing Requirements

Ah yes - requirements. What are we building, who cares, and how are we going to build it? Plans are good, are they not? And since they’re good, we should have a tool that embraces goodness, which includes: task prioritization, task tracking, team collaboration, and (maybe) integration with other tools.

Remember the code you cloned from my GitHub repository? Choose one - for example, constraineddeviceapp, and open its URL in a web browser. You’ll notice that it has some Issues (like, literally - there’s a column called ‘Issues’ on the main page). If you click on ‘Issues’, you’ll see all the tasks (requirements) that need to be implemented.

Each task contains the instructions and other details a developer (you) will need to write the code and make sure it works correctly.

So far, so good, right? These tasks - at least in GitHub - can be organized into a board, so that you can see all of the things that need to be implemented for a project. You’ve probably heard of Agile 10 project management processes such as Scrum and Kanban.

One of the workflow organizational approaches GitHub let’s you select for a project is a Kanban-based electronic board. Kanban organizes tasks as cards, which can then be placed into various columns within the board’s workflow depending on their status. This is the approach we’ll use in the book to track things that need to be done for each code repository.

Note

I’m managing all of the activities for this book within a Kanban board, too. Each card on the board represents a task I (or one of my team members) needs to complete. Cards only move to ‘Done’ after Sarah approves (thanks Sarah!)

This approach is really powerful, and I think you’ll see how it will help you organize the work you need to do and track your progress as you go through this book.

Setting up a Cloud Project and Repositories

This book is actually organized as a large IoT project using a single Kanban board across the repositories. Each repository will have a set of tasks that are aligned to chapter-based milestones, which map back into the Kanban board.

The beauty of this approach is that, once your project and repositories are properly set up, all you really need to do is update your task status as you work through each chapter writing, testing and committing your code. The workflow engine will automatically parse your task updates, and represent the task’s card on the board for you. Neat, huh?

If you’re interested in using GitHub to set all of this up, here’s a simple 3-step process you can use to get started:

  1. Create a GitHub account

    1. If desired, create an organization associated with the GitHub account

  2. Within the your account (or organization), create the following:

    1. A new private project named ‘Programming the IoT - Exercises’

    2. A new private Git repository named ‘piot-java-components’

    3. A new private Git repository named ‘piot-python-components’

  3. Update the remote repository for both ‘piot-java-components’ and ‘piot-python-components’

    1. From the command line, execute the following commands:

      git remote set-url origin {your new URL}

      git commit -m “Initial commit.”

      git push

    2. IMPORTANT: Be sure to do this for both ‘piot-java-components’ and ‘piot-python-components’, using the appropriate Git repository URL for each!

Note

I’ve written a guide to help you configure all of this, located at the book’s homepage (https://programmingtheiot.com/programming-the-iot-book.html).

Once you complete the tasks above, your cloud-based workflow environment is now set up and ready to use. Take a look at Figure 1-12. It shows the task template I’ll use throughout the book (GitHub calls this an ‘Issue’). This is the stuff that goes into each task.

It’s rather simple, and you only have to enter five items: Name, Description, Actions, Estimate, and Validation. You can update this, but I’d recommend sticking with one template for all your cards. If you create a card that doesn’t have any validation step (for some reason), just enter ‘N/A’.

  Task template
Figure 1-12. - Task template

Most of these categories are self-explanatory. But, why only three levels-of-effort for Estimate? In this book, most of the activities should take ~2 hours or less (Small), about half a day (Medium), or about one day (Large). If you come across an activity that you’re sure will take more than a day, it should be broken down into smaller tasks that fit one of these three levels-of-effort.

For example, a ‘task’ with the name Integrate IoT solution with three cloud services certainly represents work that needs to be done, but judging by the name only, it’s clearly way too big and complicated to be a single work activity.

My suggestion is to write tasks that align to a specific code module, or even a single method, as much as possible. I’m sure you’ve heard this before: Keep it simple.

Figure 1-13 shows an example of the template filled in with the first coding task you’ll have - creating the Constrained Device Application (CDA).

  Example of a typical development task
Figure 1-13. - Example of a typical development task

And Figure 1-14 shows the result of adding the task as a Kanban card. This card was generated automatically after aligning the task to a project. Notice it’s been added into the “To do” column on the board, since it’s new and there’s no status as of yet. Once you start working on the task and change its status, it will move to ‘In progress’.

  The new example task automatically added into the Kanban board
Figure 1-14. - The new example task automatically added into the Kanban board

Source Code Control Using Git Remotes and Branching

One of the key benefits of using Git is the ability to collaborate with others and synchronize your local code repository with a remote repository stored in the cloud. If you’ve worked with Git before, you’re already familiar with remotes and branching. I’m not going to go into significant detail here, but they’re important concepts to grasp as part of your automation environment.

Branching is simply a way of enabling each developer, or team, to segment their work without negatively impacting the main code base. In Git, this default main branch is currently called ‘master’, and is typically used to contain the code that has been completed, tested, verified, and - usually - placed into production. This is the default branch for both ‘programmingtheiot-java’ and ‘programmingtheiot-python’, and while you can leave it as is and simply work off this default branch, it’s not generally recommended for the reasons I mentioned.

Branching strategies can differ from company to company and team to team, although the one I like to use has each chapter in a new branch, then once all is working correctly and properly tested, the chapter branch gets merged into the master. From there, a new branch is created from the merged master for the next chapter, and so on.

This approach allows you to easily track changes between chapters, and even go back to the historical record of an earlier chapter if you want to see what changed between, say, Chapter 2 and Chapter 5. In Eclipse, you can right-click on the project (either ‘programmingtheiot-java’ or ‘programmingtheiot-python’), and choose ‘Team > Switch To > New Branch…’ to establish a new branch for your code.

I’d suggest you use the naming convention of ‘chapternn’ as each branch name, where ‘nn’ is the two-digit chapter number. For instance, the branch for Chapter 1 will be named ‘chapter01’, Chapter 2 will be named ‘chapter02’, and so on.

Note

All the gory details on Git branching and merging are out of scope for this book, so I’d recommend reading the following guide if you’d like to dig in https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging.

Automated CI/CD in the Cloud

Within Eclipse, you can write your CDA and GDA code, execute unit tests, and build and package both applications. This isn’t actually automated, since you have to start the process yourself by executing a command like mvn install from the command line, or invoking the Maven install process from within the IDE. This is great for getting both applications to a point where you can run them, but doesn’t actually run them - you still need to manually start the applications up and then run your integration and / or performance tests.

As a developer, part of your job is writing and testing code to meet the requirements that have been captured (in cards on a Kanban board, for example), so there’s always some manual work involved. Once you know your code units function correctly, it would be pretty slick to have everything else run automatically - say, after committing and pushing your code to the remote dev branch (such as ‘chapter02’, for example).

GitHub supports this automation through GitHub actions 11. I’ll talk more about this in Chapter 2 and help you set up your own automation for the applications you’re going to build.

Automated CI/CD in your Local Development Environment

There are lots of ways to manage CI/CD within your local environment. GitHub actions can be run locally using self-hosted runners 12, for example. There’s also a workflow automation tool called Jenkins 13 that can be run locally, integrates nicely with Git local and remote repositories, and has a plugin architecture that allows you expand its capabilities seemingly ad infinitum.

Warning

There are lots of great 3rd party Jenkins plugins, but some are not well maintained and may introduce security vulnerabilities. If it’s a plugin you’re certain you need, be sure to track how often it’s updated and patched. If it’s not regularly maintained, it could introduce compatibility and security issues to your environment.

Once installed and secured, you can configure Jenkins to automatically monitor your Git repository locally or remotely, and run a build / test / deploy / run workflow on your local system, checking the success at each step. If, for example, the build fails because of a compile error in your code, Jenkins will report on this and stop the process. The same is true if the build succeeds, but the tests fail - the process stops at the first failure point. This ensures your local deployment won’t get overwritten with an update that doesn’t compile or fails to successfully execute the configured tests.

Setting up any local automation tool can be a complicated and time consuming endeavor. It’s super helpful, however, as it basically automates all the stuff you’re going to do to build, test and deploy your software. That said, it’s not required for any of the exercises in this book.

I’ve detailed my own approach using GitHub actions and self-hosted runners as well as Jenkins in a document on my website at https://programmingtheiot.com/setup/ConnectedDevices_DevEnvironmentSetup.html. See the Table of Contents to navigate to the section most relevant for your environment.

A Few Thoughts on Containerization

You’ve likely heard of containerization, which is simply a way to package your application and all of its dependencies into a single image, or container, that can be deployed to many different operating environments. This approach is very convenient, since it allows you to build your software and deploy it in such a way as to make the hosting environment no longer a concern, provided the target environment supports the container infrastructure you’re using.

Docker 14 is essentially an application engine that runs on a variety of operating systems, such as Windows, Mac, and Linux, and serves as a host for your container instance(s). Your GDA and CDA, for example, can each be containerized and then deployed to any hardware device that supports the underlying container infrastructure and runtime.

It’s worth pointing out that containerizing any application that has hardware-specific code may be problematic as it will not be portable to another, different hardware platform (even if the container engine is supported). If you want your hardware-specific application to run on any platform that supports Docker, that platform would require a hardware-specific emulator compatible with the code developed for the application.

For example, if your CDA has code that depends on Raspberry Pi-specific hardware, This is less of a concern for us at the moment, since you’ll be simply emulating sensors and actuators and won’t have any hardware-specific code to worry about until Chapter 4 (which, again, is optional). I’ll discuss this more in Chapter 4, along with strategies to overcome hardware-specificity in your CDA.

When using CI/CD pipelines in a remote, or cloud, environment, you’ll notice that these services will likely deploy to virtual machines and run your code within a container that includes the required dependencies, all configured as part of the pipeline. For many cases, this makes perfect sense and can be an effective strategy to ensure consistency and ease of deployment.

To keep things a bit more simple, I won’t walk through containerization in this book for use within your development environment and as part of your workstation, even though there are many benefits to doing so. The primary reason is that it adds another layer of complexity to manage initially and I want to get you up and running with your own applications as soon as possible.

If you’re interested in learning more about containerizing your IoT workstation and local environment, I’ve detailed my approach for the GDA, CDA, and other related applications within https://programmingtheiot.com/setup/ConnectedDevices_DevEnvironmentSetup.html. See the section labeled ‘Using Containers’ for more information.

Conclusion

Congratulations! You’ve just completed the longest - and perhaps most tedious - chapter in the book. You learned about some basic IoT principles, created a problem statement to drive your IoT solution, and established a baseline IoT systems architecture that includes the cloud tier and edge tier.

Perhaps most importantly, I introduced two applications - the GDA and CDA - which will serve as the foundation for much of your IoT software development throughout this book, and that you’ll start building in Chapter 2. Finally, you set up your development environment and workflow, learned about requirements management, explored unit, integration and performance testing, and considered some basic CI/CD concepts to help automate your builds and deployment.

You’re now ready to start building your first two IoT applications using Python and Java. If you’re ready to move on, I’d suggest you grab a good cup of coffee or tea. Let’s dig in.

1 Details on the various classes of constrained devices can be found within the IETF’s RFC 7228 (https://tools.ietf.org/html/rfc7228).

2 More information on GitHub can be found on their website (https://github.com/).

3 Virtualenv provides a way to contain your Python 3 environment and all the necessary library dependencies within a single, lightweight, virtual environment.

4 The Eclipse IDE version used for the exercises and examples listed in this book is 2020-06 and can be downloaded from the Eclipse website (https://www.eclipse.org/downloads/).

5 More information on PyDev can be found on the project’s website (http://www.pydev.org/).

6 Detailed information on JUnit and the latest version, JUnit 5, can be found on the JUnit website (https://junit.org/junit5/).

7 Detailed information on Python 3’s unittest library can be found in the unittest documentation (https://docs.python.org/3/library/unittest.html).

8 For more information on Locust, see the documentation from their website (https://docs.locust.io/en/stable/what-is-locust.html).

9 You can read more about GitHub and its Git hosting features on their website (https://github.com/).

10 Read more about the Agile Manifesto at https://agilemanifesto.org/.

11 GitHub actions is a feature available within GitHub that allows customized workflows to be created for those who have an account within GitHub. You can read more about GitHub actions on their website (https://docs.github.com/en/actions).

12 Self-hosted runners, part of GitHub actions, allow you to run your action workflows locally. There are caveats, of course, and security considerations. You can read more about self-hosted runners on the GitHub actions documentation website (https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners).

13 You can read more about Jenkins and its automation features at their website (https://www.jenkins.io/).

14 You can read more about containerization concepts and Docker containerization products at their website (https://www.docker.com/resources/what-container).

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
52.55.55.239