IoT edge devices capture data gathered in events and transmit lightweight notifications of conditions or discrete state changes. Condition changes are commonly reported in a time series of interrelated events that are then analyzed to determine what has happened. Discrete events indicating a change in state can drive the need to perform specific actions.
When thousands or more IoT devices are deployed, thousands to millions of events per second can land in the cloud for further data processing and analysis. Microsoft’s first hub capable of ingesting telemetry from large numbers of IoT devices at rates of over 1 GB per second was its Azure Event Hub cloud service in the Microsoft Azure Service Bus. The Event Hub became generally available in November 2014. It supports AMQP, AMQP over WebSockets, and HTTPS as protocols for communications to the cloud. However, though Event Hubs were designed for streaming data ingestion, they were not designed to enable communications back to IoT devices.
In February 2016, Microsoft announced general availability of the Azure IoT Hub, a cloud service designed for both IoT device-to-cloud and cloud-to-IoT device communications. Building upon previous Event Hub capabilities for ingestion, the IoT Hub also supports MQTT and MQTT over WebSockets protocols, per-device identity, file upload from devices, device provisioning, device twin and device management, device streams, and the Azure IoT Edge. Through the IoT Hub, it is possible to track device creation, device connections, and device failures. Additional communications protocols can be supported through deployment of custom Azure IoT protocol gateways in the cloud, though protocol conversion is more often deployed in the IoT Edge (as described in the previous chapter).
IoT Hub capabilities
Configuring the IoT Hub
IoT Hub Performance Monitoring
IoT Hub Device Provisioning
IoT Hub Availability and Disaster Recovery
IoT Hub Capabilities
Storing, synchronizing, and enabling querying of device metadata and state information using device twins
Providing the ability to set the device state either by individual device or based on common characteristics of multiple devices
Providing automatic response to device-reported state changes through message routing integration
The status of the IoT Hub is determined through monitoring of device identity operations, device telemetry and diagnostics, cloud-to-device commands, and connections. Each device uses its own security key to connect to the IoT Hub. You can individually whitelist or blacklist each device providing complete control over device access.
Device applications can read and receive notification of changes in desired properties that reside in the device twin in the IoT Hub. The desired properties are modifiable by the IoT solution backend. Reported properties in the device twin are modifiable by device applications, and changes are read by the backend. Tags stored in the device twin contain device metadata and are accessible by only the backend. In addition to storing device metadata and reporting on current state information, device twins can be used to synchronize the state of long-running workflows between devices and backend applications.
Azure Stream Analytics
Azure Time Series Insights
Azure Databricks
Apache Spark
Apache Storm (spout)
Azure Functions
Azure Logic Apps
Azure Functions enable creation and deployment of actions that contain custom written code in C#, F#, or Java. Azure Logic Apps provide a collection of pre-defined actions that can be orchestrated using a GUI-based development environment. Both are deployed as serverless workloads. We’ll describe the analytics, machine learning, and other related backend capabilities and components in Chapter 5.
Data can be retained in the IoT Hub’s built-in Event Hubs for up to 7 days. Messages that are at maximum message sizes are retained for 24 hours at a minimum.
Configuring the IoT Hub
An Azure IoT Hub can be created and managed in a variety of ways including through the Azure Portal, Azure CLI, or using PowerShell. Here, we’ll describe creating an IoT Hub using the Azure Portal.
The next step is to choose the pricing and scaling tier (basic or standard levels for production) based on the features desired and number of messages per day you want the IoT Hub to be capable of handling. Pricing in levels is provided on a per-month basis in the portal interface.
Basic and standard levels message scalability
Option Level | Messages/ Unit/Day | Number of Units |
---|---|---|
Basic B1 | 400 K | 1 to 200 |
Basic B2 | 6 M | 1 to 200 |
Basic B3 | 300 M | 1 to 10 |
Standard S1 | 400 K | 1 to 200 |
Standard S2 | 6 M | 1 to 200 |
Standard S3 | 300 M | 1 to 10 |
During this step, you also choose the number of IoT Hub units to be deployed in the pricing tier that you selected. An appropriate number of units is selected that will deliver the desired number of messages per day by using a slider bar in the interface.
Once you’ve completed this step, you can review your selections and create the IoT Hub. Resource capabilities that you’ve defined are then provisioned.
Managing the IoT Hub
When a hub is created, you can review parameters associated with the IoT Hub through the overview provided in the Azure Portal. You can also view the hub’s activity log of operations and adjust IoT Hub access control and settings.
For example, in settings, you can change the pricing and scale tier and number of units if the projected number of messages being handled changes. You can also adjust operations monitoring (such as turning event logging on or off) and specify valid IP address ranges that the IoT Hub will accept. Shared IoT Hub access policies that can be adjusted in settings include permissions for identity registry reads, identity registry writes, the service connect to service endpoints, and the device connect sending and receiving of messages.
Note
The identity registry contains information about the devices and modules allowed to connect to the IoT Hub and stores credentials used in authenticating the devices and modules. You must add device IDs and keys to the identity registry to enable the devices to connect to the IoT Hub, normally through the Azure IoT Hub Device Provisioning Service.
The portal also provides access to explorers for query and to IoT devices. You can use the portal to choose automatic device management (IoT Edge and IoT device configuration), designate file uploads and message routing, manually initiate failover from an IoT Hub primary to secondary location, access the Azure Security Center, and monitor the hub for alerts and metrics. Finally, you can determine resource health and/or make a support request.
Message Routing and Event Routing
Device registration to an IoT Hub.
Device deletion from an IoT Hub.
Device connection to an IoT Hub.
Device disconnection from an IoT Hub.
Device telemetry message is sent to an IoT Hub.
You can query message application and system properties, message bodies, device twin tags, and device twin properties.
Alternatively, you might want to set up a publish-and-subscribe event routing service by integrating the Azure IoT Hub with an Azure Event Grid. In this configuration, the IoT Hub publishes events to endpoint subscribers. Maximum message size is also 256 KB in this scenario. You can filter data using message properties, message body, and device twin in the IoT Hub before publishing to the Azure Event Grid.
It is important to realize that events will not necessarily arrive at endpoints in the order in which they were published. In such scenarios where order is important, message routing should be chosen over leveraging an Event Grid.
An Azure IoT Hub can support up to 500 endpoints when integrated with Azure Event Grid. Endpoint types supported include Azure Automation, Azure Functions, Azure Event Hubs, Azure Logic Apps, Storage Blobs, Custom Topics, Queue Storage, Microsoft Flow, and third-party services through WebHooks.
IoT Hub Performance Monitoring
Active devices
Total devices
Total messages
Messages per second
Failed messages
Failed device connections
Failed twin updates
If you are going to deploy new or untested devices and want to understand what monitoring output from them might look like, you can choose to simulate output from the devices during solution development. Microsoft provides a device simulation solution accelerator for this purpose. It is found on the Microsoft IoT solution accelerator web site at https://azure.microsoft.com/features/iot-accelerators.
IoT Hub Device Provisioning
The IoT Hub Device Provisioning Service is used to provision IoT devices without the presence of hardcoded IoT connection information in the device. During manufacturing of the device, the device is programmed to call the Provisioning Service when it is turned on so that it can get connection information and its IoT solution assignment.
Load balancing of devices across multiple hubs.
Connect devices to IoT solutions based on transaction data or use case.
Connect devices to IoT Hubs with lowest latency using AMQP, AMQP over WebSockets, MQTT, MQTT over WebSockets, or HTTPS.
Reprovision the device when the device is changed.
Roll keys used by the device to connect to the IoT Hub (if not using X.509 certificates to connect) .
Secure attestation support for X.509 and TPM identities
An enrollment list that includes devices or device groups that might register and device configuration information
Allocation policies that control how the Device Provisioning Service assigns devices to IoT Hubs
Monitoring and diagnostics logging
Multiple IoT Hub assignments for devices
Cross-region assignments of IoT Hubs for devices
For service operations, uses HTTPS as a protocol
Note
The Device Provisioning Service can also be used to pre-authorize devices paired with over-the-air software update solutions such as Mender.io. Such solutions can help assure that software updates are performed in a secure manner and that software on the devices is always current.
After an IoT Hub is created, an IoT Hub Provisioning Service is set up using the Azure Portal by providing a name for the Device Provisioning Service, choosing the subscription you want to assign the Provisioning Service to, creating or assigning a resource group to the new Provisioning Service, and selecting a location close to your device. The device manufacturer adds device registration information to the enrollment list.
A series of automated steps follow to establish the connection between the device and the IoT Hub and begin provisioning. The device first contacts the Provisioning Service endpoint that was set at the factory and passes its identification information to prove its identity. The Provisioning Service validates the device’s registration ID and key against the enrollment list either using a Trusted Platform Module (TPM) nonce challenge or X.509 for verification. The Provisioning Service then registers the device with an IoT Hub and populates the device’s desired twin state. After the IoT Hub returns device ID information to the Provisioning Service, the Provisioning Service returns IoT Hub connection information to the device and the device connects to the IoT Hub using one of the supported cloud protocols. The device then gets the desired state information from its device twin in the IoT Hub.
Lowest Latency. Devices are provisioned to the IoT Hub with the lowest latency to the device.
Even Weighted Distribution. Linked IoT Hubs are equally likely to have devices provisioned to them (by default).
Static Configuration using the Enrollment List. Specification of the desired IoT Hub in the enrollment list takes priority.
The Device Provisioning Service would then be linked to the second IoT Hub.
IoT Hub Availability and Disaster Recovery
Within a region, the IoT Hub service provides high availability through redundancies in nearly all service layers. Microsoft’s Service Level Agreement (SLA) for the Azure IoT Hub is 99.9 percent availability during which the Hub can send messages to and from registered devices. During this time, the service can perform create, read, update, and delete operations on the IoT Hubs. Similarly, Microsoft states that 99.9 percent of the time, provisioning service will be able to receive provisioning requests from devices and register them to an IoT Hub.
Fixing dropped network connections
Switching between network connections
Reconnecting after transient connection errors
To reach a higher level of availability, a disaster recovery plan can be put into place to account for the extremely rare instance in which a data center becomes unavailable. To implement a regional failover model, you must have a secondary IoT Hub and device routing logic in place. The secondary IoT Hub must contain all device identities through replication from the primary IoT Hub. When the primary region becomes available again, all state and data created at the secondary site must be migrated back to the primary site.
In this scenario, recovery point objectives (RPOs) are 0 to 5 minutes of data loss for identity registry, device twin data, cloud-to-device messages, and parent and device jobs. All unread messages are lost for device-to-cloud messages, operations monitoring messages, and cloud-to-device feedback messages. The recovery time objective (RTO) where manual failover is put into place using the Azure Portal ranges from 10 minutes to 2 hours. For Microsoft-initiated failovers, RTO ranges from 2 to 26 hours.
Once you’ve set up your IoT Hub(s) and start receiving data from your devices, you are likely ready to analyze the data that is being gathered. That is the subject of the next chapter.