Creating a Database

The first step in building a query-based rule engine is to define the data schema and add a data table in our time-series data storage. Let’s create this rule table, as shown in Figure 10-1.

We are naming it the ruleEngine table . In this data table, we have a primary key and an autoincrementing value, id. This is required to refer to any rule going forward. ruleName is the readable name of the rule. The active column is binary in nature and defines whether the rule is active or not. There are two columns that define a rule’s logic: topicPattern and payloadPattern . These columns hold SQL-like patterns for rule qualifications; based on their qualifications, we can use the webHook value to call on using a defined method in the method column. By default, method is set to GET request.

There could be several possibilities in which we can use this structure or augment it. We can certainly add a few more columns that can hold various other actions to be executed, values to be changed, scripts to be executed, or other global variables to be affected. We can also do all that in another web service or script file located at the webHook URL. It is cleaner this way.

Once this table is created, a typical entry looks like the one shown in Figure 10-2.

Let’s look at what we have added to the table. At the outset, we are calling/naming this rule timestamp rule , and the same is appearing in the ruleName column. The next field, active, is set to 1, which means the rule is active. If we want to disable this rule, we simply set active to 0. The topicPattern field has a value of timestamp%. The payloadPattern field is %, which is a pattern similar to what you would use for SQL queries. From the rules’ perspective, this means that the rule should execute if the received topic matches this pattern (timestamp%) (i.e., the topic starts with the word timestamp) and can have anything after that. For the payload, the pattern is set to % (i.e., anything is acceptable in the payload). The last two fields define the webHook information : the method field defines the type of call, and the webHook field has an actual URL to be invoked.

Building the Flow Sequence

With this sample entry ready, let’s create a sequence for in-flow rule execution. Figure 10-3 shows the created sequence.

The preceding snippet seems slightly unusual because it does not have return msg; code in it. The node first checks the length of the payload and executes a piece of code only if that length is non-zero. If there is no rule matching to the criterion, then the payload is empty, and thus we avoid going further (because there is no return statement after the if clause). However, if there is some payload, it has an array of rule objects. If there are multiple rules matching the rule condition, then there is more than one.

With the for loop, we ensure that all the rules are executed in sequence (i.e., the rule that was first created is executed first). By default, SQL results are in ascending order, which ensures that our rules are executed in the order of creation—the lowest ID executes first.

When we have a rule object, we assign an HTTP calling method and URL to it to be passed on to the HTTP request node. Then we send this packet through using a node.send statement. It forms an input to the HTTP request node, which has the settings shown in Figure 10-4.

This means that the HTTP request node will execute an HTTP call and return the output as a parsed JSON object, which we are simply sending to the debug output.

At this stage, we have our rule engine flow ready, along with one rule added to the rule table. In Figure 10-5, the webhook that we added is POST ( www.in24hrs.xyz:1880/modifiedTime/rule-engine-works ). This is essentially our own data publish API call, which means that when a rule is executed, the /pub API is called and it publishes another message under the modifiedTime topic and the rule-engine-works payload.

Testing the Rule Engine

We can test this in several ways, but the best way is to check it with the Paho MQTT utility because you are able to see the action live (i.e., when you publish a timestamp message, it is picked up and the rule engine searches for a rule). Since we already have a matching rule available, it will be executed and another message will be published. While you are on the Paho page, you see this second message coming in live, in almost no time.

To see how powerful our rule engine is, update the ruleEngine table with additional rules, as shown in Figure 10-5.

Here we have two rules matching the criterion (i.e., rule id 1 and id 2), whereas when we execute rule 2, it publishes a message, rule-engine-working-again . We have configured the third rule not to check topicPattern but payloadPattern for messages that end with the word again. This means that our second rule triggers the third rule.

Check this again in the Paho utility. You should see upon publishing something on the timestamp topic; there should be three additional messages following that.

Enable and Disable a Specific Rule

The first API flow sequence is followed by functional code, as shown in Figure 10-6.

Now if you check the database, the rule has an active=0 value.

Enable and Disable All Rules

The second API that we will create enables or disables all the rules at once. This is a straightforward creation with a minor difference: the query will not check for any id value, so it applies to all the rules. The flow sequence for this API is shown in Figure 10-7.

The preceding code snippet that enables all API query code is self-explanatory. The code written for prepare response is the same as the earlier API’s.

Create a New Rule

Now we will create the third API to add a new rule. As mentioned earlier, we can use this API to create new rules and to have applications register a callback on the go. The flow sequence is shown in Figure 10-8.

First, we are creating a new rule called testRule , which is triggered if the topic is like %stamp (which could be timestamp or anything else) and the payload has four numbers in sequence (e.g., %1234%). When this condition is met, we are setting it to send us an SMS using our SMS API. Before you add this rule, make sure that you have inserted a valid mobile number in place of the dummy one.

Upon successful creation, we get the output and its ID (which is 4 in this case.) Remember that this rule is still inactive, so we use our enable API and enable it. Once our rule is active, head over to the Paho MQTT utility and publish something on the timestamp topic. Make sure that the payload has the number sequence 1234 in it (e.g., payload = 1544312340320). If everything is set up correctly thus far, the specified mobile number will receive an SMS that says, “Pattern detected.”

We can also create an additional API to delete the rule from ruleEngine. It is not explained or demonstrated here; however, you can follow the same logic as in the /purge API, and create it yourself.

What Are Our Options?

In our implementation, we are using authentication only for data access APIs because that is the critical piece of currency our platform may hold. When you implement your own platform, you can take the lead from this implementation and extend it to all other APIs as you see fit.

What Is the Plan?

To implement a simplified API, key-based authentication, and access control, we will make use of another data table in the time-series database, which can hold API keys and relevant information for our use. We then ensure that every time an API call is received for a data request, we check a supplied key and assert access controls via modification of our base queries. In previous chapters, we provisioned most of the APIs for this usage in the form of authFilter. The minimally required data schema for authTable is shown in Figure 10-11.

This is a somewhat unconventional way to use the API key, but it is perfectly legal from a programming and coding perspective.

Adding Authentication Middleware

Now open the settings.js file and search for the httpNodeMiddleware section. As the comments for this section state, this property can be used to add a custom middleware function in front of all HTTP in nodes. This allows custom authentication to be applied to all HTTP in nodes, or any other sort of common request processing.

In the preceding code snippet, the middleware function does three main tasks.

First, it defines a user-defined function, which uses the MySQL library to connect with our time-series database and execute the supplied query. When query output is available or failed, the callback function is called with data and the status is passed to that function.

Second, middleware checks for an authorization header in every HTTP API request. If the access token is available, then it is captured in another variable and availability is set to true or false.

Once the token is captured, middleware checks for an access string in authTable , which is defined for the given token. This token, if available, is assigned to the authFilter variable, which we are using later for SQL query building. If no token was supplied, the filter will be set to '0', which will yield zero records upon query.

Enable and Test Authentication

Update settings.js appropriately, save it, and then restart Node-RED. Since Node-RED might have been already running with the forever utility, it is easy to restart it. Simply check for the PID of this process with the forever list command, and then send another command: forever restart <PID>. This restarts your Node-RED instance, and it reloads the updated settings.js file with authentication middleware code.

It is functioning as expected, and we are getting data only when we send the Bearer token . Now try adding a few more users with different tokens and change their topic access with simple SQL syntax. If you wish to provide access to multiple topics, use the OR operator to build the authFilter condition.