In this type of application style, an application is decomposed into a set of micro-services and each service interacts with the other by publishing its results to a common event-bus that delivers the event to the next in the queue.

These kinds of applications are suited to use cases such as Extract-Transform-Load or fetch-process-store. Let's look at an example to understand where such an application styles fits and how it works.

Imagine you own a huge supermarket that contains a variety of food items, both perishable and non-perishable. In order to keep the items fresh, they have to be kept at a constant temperature. For example, a frozen pizza should be maintained at a temperature below zero degrees, otherwise it will start to rot. Similarly, milk and yogurt need to be maintained between 2-5 degrees to keep them fresh.

Being such a huge store, it is not economically feasible to have staff monitoring each aspect of the store and making sure that the temperatures are all within comfortable ranges for each item.

Thus, you decide to invest in temperature sensors that you place strategically around the store. In addition, you collect the readings of the temperature sensors every 10 seconds and send them to a processing engine. This processing engine looks at the current set of readings, compares them with the reading over the last five minutes, aggregates them, and if it finds that the temperature is higher or lower than the configured limit, it sends an alert that is then paged to the maintenance staff with the location of the sensor and its readings.

You can imagine that there will be many sensors sending their information, every 10 seconds, to an underlying processing engine. What the sensor sends is an event containing the basic information about the sensor (name, location, and so on) and the readings of the temperature it has captured in the last 10 seconds.

Message-oriented applications are a great fit for this type of scenario.

The events from the sensors are collected by a Data Collector component (it could be as simple as a Raspberry Pi) and put on a queue. The events are picked up by a simple Normalizer Service that converts the incoming sensor data in a format that is suitable for the underlying processors. Once the normalizer finishes its work, it puts the data back on the queue, which is then picked up by the Decision Service. This service aggregates the data for a given sensor over a configured period of time (five minutes, in our case), compares it with the configured high and low values, and if the value is out of range, creates an Alert event and puts it on the queue. The Alert Event is then picked up by the alert service, which then sends this alert as a pager to the maintenance staff.

As you might have already noticed, every step in the processing chain was decoupled from each other. There was a queue in between each process that acted as a buffer for situations when the publisher of the event is quicker than the consumer of the event. Each response was treated as a message for the next process in the pipeline.

Here is a High-Level view of the message-oriented application style:

A message-oriented application allows event-driven data to be collected, processed, and stored or forwarded to another system.

A Source can either receive the data from the underlying system, typically referred to as a PUSH-based mechanism where the underlying system pushes the data to the Source or the source can proactively reach out to the actual source of the data. This is also referred to as a PULL-based mechanism.

A Processor typically has a very simple interface, where it receives a message from an inbound Queue, processes it, and publishes the response as a message to an outbound queue.

Typically, all Message-based components (Source, Processor, Sink) use the same interface so that they can be composed into a different solution by connecting different sources, processors, and sinks.

Another thing to note is that all the components of a Message-Based Application can be distributed physically across different physical nodes, and these components interact via a distributed Data Bus. This distributed Data Bus could be implemented using any of the many available technologies, such as Redis, Kafka, or RabbitMQ.

Since different components are deployed on different nodes, we would also require a coordination and management service that keeps an eye on each of the running services to make sure they are in good health: