If scalability is about splitting, systems integration is about rejoining. In the previous chapter, we learned how to distribute an apfunction is a simple iteration over all the connected clientsplication, fragmenting it across several machines. In order for it to work properly, all those pieces have to communicate in some way and, hence, they have to be integrated.
There are two main techniques to integrate a distributed application: one is to use a shared storage as a central coordinator and keeper of all the information, the other one is to use messages to disseminate data, events, and commands across the nodes of the system. This last option is what really makes the difference when scaling distributed systems, and it's also what makes this topic so fascinating and sometimes complex.
Messages are used in every layer of a software system. We exchange messages to communicate on the Internet, we can use messages to send information to other processes using pipes, we can use messages within an application as an alternative to direct function invocation (command pattern), and also device drivers use messages to communicate with the hardware. Any discrete and structured data that is used as a way to exchange information between components and systems can be seen as a message. However, when dealing with distributed architectures, the term messaging system is used to describe a specific class of solutions, patterns, and architectures that are meant to facilitate the exchange of information over the network.
As we will see, there are several traits that characterize these types of systems. We might choose to use a broker versus a peer-to-peer structure, we might use a request/reply or one-way communication, or we might use queues to deliver our messages more reliably; the scope of the topic is really broad. The book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf gives you an idea about the vastness of the topic. It is considered the Bible of messaging and integration patterns and has more than 700 pages describing 65 different integration patterns. This chapter explores the most important of those well-known patterns, considering them from the perspective of Node.js and its ecosystem.
To sum up, in this chapter, we will learn about the following topics:
- The fundamentals of a messaging system
- The publish/subscribe pattern
- Pipelines and task distribution patterns
- Request/reply patterns
When talking about messages and messaging systems, there are four fundamental elements to take into consideration; these are as follows:
- The direction of the communication, which can be one-way only or a request/reply exchange
- The purpose of the message, which also determines its content
- The timing of the message, which can be sent and received immediately or at a later time (asynchronously)
- The delivery of the message, which can happen directly or via a broker
In the sections that follow, we are going to formalize these aspects in order to provide a base for our later discussions.
The most fundamental aspect in a messaging system is the direction of the communication, which often also determines its semantics.
The most simple communication pattern is when the message is pushed one-way from a source to a destination; this is a trivial situation, and it doesn't need much explanation:
A typical example of one-way communication is an e-mail or a web server that sends a message to a connected browser using WebSockets, or a system that distributes tasks to a set of workers.
The request/reply pattern is, however, far more popular than the one-way only communication; a typical example is the invocation of a web service. The following figure shows this simple and well-known scenario:
The request/reply pattern might seem a trivial pattern to implement; however, we will see that it becomes more complicated when the communication is asynchronous or involves multiple nodes. Take a look at the example in the following figure:
With the setup shown in the preceding diagram, we can appreciate the complexity of some request/reply patterns. If we consider the direction of the communication between any two nodes, we can surely say that it is one-way. However, from a global point of view, the initiator sends a request and in turn receives an associated response, even if from a different node. In these situations, what really differentiates a request/reply pattern from a bare one-way loop is the relationship between the request and the reply, which is kept in the initiator. The reply is usually handled in the same context of the request.
A message is essentially a means to connect different software components and there are different reasons for doing so: it might be because we want to obtain some information held by another system or a component, to execute operations remotely, or to notify some peers that something has just happened. The message content will also vary depending on the reason for the communication. In general, we can identify three types of messages, depending on their purpose:
- Command Message
- Event Message
- Document Message
The Command Message is already familiar to us; it's essentially a serialized command object as we described it in Chapter 6, Design Patterns. The purpose of this type of message is to trigger the execution of an action or a task on the receiver. For this to be possible, our message has to contain the essential information to run the task, which is usually the name of the operation and a list of arguments to provide when it's executed. The Command Message can be used to implement Remote Procedure Call (RPC) systems, distributed computations, or more simply used to request some data. RESTful HTTP calls are simple examples of commands; each HTTP verb has a specific meaning and is associated with a precise operation: GET, to retrieve the resource; POST, to create a new one; PUT, to update it; and DELETE, to destroy it.
An Event Message is used to notify another component that something has occurred. It usually contains the type of the event and sometimes also some details such as the context, the subject, or the actor involved. In web development, we are using an Event Message in the browser when using long-polling or WebSockets to receive notifications from the server that something has just happened, as, for example, changes in the data or, in general, the state of the system. The use of events is a very important integration mechanism in distributed applications, as it enables us to keep all the nodes of the system on the same page.
The Document Message is primarily meant to transfer data between components and machines. The main characteristic that differentiates a document from a command (which might also contain data) is that the message does not contain any information that tells the receiver what to do with the data. On the other hand, the main difference from an Event Message is mainly the absence of an association with a particular occurrence, with something that happened. Often, the replies to the Command Messages are Document Messages, as they usually contain only the data that was requested or the result of an operation.
As Node.js developers, we should already know the advantages of executing asynchronous operations. For messaging and communications, it's the same story.
We can compare synchronous communication to a phone call: the two peers must be connected to the same channel at the same time and they should exchange messages in real time. Normally, if we want to call someone else, we either need another phone or to close the ongoing communication in order to start a new one.
Asynchronous communication is similar to an SMS: it doesn't require the recipient to be connected to the network the moment we send it, we might receive a response immediately or after an unknown delay, or we might not receive a response at all. We might send multiple SMSes to multiple recipients one after the other, and receive their responses (if any) in any order. In short, we have a better parallelism with the use of fewer resources.
Another important advantage of asynchronous communications is that the messages can be stored and then delivered as soon as possible or at a later time. This might be useful when the receiver is too busy to handle new messages or when we want to guarantee delivery. In messaging systems, this is made possible using a message queue, a component that mediates the communication between the sender and the receiver, storing any message before it gets delivered to its destination, as shown in the following figure:
If for any reason the receiver crashes, disconnects from the network, or experiences a slowdown, the messages are accumulated in the queue and dispatched as soon as the receiver comes online and is fully working. The queue can be located in the sender, or split between the sender and receiver, or live in a dedicated external system acting as middleware for the communication.
Messages can be delivered directly to the receiver, in a peer-to-peer fashion or through a centralized intermediary system called a message broker. The main role of the broker is to decouple the receiver of the message from the sender. The following figure shows the architectural difference between the two approaches:
In a peer-to-peer architecture, every node is directly responsible for the delivery of the message to the receiver. This implies that the nodes have to know the address and port of the receiver and they have to agree on a protocol and message format. The broker eliminates these complexities from the equation: each node can be totally independent and can communicate with an undefined number of peers without directly knowing their details. A broker can also act as a bridge between the different communication protocols, for example, the popular RabbitMQ broker (http://www.rabbitmq.com) supports Advanced Message Queuing Protocol (AMQP), Message Queue Telemetry Transport (MQTT), and Simple/Streaming Text Orientated Messaging Protocol (STOMP), enabling multiple applications supporting different messaging protocols to interact.
Note
MQTT (http://mqtt.org) is a lightweight messaging protocol, specifically designed for machine-to-machine communications (Internet of Things). AMQP (http://www.amqp.org) is a more complex protocol, which is designed to be an open source alternative to proprietary messaging middleware. STOMP (http://stomp.github.io) is a lightweight text-based protocol, which comes from the HTTP school of design. All three are application layer protocols, and based on TCP/IP.
Besides the decoupling and the interoperability, a broker can offer more advanced features such as persistent queues, routing, message transformations, and monitoring, without mentioning the broad range of messaging patterns that many brokers support out of the box. Of course, nothing stops us from implementing all these features using a peer-to-peer architecture, but unfortunately there is much more effort involved. Nonetheless, there might be different reasons to avoid a broker:
- Removing a single point of failure
- A broker has to be scaled, while in a peer-to-peer architecture we only need to scale the single nodes
- Exchanging messages without intermediaries can greatly reduce the latency of the transmission
If we want to implement a peer-to-peer messaging system, we also have much more flexibility and power, because we are not bound to any particular technology, protocol, or architecture. The popularity of ØMQ (http://zeromq.org), which is a low-level library for building messaging systems, is a great demonstration of the flexibility that we can have by building custom peer-to-peer or hybrid architectures.