The publish/subscribe communication pattern involves one event producer and several consumers. This pattern is also often named the observer design pattern. This pattern consists of a unidirectional stream of communication, where one component is the source of events (the producer), and one, or several, components are sinks of these events (the consumers). The following figure shows how this pattern works:

The principle of this pattern is to decouple the producer from the consumers. The producer emits events with no knowledge of the existing consumers. The consumers register events to a producer, so that they receive events whenever they are available. Usually, an entity which lies between the producers and consumers allows us to provide this decoupling between both parts. This is the square in the middle of Figure 12.1. This entity can be as simple as a function or a class, or very complex, such as a broker composed of several nodes forming a cluster.

The publish/subscribe pattern is used on all kinds of communication:

• Intra-process, via functions or classes: Intra-process publish/subscribe is implemented via the classical observer design pattern. This pattern is very popular in object-oriented programming. It is heavily used in GUI environments because it is event-driven. All programming environments provide APIs to use this pattern. For example, Qt and GObject implement APIs for this pattern.
• Inter-process, via IPC: Inter-process publish/subscribe is also widely available. The implementation in this case relies on a message bus, with a central broker process where all publishers and subscribers register. The role of the broker is to route messages to their recipients. D-Bus, the IPC foundation of almost all Linux distributions, is a typical example of such an implementation.
• On a network link: Publish/subscribe over a network link is also very common. More than that, it is more common because usually it provides better performance and less latency than the request/response pattern. Most of the time, the network implementation relies on a broker. In these cases, the broker is a cluster. Distributing the load on several nodes allows us to process messages even if a node fails, and to scale the messaging system horizontally. Typical implementations are Kafka, MQTT, and ZeroMQ.

As already explained in Chapter 1, An Introduction to Reactive Programming, ReactiveX is an implementation of publish/subscribe (the observer pattern done right) for intra-process communication. Because there are many implementations of this pattern for remote communication, the observable abstraction can also be used over network links. There are two ways to do this:

• The first one is by implementing a driver on top of an existing technology. For example, in an environment where Kafka is used, then one just has to wrap a Kafka library into a driver to nicely integrate a ReactiveX application with Kafka streams.
• The second one, which is the subject of this chapter, consists of implementing this broker on top of a persistent connection, by leveraging the features of ReactiveX. On one side, this requires writing more code. But on the other side, existing brokers may not be available or suitable in the operating environment.

Let's look again at the observable contract to get an idea of how an observable can be represented on a network layer, that is, via messages being sent on a network link. The observable contract requires that the observable receives notifications from the observer via one of the following events: OnNext, OnCompleted, and OnError. The observable contract also requires that the observer communicate with the observables via two events: subscribe and unsubscribe.

These five events are the only events needed to implement an observable emitting items to an observer, with both entities being on two systems. The following figure shows a sequence diagram of a typical observable life cycle on a network link:

The principle is straightforward, but there are some key points. There are eight entities involved in this diagram:

• The TCP Server and the TCP Client. The network link in this example is a TCP connection. The client must first connect to the server before any communication is possible.
• The Muxer is instantiated both on the client side and on the server side. This component is in charge of transforming network messages to API calls, and vice versa. In practice, this entity implements both multiplexing and demultiplexing but is named Muxer for simplicity. This part will be detailed in a following sub-part.

• The Observer proxy and the Observable proxy are wrappers between their respective ReactiveX API and the Muxer.
• The Observer and the Observable are the end parties that communicate with each other.

As shown in Figure 12.2, most of the communication is a direct mapping of the Observer/Observable APIs on messages. However, there are some implications, due to the fact that the Observer and the Observable are on different systems. The first one is the fact that, when an Observable proxy is created, then nothing happens on the peer. The reason is that the creation of an Observable cannot fail with the RxPY API (this is also the case in many other implementations of ReactiveX). So, this request should be forwarded to the peer. There would be no means to notify an error in a similar way other than with a local observable. The first request that goes to the peer is the subscription request. The call to subscribe cannot return errors, but it is possible to notify an issue after this call. If something goes wrong during the subscription on the network side, then the Observable proxy can complete on error.

On the publisher side, when a subscription message is received, then the Observable is created and immediately subscribed. An additional acknowledgement message is sent to notify the subscriber that the subscription succeeded. If any error is raised on the producer side, then a nack is sent to the peer so that the subscriber can stop the subscription. Such an error sequence is shown in the following figure:

The fact that the Observable creation is deferred until subscription has another consequence: any parameter provided to the Observable creation API has to be saved, and provided in the subscription message.

One last thing must be done with remote observables: hot observables should be shared after the observable proxy, that is, on the consumer side. The reason behind this is performance. If an Observable is made hot on the producer side and a client subscribes to it via several observers, then the associated messages will be duplicated as many times as there are observers. On the other hand, if the Observable proxy is made hot and observers subscribe to this hot observable, then the observable items of the producer are sent only once on the network. This is shown in the following figure: