Real-time web applications have traditionally been quite complex to develop as they once relied on a series of hacks and workarounds. This is the reason why many developers avoid going real time as they believe it will bring in more complexity to their software backends. To fully appreciate and understand this concept, it's important to know how real-time applications are implemented and how they've evolved over time.
As you may know, web applications are built over the HTTP protocol; however, for quite some time, HTTP and real-time applications didn't really go together very well. The reason for this is that the HTTP protocol wasn't built for this purpose, but it was built with client-to-server communications in mind.
Let's clarify this a bit more. At the heart of this protocol, there is a stateless request-response cycle—the client executes a request and an HTTP server reacts upon these requests, providing a response back to the client, but it doesn't keep a persistent connection to it. In a typical web-based application, the client is usually the browser.
The interaction flow between the browser and the server is illustrated in the following diagram:
As you can see, this flow allows a web application running in the browser to react to client events by sending them to the server and receiving a response, but the reverse is not easy. What if an event occurs on the server? Using the HTTP protocol, the server has no way of actively informing a client about this event in real time.
The first solution to this problem was programming the client to continuously send requests to the server, waiting for events to occur on the server. This kind of interaction is called polling.
Suppose we want to develop a chat room web application that delivers messages to all connected clients. To achieve real-time (or at least near real-time) behavior, each client will have to poll the HTTP server at regular intervals and check for new messages. As you can imagine, this technique is highly inefficient. If we program our web application so that every client polls for new messages every second and there is only one new message every 20 seconds, our server and database will be doing a lot of computations that are absolutely useless and may provide degraded performance due to the increased server load.
As continuously polling the server wasn't a feasible solution, cleverer workarounds were developed; however, these solutions were still based on the request-response flow, only using it in a more efficient way.
One of the most popular technologies is
AJAX, which stands for Asynchronous JavaScript and XML. Actually, AJAX itself isn't a new technology, but rather uses existing technologies to update data on the client side without reloading the page. From a practical point of view, AJAX uses the XMLHttpRequest object to exchange data in an asynchronous manner between the client (browser) and the server, and it uses JavaScript to update information on the client side. This technology became famous when Google started using it in Gmail and Google Maps as it gave the user the illusion that the server was pushing data to the client.
AJAX was a pretty popular technology 5-6 years ago and is still very relevant today as it doesn't block your web application while waiting for a server response. However, the drawback is that AJAX requires you to create a new HTTP connection every time you request new data, such as polling; the client always has to poll the server for data rather than receive it directly from the server. This makes AJAX rather inefficient for complex real-time web applications.
Web browsers have been evolving fast in the past years, so the famous HTML upgrade called HTML5 is being widely adopted by all modern browsers. One of the new features included in the HTML5 specification is a method for pushing data from the server to the client called WebSockets.
We can define WebSockets as a kind of bidirectional, two-way communication channel between the client and the system. This means that using this technology, both the client and server can initiate a request, allowing the server to push data directly to the client (usually, the browser).
From an architectural point of view, WebSockets are implemented as a typical TCP connection between an HTTP server and an HTTP client. The client is always the first endpoint that initializes the connection, sending a special request to the server in order to turn the TCP connection into a WebSockets connection. In technical terms, the client asks the server to "upgrade" the connection. If the server supports WebSockets, it upgrades the connection and now both endpoints, the client and server, can send messages back and forth without any additional overhead, making it a true bidirectional, full-duplex communication channel.
As you may have noticed, WebSockets offer significant advantages with respect to polling and AJAX; for this reason, they're currently the most popular data-transport technology used when building real-time web applications. Now that we know how to send data from the server to the client in real time, let's look at how to build the HTTP server.