Options

All too often, software developers and architects reach for the tools that they know best. I’m no exception; any code samples that you’ll see in this book are written in .Net. However, exploring other possibilities is not only a useful exercise but also solidifies the requirements in our minds. In each chapter, I’ll make the case for solving the problem without using technology.

It may seem like a strange thing for a book on software architecture; however, all over the world, people are solving problems without technology; in some cases, that’s the best solution. Software design and development costs money; in some cases, it costs a considerable amount of money, and it is not without risk. According to a 2017 report from the Project Management Institute, between 6 and 24% of projects end in failure. These are not only software projects; however, if we accept that as a rough guide, it means that we can reasonably expect around one in ten software projects to fail (source: www.pmi.org/learning/thought-leadership/pulse/pulse-of-the-profession-2017).

In our case, 123 Tickets has an existing system, but let’s imagine that our advice to the client is to remove that system and replace it with a manual process. What would that look like?

Manual Process

First of all, we would need to maintain a list of valid users for the system; we could keep this in an address book. Each time someone wished to be added to the system, we would write their name and address into our address book; the maintenance of this book would represent all or part of somebody’s job.

Secondly, we would keep a list of events; presumably, we’d use something like a yearly diary to do so; each event would be marked in on the day it was to happen. Somebody would then go through every event for the following two or three months and write on a sheet of paper what, and when, these events were.

Our next step would be to order the tickets from the supplier – when they arrive, our ticket count would simply be that somebody would simply count the remaining tickets for each event.

In fact, when we consider this, we realize that the manual process is actually quite neat; maybe this is the right approach. Of course, there’s a minor snag; even during the smallest festival, over 500,000 attempts were made to purchase tickets; however, before we abandon our manual approach, let’s just continue this thought experiment for another few paragraphs.

Let’s say that we did need to implement this manually and we had a single operator. What would happen if 500,000 or more people tried to phone in to buy tickets at the same time? Well, the way most basic phone systems work is that the first person would be connected, and until that sale had finished, everyone would get an engaged tone.

So how could we structure this so that, given enough time, we could deal with all these requests? One possible solution may be to divert the calls to an answering machine service (for the purpose of this, we’ll assume that the answering machine can take multiple calls at any one time without the caller getting an engaged tone), asking the customer to leave details of the venue and ticket requirements; the operator could then phone each person back as they became free.

Our manual system does have an issue; let’s say that our operator is very efficient and can process each call in five minutes; if that were the case, it would take this person around 42,000 hours to process 500,000 tickets. That’s around 22 years (based on a 35-hour week)!

How would we solve that? In fact, you’re probably thinking that the solution is very obvious: employ more people. If one person would take 22 years, it follows that 2 would take 11. If we had 1000 people, we’d clear the backlog in just over a week. While this may seem obvious, it’s easy to forget this knowledge when we start looking at automated solutions.

This is not, however, a new system; the client already has a system in and running, so let’s investigate what their existing system looks like.

Existing System

123 Tickets has an existing system. Figure 1-2 shows the architecture for their existing system.

While we seem close, the existing system has a module that manages seat allocation, but we’ve identified that this should probably fit under the ordering process. Further, it appears that we need a ticket inventory; it’s very likely that the existing system does maintain a ticket inventory – perhaps it’s inside the Booking Module.

Existing System Considerations

When working with an existing system, there’s a balance to be struck; often, if a system is already in place and working correctly, this is a much better solution than a system that doesn’t exist yet.

Let me clarify what I’m saying here and what I’m not saying. Each system that’s in place has a value; that value is comprised of factors, such as the cost of the system to develop, the value that system provides while it’s running, the cost of supporting and maintaining the system, and the cost of replacing the system. The latter is a very important figure and is not measured in just pounds and pence.

Replacing a system involves disruption; there’s the disruption caused by actually replacing the system, but also, you’ll need business domain knowledge, which means you’ll be removing people from their day-to-day jobs; the new system is unlikely to look and feel exactly like the old, and so there will be a period when people get used to the new system. There’s a risk associated with both creating the new system and implementing it; the risk will vary and can be mitigated, but it will never be zero.

This does not mean that if we have a system that works poorly, we should design around it for fear of breaking anything; however, after we establish a goal, a target architecture, it may influence how, or even whether, we choose to reach that target completely.

Something else that we should come back to at this stage is what constitutes our core system.

Minimum Viable Product

Earlier, we asked what features, about the ticket ordering system, made this a ticket ordering system; in fact, seat allocation may be one feature that is not required in this system. This is the concept of a Minimum Viable Product, or MVP, and it applies equally to replacing or upgrading a system, as to creating a new one; without defining the scope of the system, you risk any project spiralling out of control, both in terms of cost and complexity.

We should be careful how we go about defining an MVP. Clients and developers are not the only members of the team that are capable of causing scope creep. Let’s assume that your team has a product manager, and they decide, after speaking to the client, that the system needs an ice cream ordering module; without a documented MVP, it’s quite easy for a new user story to get created. In this case, everyone on the project would make an assumption that this functionality did form part of the MVP.

Alternatively, imagine that a system tester finds an issue with the system, along the lines of “The system’s ice cream ordering facility is missing or nonfunctional.” Again, this may inadvertently result in scope creep.

These are, clearly, contrived examples; and I’m not singling out the roles mentioned – pretty much anyone on, or off, the project could inadvertently instigate scope creep. I’m simply making the point that having determined what our MVP is, we should maintain that document and that everyone in the team should bear it in mind when considering changes, defects, enhancements, or requests.

That said, let’s move on to discussing what our target architecture might be.

How to Deal with High Throughput

Imagine that we have a funnel and we’re trying to pour water into the funnel (Figure 1-3). Instinctively, you’ll realize that there is a limit to the rate at which we can pour water into that funnel, above which, the funnel will fill up and eventually, the water will spill out of the top.

If we go back to our little thought experiment around how we would do this manually, we see that we have similar choices in our manual system: we can either ask our telephonist to work faster, or we can increase our staff.

Widening the Funnel

Widening the funnel in the case of a web server means essentially one of two things: increase the physical capacity of the server, or increase the capacity of the service.

Multiple Funnels

If, instead of improving our single funnel, we can increase the funnel count (Figure 1-4), we can potentially accept twice, three times, or twenty times the input data. Of course, we’re not proposing to have 20 databases, so what we’re really doing is moving the problem. Again, this approach is not free, but before we discuss the cost, let’s discuss what this might look like using our funnel example.

To some extent, the analogy breaks down a little at this point; however, this does serve to illustrate broadly what we’re doing. Essentially the trough stores the water as it comes in and releases it at a pace that the funnels can cope with, meaning that each funnel is no longer overloaded as the water comes in.

I think we’ve taken this analogy as far as it is sensible, so let’s think back again to our manual process. We thought that we could increase our throughput by increasing the number of people working through the bookings. We had, in fact, devised a system very similar to this; the answering machine was essentially like the trough above – as calls come in, they are stored on the answering machine. Our telephonists then process the calls at a rate that they can cope with. Here, we have a queue of messages, and that’s the approach that we’re going to use for our architecture – a message queue.

Message Queues

Let’s quickly review exactly what a message queue is, in a way that involves the technology itself – without an answering machine or a funnel in sight.

The principle behind this is simple: a message is sent to the queue, and one or more clients can retrieve and process that message. In practice, there is more to it than that; however, you could (and many people do) implement a message queue as, for example, a table in a database, or a series of files on a disk.

At the time, we didn’t think of this as a message queue, but that is essentially what it was. With a very small tweak, we could have introduced multiple consumers (by simply moving the file to a second directory before processing).

Let’s say that this information represents:

Product Code, Order Quantity, Unit Price, Sales Date

The EDI system is expecting four fields, but seeing five. In this case, the EDI system itself would typically crash, and every subsequent message would start to queue up; even though the subsequent messages may be fine, our system is down.

I imagine you’re thinking of several ways that this issue could be alleviated: the file could simply be ignored or skipped if there was an error, or some kind of error handling process could move the file out of the way.

What we needed here was a message broker.

Message Brokers

A message broker is a piece of software that is agnostic of the message itself; the message broker has a number of protocols that it can communicate over, but its real value is it acts as an intermediary between the sender and the consumer.

In our EDI case before, the message broker would essentially replace the file system; however, message brokers are pieces of software in their own right. Rather than just communicating via an agreed location on a file share, you communicate via the message broker: send messages to the broker, and take messages from it. In the example before, where the message cannot be read, message brokers provide functionality to move the message out of the way, while others are processed.

Typically, message brokers provide a lot of additional functionality, most of which is beyond the scope of this book. However, since we are talking about using queues, it’s worth understanding a little about the infrastructure and setup that may be involved in using a queue.

As we said earlier, there’s no free lunch here. The message broker provides a great deal of functionality for us; it maintains state, handles failures, takes care of the routing between endpoints (so no one needs to write to a disk share but rather sends a message to an endpoint), and handles transactions; however, as a result, you’re no longer simply writing files to a disk on a share, so you sacrifice some performance. In reality, with modern message brokers, using clever caching techniques, the hit is probably negligible (in fact, I imagine if you compared our EDI system back then to any modern message broker, the message broker would be far more performant) – but it’s worth bearing in mind that you are introducing processing and complexity into the system.

I’d also like to clarify a point about message brokers; there are dozens of them out there: RabbitMQ, ActiveMQ, ZeroMQ, Apache Kafka, GCP Pub/Sub, Amazon SQS, Microsoft Azure Service Bus, and the list goes on and on. They all have implementation quirks, and they all have advantages and disadvantages. It’s probably not even fair to call them all “message brokers”; but my point here is that I’m talking about generics in a world of specifics. I will be covering a specific implementation later in this chapter; but it’s very likely that you can pick any message broker and substitute it for the one that we use here.

Now that we’ve discussed what queues are, and why they can help in this specific scenario, and spoken about such things as message brokers, let’s briefly discuss other advantages that using a queue can provide.

Separation of Concerns

The reason that most people choose to use a queue initially is the same as ours: they have a large volume of data, and they want their system to be able to cope with sudden fluctuations in traffic. However, using a message bus provides other benefits. One such benefit is separation of concerns; consider Figure 1-7.

Process A can perform some task and put a message onto a bus. Process B can read the message from that bus and perform a task. However, Process A has absolutely no dependency or even knowledge that Process B exists and vice versa. In fact, we could replace Process A with another process, or several processes, without affecting Process B in the slightest.

The second advantage is resilience. Let’s imagine that Process B fails: if Process A were calling Process B, then that would cause both processes to fail; however, since all Process A is doing is writing to a queue, the messages will simply remain in the queue until such time as they are picked up.

Let’s take a real-world and very common use of a queue: email. Process A in this case is a website that processes a sales order, and Process B sends confirmation emails out to the customer. If the website were to directly call a process to send emails and that failed, then either your entire website would be down or the email confirmations would be lost. However, in the aforementioned situation, the emails are simply left in the queue.

There are many other advantages to using message queues and using a message bus in general (there are messaging patterns other than queues), and I would encourage you to investigate this paradigm further; however, an exhaustive explanation is outside the scope of this book.

Now that we’ve covered our use of a message bus, let’s see our proposed target architecture.

Target Architecture Diagram

Let’s move on to a proposed architecture and discuss the advantages and disadvantages.

What we can see in Figure 1-8 is that while the client can access most of the system directly, in order to book a ticket, it enters into a queue. This acts as a buffer between user demand and the capacity of the system. Further, once the ticket request has been added to the queue, we can have one or multiple processes that pick up the request and process it. In our case, this is our new service (entitled “Ticket Ordering Process” in the diagram). If we think back to our manual process, this is our army of staff, listening to answering machine messages one at a time.

You’ll notice that there is a second queue going back to the client; this is to allow a response from the process. That is, the service can reply to the client to indicate the result of the operation.

Finally, we’ve introduced a proxy between the rest of the system and the external API. Let’s spend a little time discussing why we might wish to do this.

Proxy

The Proxy Pattern was one of the patterns introduced in the Gang of Four book Design Patterns: Elements of Reusable Object-Oriented Software. It is very often used in this type of scenario in order to insulate your system against change; since you have no control over the third-party API, you introduce a buffer between your system and external systems which you have no control over. The advantage here is that should the owner of the third-party system decide to change their API, you have only to change your proxy and not your main system.

You may already practice this kind of architectural decision without realizing it; if you’ve ever used an interface in place of a concrete class, you’re probably doing it so that your system can be insulated against change in that class; the change that you’re insulating against might be that in your unit test, you mock the class out, but that is a change in the implementation of that class (it’s changed from providing functionality to providing no functionality).

I often like to think of real-world examples for such things; in this case, a car is an excellent example of a real-world proxy. Depending on where you live in the world, you’re likely to need to undergo some form of test in order to legally drive a car; in that test, you’ll be ascertained on your ability to move the car – for example, you’ll need to be familiar with what the pedals do and how the steering wheel functions. However, you wouldn’t expect any of this to change if you got out of a Ford and into a Volkswagen. Essentially, you’ve learned to interact with a proxy; the steering wheel may technically function differently in these two makes of car, but the public interface (turn the wheel left or right) remains consistent.

Now that we’ve explored the theory, let’s see what this might look like in a concrete example.

Before we get into the example, let’s talk a little about our choice of message broker. I’m going to use a cloud message queue service: Azure Service Bus. The choice of using a cloud service is the real architectural decision here, not the specific vendor; all of the cloud providers have relatively comparable services, and which one you choose will depend on a number of factors – few of which would fall into the category of an architectural choice.

A Note on Cloud Vendors

Certain vendors have variations in services, some of which may better suit your needs; for example, GCP (at the time of writing) doesn’t provide a FIFO message bus service – so if that were important, you may be forced to choose another vendor.

My suggestion would be unless you have a very specific need for a particular application, then choose a provider based on the familiarity of the team with that technology. If everyone in your team knows AWS, then pick that.

The caveat here is market availability for people familiar with that technology; a lot of people are familiar with Azure or AWS, and recruiting someone to work with those technologies is relatively easy; however, if you pick some more obscure system, you may find it difficult to find people to work on it. This should not be underestimated; however good a technology may be, you need people to work on it, and you need people to want to work on it. If you architect a system that uses a very old or very specific technology, it doesn’t matter how good the system design is because either you won’t be able to get people to work on it or you’ll have to pay a huge premium for them to do so.

I would also advise against an all-in or all-out approach. What I mean by that is that you should try to keep any interaction with your cloud provider abstracted to a sensible level so that if you were to want to move from one provider to another, that would be possible without much work. There are times when this will be difficult, but remember that if your business depends on, say, GCP functions and Google decides tomorrow (or in ten years) that they are moving away from them, you are unlikely to have a say in that decision.

This may sound a lot like I’m heavily advocating the use of containers, and I’m not necessarily. If you have the experience and time to manage a Kubernetes cluster, then maybe that is the right approach, but if you don’t, then be very careful about taking on an infrastructure overhead.

Finally, and somewhat at odds with what I’ve previously stated, it’s probably an idea to not distribute your system across too many cloud providers. Although they are all broadly the same, this means that you have multiple places to go and investigate when something goes wrong.

In summary, my advice is to follow the same principles that you would, were the cloud provider a database engine. Good practice is to abstract interaction with the database where feasible in case you needed to change the database engine; but you would be unlikely to interact with both SQL Server and Oracle because the maintenance overhead would outweigh any benefits they might provide.

Obviously, this is an opinion, and it’s subject to change; as Kubernetes matures, or other technologies emerge, this may become a solved problem.

Why Cloud?

One of the driving requirements in this case is the ability to cope with a traffic spike. Don’t misunderstand what I’m saying here – you absolutely can create a system to handle spikes in traffic that works on-premis.

However, what the cloud (any cloud) offers, is a cost benefit. You are not responsible for maintaining the infrastructure, and you only pay for what you use. If you were to build this up locally, you would have to provision a server that could cope with the highest spike in traffic, even though for most of the time, you could probably run the system fine on a far less powerful machine.

If you had multiple jobs that all had different spikes in traffic and you had a massive scale, it might make sense to consider a private cloud; this is the last I’ll say in this book about private cloud – while it does have its place, its use cases are very specific; and certainly none of the chapters in this book lend themselves to it.

Now that we’ve briefly discussed our choices, let’s delve into some actual code. We can talk about some other decisions and principles as we go through.

External APIs

The first thing we’ll need is something to simulate the third-party systems that we’ll be interfacing with. The code for this can be found in the GitHub repo associated with this book:

https://github.com/Apress/software-architecture-by-example

You can simply clone this repo and run the API; however, if you wish to follow along and create this API yourself, instructions can be found in Appendix A – Chapter 1.

Assuming that you have either cloned the repo or have followed the instructions, then the endpoints for our dummy third-party system will be as follows:

https://localhost:5001/externalticketbooking/gettickets

https://localhost:5001/externalticketbooking/reserveticket

https://localhost:5001/externalticketbooking/purchaseticket

Now that we’ve covered the third-party API, let’s discuss the two main slices of our system: reading the ticket availability and ordering a ticket.

Getting Ticket Availability

The call to get ticket availability would probably not change between the existing and the new system; however, we have decided to introduce a proxy between our API and the third-party one.

The idea behind an API is to act as a public interface to your system, so if it’s designed well (i.e., generically), then you should be able to change the functionality and structure behind the API, without affecting the interface.

If we follow Figure 1-8, we’ll see that the interface between the external APIs and our system, in our revised version, is dealt with by a proxy and no longer by the API itself.

Let’s drill into a few specific points on retrieving ticket information. In this instance, we will be calling the API directly, rather than via a service. Let’s have a look at our implementation for this method in the API.

Why haven’t we used the queue here as well as in the call to order a ticket? Well, you certainly could do that; however, it does add an overhead to the call. It’s also worth considering what benefits that would give. Let’s imagine that the system is hugely overloaded, and we call the API to get ticket availability; because the system is so busy, the call times out. Now let’s imagine the same scenario, but this time we’ve added a message to a queue; the call would immediately return, and when the system was successfully able to return the data, it would. However, it’s possible that since the call was made, the data that’s returned is now out of date.

We’ll talk more about the proxy and some of the decisions made in this code later. The one thing that I would like to say here is that we are deserializing the JSON data that is returned. I’ve done this to make the code simpler; however, if you are dealing with an API that you expect to change, then consider manually parsing the JSON. This can be done using a strategy such as XPath – you don’t need to rewrite JSON.NET.

While manually parsing the data does represent more work, it should make your system more stable; deserialization is very dependent on the shape of the data being returned. Obviously, if the third-party API was returning a field called Price and they change it to a field called Cost, there’s pretty much nothing you can do – hence the use of a proxy.

Ordering a Ticket

Ordering a ticket is the big change to the system. The previous system was overwhelmed by the quantity of traffic, and so here, we’ve introduced a buffer (in the form of a queue) between the request and the API. We’ve already spoken about what a queue is and even how a queue can help with this specific scenario, so let’s see what using the queue looks like in practice.