What Works

For the most part, containers deliver good support to the development community during the move to cloud computing. With containers providing a base architecture, they changed the way we think about software development in the cloud as container orchestration and Kubernetes provided mechanisms for scalability and reliability. The encouraging part is that containers were more of a grass-roots movement built on open-source origins. Thus, there were no big-name backers of containers trying to sell them as “their solution.” However, some of the larger brands did jump in feet first to promote container and container orchestration. Those were the smarter players. Others saw no obvious way to make money and ignored containers. Those players ended up doing serious catchup later in the market evolution, mostly by selling containers and container orchestration as public cloud services.

Containers now provide a sound development and deployment technology for the cloud, which in turn promotes the use of cloud-based platforms. Given their impact on the marketplace, we can consider containers a success. Indeed, the “cloud native” movement, as we discuss later in this chapter, was largely built on containers, container orchestration (Kubernetes), and micro services. All are tightly coupled technologies.

So, the core list of what containers make work includes

• A standard development and deployment platform for applications.

• Built-in application portability approaches. While not perfect, containers seem to provide better portability.

• Cost-effective solutions. Containers are usually less expensive than traditional development and deployment technology.

• A large ecosystem of technologies that support containers such as databases, storage, backup and recovery, and security, many of which are now several generations old.

Containers: What Doesn’t Work

The downside of leveraging containers includes issues around their rapidly increasing popularity, and thus the hype that surrounds containers. Like any other technology, containers are not right for all applications. Existing systems need to be carefully analyzed before they get “containerized.” Remember the old adage: Just because you can doesn’t mean you should.

Today the core downside of containers is the overapplication of container development and the overporting of existing applications to containers. The reality is that containers are contraindicated for many applications and application types. However, to date, some businesses saw their way clear to spend twice as much moving an application to containers as they spent to develop the application in the first place. They also chased the benefit of portability when the application was unlikely to ever move from its current host platform. Moreover, they did not understand that to truly take advantage of what containers have to offer means a complete rearchitecture of the application, which they typically didn’t do.

Of course, the mistakes extend to net new applications as well, or new applications that leverage containers as their development approach of choice. Again, many applications won’t find advantages with container-based development. As we cover in the next section, enterprises could spend as much as four times the money to build the same application if container-based development and deployment are used versus more traditional methods. Moreover, the end product could cost more to operate by using more cloud-based resources, such as storage and compute. In many of these situations, containers were force fit due to the decision makers being motivated by the hype surrounding containers, not understanding the business problems, or not caring about the problems, or all three.

So, here’s what doesn’t work (or rarely works) with containers:

• Listening to hype reasons, and not business reasons.

• Overstating the benefits, such as portability.

• Not considering the operational costs.

• Underestimating the amount of time and money needed to move an existing application to a container.

• Not designing an existing application or a net new application as a purpose-built containerized application.

• Being able to forecast future costs based on the short life cycle of a container.

• Failing to account for the increased price of container development, deployment, and operations talent due to its popularity.

Container: Cost Considerations

The obvious key to figuring out what will and won’t work is to determine the cost of leveraging containers. This includes the cost of development, deployment, and operations. Consider these costs against more traditional and commodity-type costs for similar development, deployment, and operations. The gap between these two choices is the “container tax.” It’s the additional money you’ll spend just to leverage containers.

Make sure the benefits outweigh the costs. Containers are a good architectural choice for the right projects. However, consider the benefits you want versus the benefits you need with a clear eye. Are the odds high, medium, or low that your business will use a hyped tech’s costly but nice-to-have features? There should be at least a medium to high probability that the enterprise will see a return on its investment in any technology.

Portability, for example, is something that many believe is a great feature to have in their back pocket because it removes the risk of provider lock-in. Let’s say you leverage a cloud provider using an application that’s tightly coupled to the cloud provider’s services and APIs. If the cloud provider raises prices, or lowers the quality of services, or if the platform is no longer a good fit for other reasons, a tightly coupled application will require additional costs and development time to port the application, and that translates into business risk.

An application that runs within containers and is actually built using containers provides better portability. That puts the application in a better, more portable state that can be easily and cheaply moved to another cloud provider.

However, it’s rarely a clearcut choice. Applications that leverage containers cost about 40 percent more to build. Container development prices are higher, as are the tools and technology prices, and, on average, it just takes more time to build an application using containers compared to more traditional approaches and tooling. Moreover, containers cost about 20 percent more to operate on a public cloud provider. These are costs you hope will be justified due to the lock-in risk they avoid, which should provide the ability to quicky move from cloud to cloud without a great deal of cost. But you must weigh these extra costs against the likelihood of that move ever occurring.

The cost issues with containers are not only around the costs to build them; they’re not that much more. It’s the increased costs to operate them ongoing. That means you need a solid business reason to leverage containers in the first place.

When I’m called into projects after containers are preselected as the development and deployment technology of choice, I usually discover that few if any of the team’s reasons to leverage containers justify their use for either net new or the “containerization” of existing applications. This leads to many frowny faces when I make the unpopular announcement that containers are contraindicated for the project. Unfortunately, in many cases, the teams move forward with containers despite my feedback and just ignore the extra cost of development and operations. Of course, containerized or not, the application works just fine. However, one approach costs the business 20–40 percent more for the same application with ongoing cost overruns. Obviously, that expenditure could shortchange vital developments in other areas of the business.

More often, enterprises evaluate the use of containerization on an application-by-application basis, as it should be. Some applications will cost justify the use of containers, either net new or existing. Many will not. You need to cost justify every architectural or development decision to meet the core objective of supporting the needs of the business. The day we put technology trends over business needs is the day we should look for another job. Next week, next month, or next year, at some point you will be asked to cost justify your decisions. “All the cool kids were choosing containers” is the wrong answer.

Containers: Make the Tough Choices

Containers are an amazing technology and a concept that keeps evolving. It’s an architectural pattern that allows us to compartmentalize applications and data for portability and design simplicity. It comes with the added benefit of providing “cloud scale” capabilities, which means we can scale to any number of users or processing loads and we can perform almost any number of operations.

But containers and container orchestration are not always slam-dunk choices. The business case must justify the need to leverage containers and include how it will benefit the business ongoing.

Although the trends will become a bit clearer as time progresses, containers are or will become a part of all cloud computing architecture. However, we should leverage a technology such as containers only when there is a definable business benefit.

Container Orchestration: What Works

Containers and container orchestrations generally live up to the expectations and hype. They offer cost-effective ways to build highly scalable and reliable applications on cloud or non-cloud platforms, and they provide portability within and between platforms. Vast ecosystems have grown around container orchestration, including security systems, databases, governance systems, operations systems, and other systems needed to meet specific requirements of an application that leverages this technology.

The ability to run containers that leverage redundancy, meaning either running cloned containers within a cluster manager or running more than a single cluster, provides both scalability and resiliency. Developers use containers to define an application that can solve business problems and then leverage container orchestration and clustering to effectively execute the containers and manage most aspects of the execution. Those magical capabilities are why most enterprises now choose container orchestration and clustering to build net-new cloud applications, as well as to redesign and rebuild applications to function on public clouds.

Of course, all this magic comes with trade-offs. Traditional development may still have more value over containers or container orchestration. Figure 5-4 depicts what most enterprises encounter when they adopt containers. The first trade-off is that, as mentioned earlier, containers cost more to build and deploy than traditional development approaches. It doesn’t matter if the application is net new or an application that must be refactored to move to a public cloud platform. There are exceptions, of course, but they are not the rule.

On the magical side of the argument, the longer the containerized applications exist, the more business value they can create. This is often preceded by a dip in value for the containerized applications as enterprises go through an initial learning curve. Eventually, the enterprise will find the value drivers of the containerized applications, such as portability, scalability, and resiliency, and the recognized value inflects as you can see with the value curve container-based development in Figure 5-4. At the same time, traditional development-based applications slowly reduce in value over time, much as they did in the past.

If you’re sold on containers and the value they can bring to the business, first consider the added costs and then the longer-term benefits container-based development will likely bring. Many just assume that containers will be the best way to go. Perhaps, but the value this technology can bring will not be consistent from application to application. The values of portability, scalability, and resiliency are not the same for each application.

As a rule, container-based application development costs more and takes more time. You’ll need skilled developers and architects to do a container-based application right versus more traditional approaches that we’ve leveraged for years. Do you have that talent on staff? One skill set is new; the other is not. New skills are usually more expensive to find, develop, and/or hire, and this is no different.

Keep an eye on the values when you do a business case. The additional build costs, staffing requirements, and learning curve of container-based applications remove value. Be prepared to put specific values on the benefits of scalability, portability, and resiliency. The value curve will determine if containers are worth it for your specific application or the enterprise itself. If you find the value, then containers will work for you.

What Doesn’t Work

In two words, what doesn’t work: operational complexity. Containers bring additional complexity, and that means you need abstraction and automation tools to deal with the complexity. This is not an unsolvable problem, but you need additional resources to run most containerized applications versus those that are more traditionally created.

Why? Containers and container orchestration systems are new platforms that must be managed in addition to existing platforms that may or may not reside on public clouds. Container systems are not more difficult to operate; many could be easier to use when you consider the available operations tools. It’s that you must operate them in addition to the traditional systems, processes, and applications that still exist. This makes operations more complex, thus more involved and expensive to carry out.

In other words, containers are typically an additive technology, meaning it doesn’t replace an existing technology. The containerized solutions require new skill sets and possibly new staff to operate and manage the new tools that will support containerized applications that are usually net new as well. As the numbers of moving parts rise, the amount of complexity and the cost of managing this complexity increase ongoing.

As the boom in container-based development and the number of container orchestration instances continue to rise, additive system issues will continue to fall away. However, today’s containerized applications make things more complex; thus, operations will be more difficult and more costly for now.

Obviously, the complexity impact on your specific applications and business requirements will vary. Here we’re talking about what’s generally true without illustrating the myriad of exceptions. I urge you to do your own business cases to determine your specific impact. It’s a wide variable.

Figure 5-5 shows what we currently see when it comes to complexity and the use of containers. As the number of container-based applications grows, with or without the use of container orchestration, the amount of complexity and the costs incurred from increased complexity rise as well. At a certain point between three to six years of operation, the complexity curve levels out and even begins to decline. Again, this will happen as enterprises become more adept at managing their new container-based solutions, or when the traditional applications that must also be maintained retire and are removed as an operational responsibility. That’s the good and bad news here: You must first endure the bad news on your journey to containerized solutions.

Other things that may be a downside of leveraging containerized solutions include finding the skills you need to build, deploy, and operate these solutions. Building containerized solutions correctly requires training and experience to make the right calls as to what technology to leverage. If they can’t find the talent, many enterprises either settle for not-as-qualified developers and designers, or they delay moving to containers until the skills can be found. This approach often leads to additional costs and risk. In many instances, the skills acquisition issues around containers are not even considered as part of the cost/benefit analysis to determine whether the technology is a good fit for the business. That’s not the way to find value for the business.

Container Orchestration: Cost Considerations

Much of what we called out earlier in the “Container: Cost Considerations” section of this chapter is applicable here as well. This includes the costs of skills, tools, and the time it takes to design and deploy a containerized application versus more traditional approaches. Your “container tax” may or may not be significant. Regardless, leveraging containers and container orchestration will cost you more than a traditional path.

So, let’s look specifically at a few interesting container orchestration trends that affect costs. As you can see in Figure 5-6, as enterprises leverage more container orchestration such as Kubernetes, the costs are initially very high, but they decline over time.

It typically takes two to four years of container orchestration use until the costs decline enough to make it easier to define the hard values of using containers. As you may recall from Chapter 1, these hard costs are easy to understand, such as direct costs associated with operations and security. However, just as we defined in Chapter 1, the true benefit to the business is aligned more to the soft values, that is, the business abilities gained from the use of container technology. These new abilities include agility, compressed time-to-market, and innovation. These are all common soft values enterprises find with the use of containers and container orchestration.

It’s important to understand the costs and the trade-offs of using container orchestration technology, which are depicted in Figure 5-6. The curve angles will vary depending on the type of business involved, the types of problems to solve, and the hard and soft values benefits for a particular business operation. It’s the old “it depends” answer when it comes to costs, and how costs align with business value for each enterprise. This is an individual journey, but your best jumping-off point is to understand what’s generally true, as outlined here.

Container Orchestration: Make Tough Choices

Following the crowd is often the easiest thing to do, but it’s rarely the correct thing to do. Consider why IT exists in the first place: to bring value to the business. However, I’ve been in IT for a very long time, so I suspect that many of you are already off to containers and container orchestration and have not yet considered the business benefits of doing so. If that’s you, you’re not alone. In the old days I called this “management by magazine.” Others call it “chasing the hype” to a technology because it’s popular, without first considering the desired outcome.

It takes a bit of courage to understand the core purpose of leveraging any technology and how that technology stacks up with your business and technical requirements. If that technology does not fit, it takes more courage to push back on the hype and look for better solutions that will bring more value to the business.

Containers and container orchestration and clustering are valid solutions for many purposes and should always be considered not only for their direct value but also the value that they are likely to bring in the future. In many instances, they are the correct and optimized solution to leverage, but in some cases they are not. It’s all about doing the up-front work to ensure that the right decisions are made, and your cloud solutions are as optimal as they possibly can be.

What’s the Right Definition of Cloud Native?

What concept should we be casing as solution developers when we build and deploy next-generation cloud computing systems? Or: What the hell is cloud native, and why should we care?

Given the way the term has evolved, and how it will likely continue to evolve, what definition describes the value that you can employ, and what technology stack does that definition promote? The original and cloud-washed definitions are still out there, and they’re probably not what you’re looking for when you consider cloud native development for your enterprise. Be sure you know exactly what you sign up for. Just as in the early days of cloud computing, where we experienced “cloud-washing,” what is called cloud native may not be what you’re seeking. It’s perhaps the most overused and multimeaning term that I’ve seen in this industry.

Today’s CNCF cloud native applications are built around containers, container orchestration, and microservices, and focus on the architecture of the applications more than the clouds they will deploy on. CNCF cloud native commoditizes public cloud providers by building applications above the clouds and then leveraging the public clouds as mere service providers to support the applications. This approach stands in contrast to the classic definition of cloud native that required building walled gardens within one provider’s cloud that limits the types and selection of services you can use.

We’ll use the CNCF definition of cloud native for the remainder of this chapter. It works with the larger idea that CNCF cloud native applications provide dynamic and scalable application attributes, functioning on many platforms that include public cloud providers, private clouds, and even traditional computing systems (legacy). The enabling technology stacks that CNCF cloud native promotes, as we stated previously, are containers, container orchestration, and microservices. The objective of leveraging this technology is to avoid cloud and platform lock-in.

The platforms in this CNCF architecture are foundational. However, foundational clouds don’t typically provide services directly to the applications. Instead, you’ll work through well-defined APIs that allow the underlying platforms, both cloud and traditional, to be abstracted away from the application. This provides better portability and should do so with very few trade-offs.

Figure 5-7 is a good representation of this idea. The top of the stack is where development, delivery, and deployment occur. Or, where the developers leverage a cloud native architecture. Moving down the stack, you’ll notice that developers and applications communicate with the container and container orchestration layers using microservices. These microservices define discrete interfaces that developers and applications can consistently leverage. They also remove the applications from any dependency on a specific proprietary service because they consistently use the same APIs for any service the application may need. Containers and container orchestration services (clustering) carry out the heavy lifting of application processing, and they leverage the underlying platforms (cloud and not cloud) to provide the primitive platform services (storage, compute, databases, networking, security, and so on). With this architecture, the public cloud providers (all platforms really) are no longer at the center of the architecture but are mere compute and storage service providers.

This version of a cloud native architecture is nothing new. When containers and microservices came onto the scene several years ago, developers and application designers figured out that this version of a cloud native architecture was the best path. Thus, before this was labeled as a cloud native architecture, it was a best practice. There is no magic here, and this version of a cloud native architecture does come with certain advantages, including

• The use of open systems throughout; thus, this solution tends to be more cost effective.

• Portability is built into this version of a cloud native architecture; you don’t have to constantly worry about leveraging vendor proprietary features that will officially lock in your application.

• Scalability is built into this architecture as well, allowing container orchestration to scale the applications as needed, without changing the underlying architecture to accommodate scalability. It’s always there.

• The use of public clouds as just services providers that are largely decoupled from your application deployment means you can switch out public clouds that do not provide the best pricing and/or do not behave in the best ways. While this makes cloud providers a bit nervous around the future impact of this version of a cloud native architecture, the reality is that the cloud providers will likely sell more services without having to invest in their platform’s customer interface. The cloud providers no longer deal with the user interfaces, but instead the cloud providers deal directly with the APIs that are native to a specific cloud provider.

Portability Argument

The portability benefits of being cloud native and building cloud native applications are the first things you’ll hear about from those who support this type of cloud native architecture and technology stack. As I mentioned previously, because we abstract the cloud providers from the applications using a layer of technology that deals with the differences, the applications are truly portable. We can move them from cloud provider to cloud provider at will, when needed, and this architectural approach supports whatever business needs to do.

There are a few downsides to this argument.

First, few applications ever leave the platforms they were developed on. We like the potential of built-in portability, and this benefit usually comes up many times when we design applications and pick the architecture. However, for all the time and money we will spend to avoid lock-in, that benefit may never materialize.

Second, is the cloud native (CNCF version) stack not a platform unto itself? We define portability as the ability to move between cloud providers by leveraging a cloud native stack to abstract the cloud providers from the application. Although we can indeed move our applications between cloud providers somewhat easily by not being locked into the provider, we’re still locked into the architecture we selected, in this case, a cloud native architecture. If you need to move to another architecture, say a proprietary architecture that provides some new key features that would bring value to the business, you’ll need to rewrite most of the systems and subsystems built for your cloud native architecture.

Most would chuckle at this scenario, considering this CNCF architecture is built to be portable across cloud providers and the original goal was to avoid lock-in. However, having done this for many years, more than once I’ve made application design and architecture decisions to avoid vendor lock-in only to find that the architecture and development approach I chose led me to a configuration that was hard to move to another vendor’s configurations.

Finally, it costs more to avoid lock-in. I cover this later in greater detail, but portability does not come cheap. A term used among developers is a “portability tax,” or the extra money you’ll spend to get something that minimizes lock-in. Note: We use the word minimize, and not eliminate because lock-in can never be eliminated, just reduced, no matter what architectural approach you leverage.

Optimization Argument

Optimization of this CNCF architecture, or other architectural approaches, will provide the best performance and functionality with the smallest price tag. Thus, we need proof that this architecture is the best approach with the understanding that it’s the closest we can get to being fully optimized.

There is less historical evidence around optimization related to cloud native architecture. However, we know enough about optimization itself that cloud native costs in time and or money should factor into the decision-making process to determine whether a cloud native architecture and cloud native applications are truly the best choice for the enterprise. Business value is always the top objective.

Figure 5-8 shows our optimization curve again, this time with the added dimension of cost in the curve moving up and to the right. Note that we typically have a sweet spot of investment being made. Too little investment shows less optimization, and thus less value is returned to the business. On the optimization side of the curve, we can see that too much investment reduces the value returned to the business because we spent (and often continue to spend) too much on the technology and that dilutes the value. Thus, you want to target a point in the middle where the investment made at a specific level optimizes its business value.

Although this value curve relates specifically to cloud native architecture and application development, it can be applied to any number of technology solution patterns. It’s not about finding where technology is fully optimized when compared to the investment, but how does the curve compare against other development and architectural options, even proprietary cloud provider development that leverages proprietary tools and deployment platforms? Also consider architectures that are more in the center, with a mix of proprietary and open technologies that you can mix and match as needed. Most cloud-based solutions fall in the center. They allow you to mix and match technologies to find the right solution, but you must do so on a particular vendor’s platform.

We can’t really talk about optimization without talking about the cost of the overall investment (covered next). But the architectural choices really come down to your application requirements, as related to your business requirements, and how they should fit together to form a whole. There are a lot of technology and architectural choices out there, cloud native and many others. Don’t fall into the trap of just moving along with the crowd, no matter how compelling the arguments are to use some hyped approach or technology. Doing so will quickly lead you to underoptimized solutions, and your decisions could easily end up damaging the business. Don’t be that person.

Cost Argument

Believe it or not, cost is often overlooked when selecting the best development and architectural approaches. Or, should I say, we overlook the actual costs versus the known costs? It’s easy to consider the hard costs, or what investment it will take to put an architecture and technology stack into production over the years. The soft costs often prove more valuable in the long view, but they are more difficult to identify at the outset.

For example, using a cloud native architecture to build a set of applications could provide 20 percent more agility over a five-year period. That scenario might translate into a $20 million windfall for the business because the business can quickly adapt to market and customer changes, and quickly retool to build products that are better aligned with the market. The trouble is that the value of agility for your specific business is rarely easy or simple to determine. However, this is more valuable than any hard value that you will define and thus needs special attention.

Figure 5-9 shows our friend the value curve again, this time looking at soft values over fully loaded cost. As we model different levels of investment, a different amount of soft value comes back to the business, values such as agility, speed-to-market, and innovation, that will peak and then bend the value curve at the point of overinvestment.

Several indicators determine where the value drop-off occurs as the soft value curve crosses the cost curve. First, the project hits the saturation point at a level of spending where no additional soft value benefits can be returned to the business just by spending more money. Second, if spending continues to rise, soft value drops significantly from that point. This is a classic case of tossing good money after bad. The chart displayed in Figure 5-9 follows the same pattern as the optimization curve in Figure 5-8. You’ll see this pattern repeat as we attempt to define something that is nearly fully optimized. Also, remember that soft values can be 50 times that of hard values, in terms of what’s returned to the business.

Figure 5-10 looks at the behavior of hard value returned to the business at different levels of spending. Unlike soft values, hard values mirror the spending or cost curve with an increase in value at the end of the curve, likely from discounts the business receives by spending more on something. For example, our hard costs include the underlying cloud services that support a cloud native architecture. Most cloud providers offer volume discounts as those support costs increase over time, which is reflected in the “hard value” curve that becomes steeper about two-thirds through the curve.

Hard value is soft value’s boring little brother. While you can track 10–20 percent savings over time in hard cost savings, soft value is the real core multiplier you need to consider. As I already mentioned, it often brings 50 to 100 times more value back to the business for the money invested. The trick is to understand how to model soft value for your specific use case and how to find the best optimized point for your particular type of business. You must also determine which solution offers the best optimization point of many possible solutions, including cloud native and other technology stacks.