I spend a significant chunk of my time helping companies improve their systems. The most challenging part of my job is going back home after an engagement knowing that the next time I visit the same company, I will discover that there was no significant improvement. I cannot say that is not partly my fault. It certainly is. I might not be very good at what I do. Or maybe I am not good at conveying the right message. Maybe my advice was wrong. There can be many reasons for those failures, and I do admit that they are probably mostly my fault. Still, I cannot shake the feeling that my failures are caused by something else. I think that the root cause is in false expectations.
People want to improve. That is in our nature. Or, at least, most of us do. We became engineers because we are curious. We like to play with new toys. We love to explore new possibilities. And yet, the more we work in a company, the more we become complacent. We learn something, and then we stop learning. We shift our focus towards climbing company ladders. The more time passes, the more emphasis we put on defending our positions which often mean the status quo.
We become experts in something, and that expertise brings us to glory and, hopefully, it lands a promotion, or two. From there on, we ride on that glory. Look at me, I'm a DB2 expert. That's me, I set up VMWare virtualization. I brought the benefits of Spring to our Java development. Once that happens, we often try to make sure that those benefits stay intact forever. We won't switch to NoSQL because that would mean that my DB2 expertise is not as valuable anymore. We won't move to Cloud, because I am the guru behind VMWare. We will not adopt Go, because I know how to code in Java.
Those voices are critical because they are being voiced by senior people. Everyone needs to listen to them, even though the real motivations behind those voices are selfish. They are not based on actual knowledge, but often on repeated experience. Having twenty years of experience with DB2 is not truly twenty years of improvement, but rather the same experience repeated twenty times. Yet, twenty years has weight. People listen to you, but not because they trust you, but because you are senior and management believes in your capabilities to make decisions.
Combine voices from the old with management's fear of unknown and their quest for short-term benefits. The result is often status quo. That worked for years, why would we change it to something else. Why would I trust a junior developer telling me what to do? Even if a claim for change is backed by the experience from giants like Google, Amazon, and Netflix (just to name a few), you are likely to get a response along the following lines. "We are different". "That does not apply here." "I'd like to do that but regulations, which I do not truly understand, prevent me from changing anything."
Still, sooner or later, a directive to change comes along. Your CTO might have gone to the Gartner meeting where he was told to switch to microservices. Too many people spoke about Agile for upper management to ignore it. DevOps is a huge thing, so we need to employ it as well. Kubernetes is everywhere, so we'll start working on a PoC soon.
When those things do happen, when a change is approved, you might be ecstatic. This is your moment. This is when you'll start doing something delicious. That is often the moment when I receive a call. "We want to do this and that. Can you help us?" I usually (not always) say yes. That's what I do. And yet, I know that my engagement will not produce a tangible improvement. I guess that hope dies last.
Why am I so pessimistic? Why do I think that improvements do not produce tangible benefits? The answer lies in the scope of required changes.
Almost every tool is a result of specific processes. A process, on the other hand, is a product of a particular culture. Adopting a process without making cultural changes is a waste of time. Adopting tools without accepting the processes behind them is a futile effort that will result only in wasted time and potentially substantial license costs. In infrequent occasions, companies do choose to accept the need to change all three (culture, processes, and tools). They make a decision, and sometimes they even start moving in the right direction. Those are precious cases that should be cherished. But they are likely to fail as well. After a while, usually a few months later, we realize the scope of those changes. Only the brave will survive, and only those committed will see it through.
Those who do choose to proceed and indeed change their culture, and their processes, and their tools, will realize that they are incompatible with the applications they've been developing over the years. Containers work with everything, but benefits are genuinely tremendous when developing microservices, not monoliths. Test-driven development increases confidence, quality, and speed, but only if applications are designed to be testable. Zero-downtime deployments are not a myth. They work, but only if our applications are cloud-native, if they follow at least some of twelve factors (https://12factor.net/), and so on.
It's not only about tools, processes, and culture, but also about getting rid of the technical debt you've been accumulating over the years. By debt, I don't necessarily mean that you did something wrong when you started, but rather that time converted something awesome into a horrible monster. Do you spend fifty percent of your time refactoring? If you're not, you're accumulating technical debt. It's unavoidable.
When faced with all those challenges, giving up is the expected outcome. It's human to throw down the towel when there's no light at the end of the tunnel. I don't judge you. I feel your pain. You're not moving forward because the obstacles are too big. Still, you have to get up because there is no alternative. You will continue. You will improve. It'll hurt a lot, but there is no alternative, except slow death while your competition is looking over your soon-to-be corpse.
You got this far, and I can assume only two possible explanations. You are one of those who read technical books as a way to escape from reality, or you are applying at least some of the things we discussed thus far. I hope it's the latter. If that's the case, you do not fall into "yet another failure of mine." I thank you for that. It makes me feel better.
If you do employ the lessons from this book, without faking, you are indeed doing something great. There is no way of pretending continuous delivery (CD). Every commit you make is ready for production if all the stages are green. The decision whether to deploy it to production is based on business or marketing needs, and it is not technical in any sense. You can even take a step forward and practice continuous deployment (CDP). It removes the only action performed by a human and deploys every green commit to production. Neither of the two can be faked. You cannot do CD or CDP partly. You cannot be almost there. If you are, you're doing continuous integration, it-will-be-deployed-eventually process, or something else.
All in all, you are, hopefully, ready to do this. You will take a step towards continuous deployment inside a Kubernetes cluster. By the end of this book, the only thing left for you is to spend an unknown amount of time "modernizing" architecture of your applications or throwing them to thrash and starting over. You'll be changing your tools, processes, and culture. This book will not help you with all of those. We're focused on tools and processes. You'll have to figure out what to do with your culture and architecture. The same holds true for the way you write your tests. I won't teach you testing, and I won't preach TDD. I'll assume that you already know all that and that we can focus on continuous deployment pipeline only.
At this moment, you might feel desperate. You might not be ready. You might think that you don't have a buyout from your management, that the people in your company will not accept this direction, or that you don't have enough time and funds.
Do not get depressed. Knowing the path is the most crucial part. Even if you cannot get there any time soon, you should still know what the destination is, so that your steps are at least moving you in the right direction.