Let's step back and paint a high-level picture of the continuous delivery pipeline we created. To be more precise, we'll draw a diagram instead of painting anything. But, before we dive into a continuous delivery diagram, we'll refresh our memory with the one we used before for describing continuous deployment.

The continuous deployment pipeline contains all the steps from pushing a commit to deploying and testing a release in production.

Continuous delivery removes one of the stages from the continuous deployment pipeline. We do NOT want to deploy a new release automatically. Instead, we want humans to decide whether a release should be upgraded in production. If it should, we need to decide when will that happen. Those (human) decisions are, in our case, happening as Git operations. We'll comment on them soon. For now, the important note is that the deploy stage is now removed from pipelines residing in application repositories.

The fact that our application pipeline (for example, go-demo-5) does not perform deployment does not mean that it is not automated. The decisions which versions to use and when to initiate the upgrade process is manual, but everything else proceeding those actions is automated.

In our case, there is a separate repository (k8s-prod) that contains a full definition of what constitutes production environment. Whenever we make a decision to install a new application or to upgrade an existing one, we need to update files in k8s-prod and push them to the repository. Whether that push is performed directly to the master branch or to a separate branch, is of no importance to the process that relies solely on the master branch. If you choose to use separate branches (as you should), you can do pull requests, code reviews, and all the other things we usually do with code. But, as I already mentioned, those actions are irrelevant from the automation perspective. The master branch is the one that matters. Once a commit reaches it, it initiates a Webhook request that notifies Jenkins that there is a change and, from there on, we run a build that upgrades the production environment and executes light-weight tests with sanity checks.

Except, that we did not set up GitHub Webhooks. I expect that you will have them once you create a "real" cluster with a "real" domain.

How does continuous delivery of applications combine with unified deployment to the production environment?

Let's imagine that we have four applications in total. We'll call them app 1, app 2, app 3, and app 4. Those applications are developed independently of each other. Whenever we push a commit to the master branch of one of those applications, corresponding continuous delivery pipeline is initiated and, if all the steps are successful, results in a new production-ready release. Pipelines are launched when code is pushed to other branches as well, but in those cases, production-ready releases are NOT created. So, we'll ignore them in this story.

We are accumulating production-ready releases in those applications and, at some point, someone makes a decision to upgrade the production environment. Those upgrades might involve an update of a single application, or it might entail update of a few. It all depends on the architecture of our applications (sometimes they are not independent), business decisions, and quite a few other criteria. No matter how we made the decision which applications to update and which releases to use, we need to make appropriate changes in the repository that serves as the source of truth of the production environment.

Let's say that we decided to upgrade app 1 to the release 2 and app 4 to release 4, to install the release 3 of app 2 for the first time, and to leave app 3 intact. In such a situation, we'd bump versions of app 1 and 4 in requirements.yaml. We'd add a new entry for app 2 since that's the first time we're installing that application. Finally, we'd leave app 3 in requirements.yaml as-is since we are not planning to upgrade it.

Once we're finished with modifications to requirements.yaml, all that's left is to bump the version in Chart.yaml and push the changes directly to master or to make a pull request and merge it after a review. No matter the route, once the change reaches the master branch, it fires a Webhook which, in turn, initiates a new build of the Jenkins job related to the repository. If all of the steps are successful, the Chart representing the production environment is upgraded and, with it, all the applications specified in requirements.yaml are upgraded as well. To be more precise, not all the dependencies are upgraded, but only those we modified. All in all, the production environment will converge to the desired stage after which we'll execute the last round of tests. If something fails, we roll back. Otherwise, another iteration of production deployments is finished, until we repeat the same process.