Automating upgrade of the production environment

Now that we have a new release waiting, we would go through the same process as before. Someone would make a decision whether the release should be deployed to production or to let it rot until the new one comes along. If the decision is made that our users should benefit from the features available in that release, we'd need to update a few files in our k8s-prod repository.

 1  cd ../k8s-prod

The first file we'll update is helm/requirements.yaml. Please open it in your favorite editor and change the go-demo-5 version to match the version of the Chart we pushed a few moments ago.

We should also increase the version of the prod-env Chart as a whole. Open helm/Chart.yaml and bump the version.

Let's take a look at Jenkinsfile.orig from the repository.

 1  cat Jenkinsfile.orig

The output is as follows.

import java.text.SimpleDateFormat
    
pipeline {
  options {
    buildDiscarder logRotator(numToKeepStr: '5')
    disableConcurrentBuilds()
  }
  agent {
    kubernetes {
      cloud "kubernetes"
      label "prod"
      serviceAccount "build"
      yamlFile "KubernetesPod.yaml"
    }      
  }
  environment {
    cmAddr = "cm.acme.com"
  }
  stages {
    stage("deploy") {
      when {
        branch "master"
      }
      steps {
        container("helm") {
          sh "helm repo add chartmuseum http://${cmAddr}"
          sh "helm repo update"
          sh "helm dependency update helm"
          sh "helm upgrade -i prod helm --namespace prod --force"
        }
      }
    }
    stage("test") {
      when {
        branch "master"
      }
      steps {
        echo "Testing..."
      }
      post {
        failure {
          container("helm") {
            sh "helm rollback prod 0"
          }
        }
      }
    }
  }
}

 

This time we're using the kubernetes Cloud configured to spin up Pods in the prod Namespace. The build ServiceAccount already has the permissions to access Tiller in kube-system thus allowing us to install applications anywhere inside the cluster. We won't need to go that far. Full permissions inside the prod Namespace are more than enough.

Just as with the Jenkinsfile inside the go-demo-5 repository, the definition of the agent Pod is inside the KubernetesPod.yaml file. I'll let you explore it yourself.

The environment block contains cmAddr set to cm.acme.com. That's why we're exploring Jenkinsfile.orig. We'll need to create our own Jenkinsfile that will contain the correct address.

We have only two stages; deploy and test. Both of them have the when block that limits the execution of the steps only to builds initiated through a commit to the branch "master".

The deploy stage runs in the helm container. The steps inside it are performing the same actions we did manually a while ago. They add chartmuseum to the list of repositories, they update the repos, they update the dependencies, and, finally, they upgrade the Chart. Since we already executed all those steps from a terminal, it should be pretty clear what they do.

The test stage has a simple echo step. I'll be honest with you. I did not write tests we'd need, and the echo is only a placeholder. You should know how to write your own tests for the applications you're developing, and there's probably no need for you to see yet another set of tests written in Go.

The critical part of the stage is the post section that'll rollback the upgrade if one of the tests fail. This is the part where we're ignoring GitOps principles. The chances that those tests will fail are meager. The new release was already tested, and containers guarantee that our applications will behave the same in any environment. The tests we're running in this pipeline are more like sanity checks, than some kind of in-depth validation.

If we'd adhere fully to GitOps, if tests do fail, we'd need to change the version of the Chart to the previous value, and we'd need to push it back to the repository. That would trigger yet another build that would perform another upgrade, only this time to the previous release. Instead, we're rolling back directly inside the pipeline assuming that someone will fix the issue soon after and initiate another upgrade that will contain the correction.

A note to minishift users
We already created a Route as a substitute for Ingress. Since, from now on, we're only updating the go-demo-5 application while preserving the related Service, there's no need to add the oc create route command to the pipeline.

As you can see, we are applying GitOps principles only partially. In my opinion, they make sense in some cases and do not in others. It's up to you to decide whether you'll go towards full GitOps, or, like me, adopt it only partially.

Now, let's create Jenkinsfile with the correct address.

 1  cat Jenkinsfile.orig 
 2      | sed -e "[email protected]@$ADDR@g" 
 3      | tee Jenkinsfile

With all the files updated, we can proceed and push the changes to GitHub. Just as before, we're taking a shortcut by skipping the processes of making a pull request, reviewing it, approving it, and executing any other steps that we would normally do.

 1  git add .
2 3 git commit -m "Jenkinsfile"
4 5 git push

The only thing left is to create a new Jenkins job and hope that everything works correctly.

 1  open "http://$JENKINS_ADDR/blue/pipelines"

Please click the New Pipeline button, and select GitHub and the organization. Next, we'll choose k8s-prod repository and click the Create Pipeline button.

The new job was created, and all we have to do is wait for a few moments until it's finished.

Figure 8-4: k8s-prod build screen

Let's see the history of the Chart.

 1  helm history prod

The output is as follows.

REVISION UPDATED      STATUS     CHART          DESCRIPTION
1        Wed Aug  ... SUPERSEDED prod-env-0.0.1 Install complete
2        Wed Aug  ... SUPERSEDED prod-env-0.0.1 Upgrade complete
3        Wed Aug  ... DEPLOYED   prod-env-0.0.2 Upgrade complete

We can see from the CHART column that the currently deployed release is 0.0.2 (or whichever version you defined last).

Our system is working! We have a fully operational continuous delivery pipeline.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.97.187