We're almost finished with the pipeline, at least in its current form.

The purpose of the deploy stage is to install the new release to production and to do the last round of tests that only verify whether the new release integrates with the rest of the system. Those tests are often elementary since they do not validate the release on the functional level. We already know that the features work as expected and immutability of the containers guarantee that what was deployed as a test release is the same as what will be upgraded to production. What we're not yet sure is whether there is a problem related to the configuration of the production environment or, in our case, production Namespace.

If something goes wrong, we need to be able to act swiftly and roll back the release. I'll skip the discussion about the inability to roll back when changing database schemas and a few other cases. Instead, for the sake of simplicity, I'll assume that we'll roll back always if any of the steps in this stage fail.

Let's go back to go-demo-3 configuration screen and update the pipeline.

 1  open "http://$JENKINS_ADDR/job/go-demo-3/configure"

If you are NOT using minishift, please replace the existing code with the content of the cdp-jenkins-deploy.groovy (https://gist.github.com/vfarcic/3657e7262b65749f29ddd618cf511d72) Gist.

If you are using minishift, please replace the existing code with the content of the cdp-jenkins-deploy-oc.groovy (https://gist.github.com/vfarcic/1a490bff0c90b021e3390a66dd75284e) Gist.

The additions to the pipeline are as follows.

 1  ... 
 2  env.PROD_ADDRESS = "go-demo-3.acme.com" 
 3  ... 
 4      stage("deploy") { 
 5        try { 
 6          container("helm") { 
 7            sh """helm upgrade  
 8              go-demo-3  
 9              helm/go-demo-3 -i  
10              --tiller-namespace go-demo-3-build  
11              --namespace go-demo-3  
12              --set image.tag=${env.TAG}  
13              --set ingress.host=${env.PROD_ADDRESS} 
14              --reuse-values""" 
15          } 
16          container("kubectl") { 
17            sh """kubectl -n go-demo-3  
18              rollout status deployment  
19              go-demo-3""" 
20          } 
21          container("golang") { 
22            sh "go get -d -v -t" 
23            sh """DURATION=1 ADDRESS=${env.PROD_ADDRESS}  
24              go test ./... -v  
25              --run ProductionTest""" 
26          } 
27        } catch(e) { 
28          container("helm") { 
29            sh """helm rollback  
30              go-demo-3 0  
31              --tiller-namespace go-demo-3-build""" 
32            error "Failed production tests" 
33          } 
34        } 
35      } 
36    } 
37  }

We added yet another environment variable (PROD_ADDRESS) that holds the address through which our production releases are accessible. We'll use it both for defining Ingress host as well as for the final round of testing.

Inside the stage, we're upgrading the production release with the helm upgrade command. The critical value is image.tag that specifies the image tag that should be used.

Before we proceed with testing, we're waiting until the update rolls out. If there is something obviously wrong with the upgrade (for example, the tag does not exist, or there are no available resources), the rollout status command will fail.

Finally, we're executing the last round of tests. In our case, the tests will run in a loop for one minute.

All the steps in this stage are inside a big try block, so a failure of any of the steps will be handled with the catch block. Inside it is a simple helm rollback command set to revision 0 which will result in a rollback to the previous release.

Just as in the other stages, we're jumping from one container to another depending on the tool we need at any given moment.

Before we move on, please make the necessary changes to the values of the environment variables. Just as before, you likely need to change vfarcic to your Docker Hub and GitHub users as well as acme.com to the value of the environment variable ADDR available in your terminal session.

Please click the Save button once you're finished with the changes that aim at making the pipeline work in your environment. The rest of the steps are the same as those we performed countless times before. Click the Open Blue Ocean link from the left-hand menu, press the Run button, and click on the row of the new build.

Wait until the build is finished.

Since this is the first time we're running the deploy stage, we'll double-check that the production release was indeed deployed correctly.

 1  helm ls 
 2      --tiller-namespace go-demo-3-build

The output is as follows.

NAME      REVISION UPDATED        STATUS   CHART           NAMESPACE
go-demo-3 1        Wed Jul 18 ... DEPLOYED go-demo-3-0.0.1 go-demo-3

This is the first time we upgraded go-demo-3 production release, so the revision is 1.

How about Pods? Are they running as expected inside the go-demo-3 Namespace dedicated to production releases of that team?

 1  kubectl -n go-demo-3 get pods

The output is as follows.

NAME           READY STATUS  RESTARTS AGE
go-demo-3-...  1/1   Running 2        6m
go-demo-3-...  1/1   Running 2        6m
go-demo-3-...  1/1   Running 2        6m
go-demo-3-db-0 2/2   Running 0        6m
go-demo-3-db-1 2/2   Running 0        6m
go-demo-3-db-2 2/2   Running 0        5m

All the Pods are indeed running. We have three replicas of the API and three replicas of the database.

Finally, we'll send a request to the newly deployed release and confirm that we are getting the correct response.

 1  curl "http://go-demo-3.$ADDR/demo/hello"

The output should be the familiar hello, world! message.