There are different deployment approaches or patterns that can be used to address issues like down-times, reduce risks with deployment, and for a seamless development and deployment of software.
- Continuous deployment: Continuous deployment is a deployment model where software is ready to go live at any time. Continuous delivery is possible only if tiers, including development, testing, and staging, are integrated continuously. In a continuous deployment model, multiple production deployments can occur in a day, and automatically, via a deployment pipeline. Since one is constantly deploying incremental changes, the continuous deployment mode minimizes deployment risks. In agile software development houses, it also helps the customer to track progress directly by seeing live code in production almost as soon as it leaves development and testing. There is also the added advantage of getting user feedback faster allowing faster iterations to the code and features.
- BlueGreen deployment: We already discussed this in Chapter 5, Writing Applications that Scale. Blue green deployments keep two production environments, closely identical to each other. At a given instance, one environment is live (Blue). You prepare your new deployment changes to the other environment (Green), and, once tested and ready to go live, switch your systems—Green becomes active and Blue becomes the backup. BlueGreen deployments reduce deployment risks considerably, since, for anything that goes wrong with the new deployment, you just need to switch your router or load-balancer to the new environment. Usually, in typical BlueGreen systems, one system is the production (live) and other the staging, and you switch the roles between them.
- Canary releases: If you want to test the changes in your software on a subset of users before deploying it for the entire audience of your customers, you can use this approach. In canary release, the changes are rolled out to a small subset of users first. A simple approach is dogfooding, where the changes are rolled out internally to the employees first. Another approach is beta-testing, where a select group of audience is invited to test out your early features. Other involved approaches include selecting users based on their geographic location, demographics, and profiles. Canary releases, apart from insulating the company from sudden user reaction to badly managed features, also allow you to manage load and capacity scaling in an incremental way. For example, if a particular feature becomes popular, and starts driving, say, 100X users to your servers than before, a traditional deployment may cause server failures and availability issues as opposed to a gradual deployment using a Canary release. Geographical routing is a technique that can be used to select a subset of users if you don't want to do complex user profiling and analysis. This is where the load is sent more to nodes deployed in a particular geography or data center as opposed to other nodes. Canary release is also related to the concept of incremental rollout or phased rollout.
- Bucket testing (A/B testing): This is the technique of deploying two dissimilar versions of an application or a webpage to production to test out which version is more popular and/or has more engagement. In production, a subset of your audience sees the A version of the app (or page)—the control or basic version—and the other subset sees the B version or the modified (variant) version. Usually, this is a 50-50 split, though, as with Canary releases, user profiles, geo locations, or other complex models can be used. User experience and engagement is collected using an analytics dashboard, and then it is determined whether the change had a positive, negative, or neutral response.
- Induced chaos: This is a technique of purposely introducing errors or disabling part of a production deployment system to test its resilience to failures and/or level of availability.
Production servers have the problem of drift—unless you use continuous deployment or similar approaches for sync, production servers, usually, tend to drift away from the standard configuration. One way to test your system is to intentionally disable part of the production system—this can be done, for example, by disabling a random 50% of the nodes in a load-balancer configuration, and see how the rest of the system performs.
A similar approach in finding out and weeding unused parts of code is to inject random secrets in parts of the configuration using, say, an API that you suspect is redundant and no longer required. You then observe how the application performs in production. Since a random secret will fail the API, if there is an active part of the application which still uses the dependent code, it will fail in production. Otherwise, it is an indication that the code can be safely removed.
Netflix has a tool called Chaos Monkey, which automatically injects failures in production systems, and then measures the impact.
Induced Chaos allows the DevOps engineer and architect to understand weak points in the system, learn about systems which are undergoing configuration drift, and find and weed out unnecessary or unused parts of an application.