Does the Tool Provide the Extensibility You Need?
Another key point is making sure that the tool you select is designed for and capable of extensibility. Most out-of-the box solutions are already at a stage where processes, from check-in, to build, to pushing the bits to production, is so well formalized and documented that you will hardly need to do any additional configuration to make sure that it works. That said, an organization rarely relies solely on the developer toolchain to maintain a product. In a world where users are a core contributing force to the direction of the product, several moving pieces are most likely to be in play. In a hypothetical scenario with a startup, team communications are done through Slack, external users are providing a good amount of feedback through UserVoice, while contributors to the open-source segment of the product are opening issues through GitHub. For tasks the team uses Trello, or Jixee, or a Kanban Tool. In any of these cases, you need the flexibility to make sure the tools you already use are integrated, so deployment errors are sent via Slack messages to the operations team, for example.
When choosing the CD platform, extensibility should not be overlooked as it will be an indicator of how viable it will be to use the platform in the long run. As your requirements change, your tools should be able to change and adapt with them. You and your organization do not want to be in a position where you will need to readjust the entire infrastructure and manually try to extend its capabilities because you switched the customer feedback channel.
Below are some key criteria to consider when choosing DevOps tools.
Does the Product or Service Include Tools to Provision and Deploy using Azure and/or Docker?
Microsoft Azure provides a set of services, such as VM provisioning through Azure Resource Manager templates, extensible storage, and API hooks for virtualized resources, tremendously simplifying the DevOps process. It is important to consider whether the tool you select already supports Azure as a first class deployment target to ensure that not only can you deploy, but that you receive rich diagnostic information when the deployment fails. Similarly, what set of tools or services exist for Docker? Are common workflows, like building Docker images, using Docker Compose for deployments, or pushing/pulling images to a Docker registry included? How difficult is it to integrate orchestration tools like Kubernetes or Mesosphere in your deployments?
Does the Tool Include Integration With Popular Testing Tools?
Whatever the project is that you are working on, you need to make sure that it is rigorously tested across all stages, which means that your CD pipeline should be tightly integrated with a range of possible test tools, such as SonarCube, xUnit, Selenium, and many more. Identify what testing tools your team will use and ensure that the tools provide a way to easily integrate them as part of your continuous delivery process.
Does the Tool Include Ways to Manually or Automatically Promote between Environments?
Depending on your organization’s process, microservice deployments could be fully automated from check-in to production (continuous deployment), mostly automated where the process is automated up to production but production deployments are manual (continuous delivery), or fully manual where promotion of each environment is done and signed off by a QA team. Whatever tool you select should ideally have the option for all three or be able to switch between manual and automated environment promotion. One caveat to consider for full automation is the scenario where a QA engineer is still validating an existing build while another build is being pushed into the same environment. It would be inconvenient to just swap the bits, thereby invalidating all the previously-done work. That, of course, is just one scenario, and automated approval is just as important—to move code between environments, you need to decide what the right balance of manual versus automated testing makes sense for your services.
Does the Tool Include Ways to Track a Deployment for Auditing?
We are no longer operating in a world where there is a single deployment happening over the course of a week. Instead, tens (and in some cases hundreds) of deployments are happening, all within a single day. At that point, it is important to know what changes are done, by whom, and at what stages of the process. When picking the tools for your CD infrastructure, consider how extensive the audit coverage is. As an example, Octopus Deploy captures changes to deployment processes, variables, events for projects, environments, deployment targets and releases, as well as enables queueing or cancelling deployments depending on the captured status.