Chapter 8: Integrate Git in Your Development Cycle

In earlier chapters, we looked at various Git concepts, managing a codebase in a team environment, and commonly used Git techniques to make your workflow efficient. These things help you become better at managing your code with Git, but one thing we haven’t discussed yet is how to integrate Git into your development cycle.

This chapter focuses on using Git to bring efficiency into your entire software development cycle. If you’re using Git for your personal projects, you may still find the concepts discussed here relevant.

Git and DevOps

How did DevOps come into being? Traditionally, there was a clear distinction between teams that were responsible for development, testing, deployment, and maintenance. While this distribution of functions was based on the skills required, teams realized over the years that inefficiencies were introduced due to the friction between these teams. This led to the birth of DevOps, a cross between development, operations and QA (quality assurance). DevOps is a set of guidelines that automates processes in the software development cycle, thus lowering the time to market. A DevOps engineer oversees the code release cycle and ensures that the process is smooth.

Though the stages in the DevOps cycle can vary, it generally involves the following seven steps:

• Continuous development. This involves planning a roadmap depending on the end-user’s requirements, and then starting development.
• Continuous integration. This involves testing all code on every push, and merging with the main codebase if all tests pass. An additional code-review process may be included.
• Continuous testing. This involves testing the application in a live environment to verify the product’s functionality.
• Continuous monitoring. In this phase, critical metrics of a product’s output and usage are monitored in order to identify any anomalies.
• Continuous feedback. Feedback from earlier stages is used to update the roadmap and proactively include features and bug fixes.
• Continuous deployment. This ensures that the availability of the end product isn’t affected by releases.
• Continuous operations. The objective of this step is the automation of the release process, which leads to a shorter development cycle.

The use of Git is critical in the early stages of the DevOps cycle. Git ensures proper code management, attribution, and integration with other stages. Git can trigger a set of processes at the onset of an action, like a code push or a pull request. Without the use of any version control, it would be difficult to create triggers based on code changes.

Using Git Hooks

While we’ve seen that Git is important to DevOps, this section deals with “Git hooks”, a technique to seamlessly integrate Git with the DevOps cycle. A Git hook is a custom script that executes when a pre-defined action or event occurs. Git hooks are of two types: client-side and server-side. A client-side Git hook is reactive to local changes: it happens on actions such as a commit or merge. A server-side Git hook gets initiated on a network-based action, like a push to a remote.

As Git hooks are executable scripts, there’s no limit to the actions you can perform with them. The simplest use case of a Git hook is to send an email on every new commit. You could initiate unit tests to be run on every commit. You could raise a pull request on every push to a new branch.

Git hooks reside in the .git/hooks directory of your repository. When you initialize a Git repository, Git populates this directory with sample scripts with the .sample extension.

If you’d like to create a hook, remove the .sample extension from any of them. Since they’re executed on an action or event, double-check the file permissions to ensure they’re executable.

If you view the contents of any of these files, you’ll notice that they’re Bash scripts with an opening line of #!/bin/sh. While the sample files are Bash scripts, you can change the first line to change the language of the script. While local Git hooks generally work around changes (pre-commit, commit-msg, post-commit), server hooks (pre-receive, update, post-update) are mostly centered around updates. Here’s a list of common hooks and when they’re triggered:

• pre-commit: before a commit message is entered
• commit-msg: with the commit message (it’s a path to a temporary file)
• post-commit: after a commit process is complete
• pre-push: before changes are sent to a remote
• post-merge: after a merge is complete
• pre-receive: before a push from a client
• post-receive: after a push from a client is completed

The most important Git hook is the pre-commit, where you can integrate unit tests:

#!/usr/bin/env bash

echo "Running pre-commit Git hook to run unit tests"
python ../../tests.py

# $? stores exit value of the last command
if [ $? -ne 0 ]; then
 echo "Tests did not pass! Aborting commit."
 exit 1
fi

While using Git hooks provides you with a lot of functionality to integrate with the software development cycle, it may be difficult for the average user to write custom scripts to achieve the objectives. However, you can utilize continuous integration (CI) tools to achieve this task.

Integrating Travis CI with GitHub

Continuous integration is the second step in the DevOps cycle, and it involves substantial use of Git. Continuous integration ensures a smooth transition between the code push and beta testing. It ensures that any new changes to the repository won’t break things.

Travis CI is a popular, hosted continuous integration tool for GitHub repositories. It’s free to use for open-source GitHub projects, with paid plans for commercial projects. You can only test repositories hosted on GitHub with Travis CI. However, you can create a workaround by adding submodules to test repositories hosted on other platforms.

Travis CI works like this: on every push to GitHub, the tool essentially clones the repository on a virtual machine on the cloud, installs the requirements on the fly, and runs pre-defined unit tests to determine if the new code breaks the existing functionality of your project.

Getting Started with Travis CI

To integrate Travis CI with your GitHub repository, you need to first create an account on the Travis CI website. If you log in through your GitHub account, you can skip the additional step of linking GitHub with Travis CI.

After logging in, go to the settings page of your Travis CI account to view the list of public repositories in your connected GitHub account. Search the repository you’d like to link to Travis CI and select the toggle button next to the repository.

Next, you need to add a configuration file to the root directory of your repository. Name the configuration file .travis.yml. Here are the minimal settings for your Travis CI configuration file:

language: python
python:
    - "3.7"
    - "3.8"

# command to install dependencies
install: "pip install -r requirements.txt"

# command to run tests
script: python tests.py

First, you instruct Travis CI to test each build on Python and then specify the versions 3.7 and 3,8. Note that, for each version you specify, a new build will be created and tested, thereby increasing the time to perform the test. Next, you provide the command to install the dependencies of the repository. Finally, you provide the command to run the unit tests, against which the push will be evaluated.

After you’ve created the configuration file, push a commit to the repository to trigger a Travis CI build. Interestingly, without the configuration file in your repository earlier, the commit that introduces this file in the repository would also trigger a build on Travis CI.

Travis CI Build Results

How long Travis CI takes to run tests on a single commit depends on a few factors:

• how many versions your build needs to be tested on
• how many requirements your repository has
• how detailed your unit tests are

Once your tests are complete, you receive an email with the results of the test. You can also view the status of your recent tests on your Travis dashboard.

If the tests are still running, Travis CI shows the live information about the test. You can cancel the build if it’s taking a long time, or even restart a build after it has completed. You can view the configuration file from within the build page to ensure the tests ran the way you intended them to.

Advanced Configuration Settings

Now that we’ve run a successful test on Travis CI, let’s examine some advanced features of Travis CI and their use cases. (You can view a full list of Travis CI configuration settings in the Travis documentation.)

• Imagine you intend to run Git-related operations in the build. If a unit test fails, you’d like to automate the process of git bisect within this build to find out which commit introduced a bug. To go ahead in this scenario, it isn’t necessary to download the full history of the project. With the depth option, you can set the number of Git commits to clone through the following setting:

    git:
      depth: 3
  

• Next, while unit tests test how your code runs, your project may have detailed requirements before installation of the prerequisites. In such cases, you may feel the need to run custom configuration commands on the server before running your projects’s build. You can also set a list of commands to run before installing the prerequisites. You can pass terminal commands as a list to run in this setting. The echo command in the example below prints a message in your log:

    before_install:
      - echo "running before_install commands"
      - python -c '# some_python_command'
      - echo "pre-installation config complete"
  

• On similar lines, you can set a list of commands to run after the build is complete. For instance, did the right version of your project execute? Do you want to check if data integrity is maintained on your server? The following terminal commands are sequentially run after the build is complete:

    after_script:
      - echo "running after_script commands"
      - python -c "#some_python_command"
      - echo "after_script commands executed"
  

• It’s possible that, during the process of a build, you may only be interested in how the current version of your project runs. To make the debugging process easier, you may wish to remove Git-related messages from your logs altogether. To ignore Git-related messages in your log files, you can set the quiet option in git to true:

    git:
      quiet: true
  

• While multiple developers work on their own features, you probably won’t want to trigger a build every time a developer sends a push. You can safelist and blacklist branches to trigger builds for, or ignore changes to specific branches:

    # blocklist
      branches:
        except:
        - some_experimental_feature
  
      # safelist
      branches:
        only:
        - master
        - dev
  

• While safelisting branches makes sense for a smaller team, you may still be dealing with a large number of push operations. There’s an option to ignore build triggers with every push altogether — by setting up cron jobs on Travis CI to run a set of tasks periodically. Go to the Cron Jobs section on the settings tab of any repository to set up cron jobs.

  

• Travis can integrate with Docker to create a custom build environment to test your code on. You need to enable Docker under services and set up custom Docker commands in the before_install section:

    services:
      - docker
  
    before_install:
    # Custom docker commands
  

Conclusion

What Have You Learned?

In this chapter, we covered the integration of Git with DevOps to enable you to bring more efficiency into your software development cycle. We first looked at custom scripts that can be used through Git hooks for various events and actions in Git. We then discussed how to integrate Git with Travis CI to perform the task of continuous integration, one of the critical steps in the DevOps cycle.

What’s Next?

In the next chapter, we’ll look at some GUI tools for Git, examining how they handle the commands we’ve already discussed.