CI and Version Control

Version control is essential for CI. A CI system needs to acquire the most recent version of a project without human intervention (at least once things have been set up).

I create a repository on my git server as I did in Chapter 17:

$ sudo mkdir /var/git/userthing
$ sudo chown git:git /var/git/userthing
$ sudo su git
$ cd /var/git/userthing
$ git --bare init

I created the userthing directory, and changed its owner and group. Then, as the git user, I created the repository. Now to import the simple userthing codebase from my local development machine.

$ cd /home/mattz/work/userthing
$ git init
$ git add .
$ git commit -m 'first commit'
$ git remote add origin [email protected]:/var/git/userthing
$ git push origin master

I navigated to my development directory and initialized it. Then I added the origin remote, to which I pushed the code. I like to confirm that everything is working by performing a fresh clone:

$ git clone [email protected]:/var/git/userthing

Cloning into 'userthing'...
Enter passphrase for key '/home/mattz/.ssh/id_rsa':
remote: Counting objects: 61, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 61 (delta 14), reused 0 (delta 0)
Receiving objects: 100% (61/61), 9.09 KiB, done.
Resolving deltas: 100% (14/14), done.

Now I have a userthing repository, and a local clone. Time to automate build and test.

Phing

We encountered Phing in Chapter 19. Here is how I installed the library:

$ pear channel-discover pear.phing.info
$ pear install phing/phing

I will be using this essential build tool as the glue for my project’s CI environment. I will define targets for building and testing the code and for running the various other quality assurance tools you will meet in this chapter.

Let’s start with the basics:

<project name="userthing" default="build">
    <property name="build" value="./build" />
    <property name="src" value="./src" />
    <property name="version" value="1.1.1" />
 
    <target name="build">
        <mkdir dir="${build}" />
        <copy todir="${build}/userthing">
            <fileset dir="${src}/userthing">
            </fileset>
        </copy>
 
        <copy todir="${build}/test">
            <fileset dir="${src}/test">
            </fileset>
        </copy>
    </target>
 
    <target name="clean">
        <delete dir="${build}" />
    </target>
    <!-- ... -->
</project>

I set up three properties. build refers to the directory in which I will assemble my files before generating a package. It’s in here that I’ll ultimately run my tests and the other tools that this chapter covers. src refers to the source directory. version defines the version number for the package.

The build target copies the userthing and test directories into the build environment. In a more complex project I might also perform transformations, generate configuration files, and assemble binary assets at this stage. This target is the project’s default.

The clean target removes the build directory and anything it contains.

Let’s run a build:

$ phing

Buildfile: /home/mattz/work/userthing/build.xml
userthing > build:
    [mkdir] Created dir: /home/mattz/work/userthing/build
     [copy] Created 4 empty directories in /home/mattz/work/userthing/build/userthing
     [copy] Copying 3 files to /home/mattz/work/userthing/build/userthing
     [copy] Created 1 empty directory in /home/mattz/work/userthing/build/test
     [copy] Copying 2 files to /home/mattz/work/userthing/build/test
 
BUILD FINISHED
 
Total time: 0.7534 seconds

Unit Tests

Unit tests are the key to continuous integration. It’s no good successfully building a project that contains broken code. I covered unit testing with PHPUnit in Chapter 18. If you’re reading out of order, however, you’ll want to install this invaluable tool before proceeding.

$ pear config-set auto_discover 1
$ pear install --alldeps pear.phpunit.de/phpunit

Also in Chapter 18 I wrote tests for a version of the userthing code I’ll be working with in this chapter. Here I run them once again (from within the src directory), to make sure my reorganization has not broken anything new.

$ phpunit test/ 3.7.24 by Sebastian Bergmann.
 
.....
 
Time: 317 ms, Memory: 3.75Mb
 
OK (5 tests, 5 assertions) 

So this confirms that my tests work. However, I would like to invoke them with Phing.

Phing provides an exec task which we might use to invoke the phpunit command. However, it’s always best to use a specialized tool if there’s one available.  There is a builtin task for this job:

<target name="test" depends="build">
        <phpunit>
            <formatter type="plain" usefile="false"/>
            <batchtest classpath="${build}">
                <fileset dir="${build}/test">
                    <include name="**/*Test.php"/>
                </fileset>
            </batchtest>
        </phpunit>
    </target>

The test target depends upon the build target. Because of this, it can be sure that there’s an environment in place under the build directory with which to work. Among many other attributes, the phpunit task accepts a printsummary attribute, which causes an overview of the test process to be output.

Much of this task’s functionality is configured using nested elements. The formatter element manages the way that test information is generated. In this case, I have opted to output basic human-readable data. batchtest lets you define multiple test files using the nested fileset element. Note that batchtest accepts a classpath attribute. This sets the include path for your tests (the name derives from a Java term).

Here, I run the tests with Phing:

$ phing test

Buildfile: /home/mattz/work/userthing/build.xml
userthing > build:
userthing > test:
 
  [phpunit] Testsuite: ValidatorTest
 
  [phpunit] Tests run: 2, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.07081 s
  [phpunit] Testsuite: UserStoreTest
  [phpunit] Tests run: 3, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.02879 s
  [phpunit] Total tests run: 5, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.10772 s
 
BUILD FINISHED
 
Total time: 1.0921 second

Documentation

Transparency is one of the principles of CI. When you’re looking at a build in a Continuous Integration environment, therefore, it’s important to be able to check that the documentation is up to date, and covers the most recent classes and methods. I examined phpDocumentor in Chapter 16, so I’ve already run an install like this:

$ pear channel-discover pear.phpdoc.org
$ pear install phpdoc/phpdocumentor

I’d better invoke the tool just to be sure, this time from the build directory:

$ mkdir docs
$ phpdoc --directory=userthing --target=docs --title=userthing --template=abstract

This generates some pretty bare documentation. Once it's published on a CI server, I'm sure to be shamed into writing some real inline documentation.

Once again, I want to add this to my build.xml document. Although there is a task named phpdoc2 that is designed to integrate with PHPDocumentor, at the time of this writing this wrapper is broken. Never mind, however; I can use the exec task to invoke the command line phpdoc tool:

    <target name="doc" depends="build">
        <mkdir dir="${build}/docs" />
        <exec executable="phpdoc" dir="${build}">
          <arg line="--directory=userthing --target=docs --title=userthing --template=abstract" />
        </exec>
    </target>

Again, my doc target depends upon the build target. I create the build/docs output directory, and then invoke the exec task. I specify phpdoc in the executable attribute. The dir attribute defines the directory from which the command is to be run. exec accepts a nested arg element, which will pass any further arguments to the command. Where there is a single argument , you can set the arg element’s value attribute. In this case, however, I have a list of arguments and the line attribute, which accepts multiple values separated by spaces, is more appropriate.

Code Coverage

It’s no good relying on tests if they don’t apply to the code you have written. PHPUnit includes the ability to report on code coverage. Here’s an extract from PHPUnit’s usage information.

  --coverage-html <dir>    Generate code coverage report in HTML format.
  --coverage-clover <file> Write code coverage data in Clover XML format.

In order to use this feature, you must have the Xdebug extension installed. You can find more about this at http://pecl.php.net/package/Xdebug (installation information at http://xdebug.org/docs/install). You may also be able to install directly using your Linux distribution’s package management system. This should work for you in Fedora, for example:

$ yum install php-pecl-xdebug

Here I run PHPUnit from the src/ directory with code coverage enabled:

$ mkdir /tmp/coverage
$ phpunit --coverage-html /tmp/coverage test
 
PHPUnit 3.7.24 by Sebastian Bergmann.
 
.....
 
Time: 3.36 seconds, Memory: 4.75Mb
 
OK (5 tests, 5 assertions)
Generating code coverage report in HTML format ... done

Now you can see the report in your browser. See Figure 20-2.

It’s important to note that achieving full coverage is not the same as adequately testing a system. On the other hand, it’s good to know about any gaps in your tests. As you can see from Figure 20-2, I’ve still got some work to do.

Having confirmed that I can check coverage from the commandline, I need to add this functionality to my build document:

<target name="citest" depends="build">
    <mkdir dir="${build}/reports/coverage" />
 
    <coverage-setup database="${build}/reports/coverage.db">
        <fileset dir="${build}/userthing">
            <include name="**/*.php"/>
        </fileset>
    </coverage-setup>
 
        <phpunit codecoverage="true">
            <formatter type="plain" usefile="false"/>
            <formatter type="xml" outfile="testreport.xml" todir="${build}/reports" />
            <formatter type="clover" outfile="cloverreport.xml" todir="${build}/reports" />
            <batchtest classpath="${build}">
                <fileset dir="${build}/">
                    <include name="test/**"/>
                </fileset>
            </batchtest>
        </phpunit>
 
        <coverage-report outfile="${build}/reports/coverage.xml">
           <report todir="${build}/reports/coverage" />
        </coverage-report>
    </target>

I have created a new task named citest. Much of it is a reproduction of the test task you have already seen.

I start by creating a reports directory and a coverage subdirectory.

I use the coverage-setup task to provide configuration information for the coverage feature. I specify where raw coverage data should be stored using the database attribute. The nested fileset element defines the files that should be subject to coverage analysis.

I have added two formatter elements to the phpunit task. The formatter of type xml will generate a file named testreport.xml, which will contain the test results. The clover formatter will generate coverage information, also in XML format.

Finally, in the citest target, I deploy the coverage-report task. This takes existing coverage information, generates a new XML file and then outputs an HTML report.

Coding Standards

I can argue all day about the best place to put a brace, whether to indent with tabs or spaces, or how to name a private property variable. Wouldn’t it be nice if I could enforce my prejudices with a tool? Thanks to PHP_CodeSniffer, I can. CodeSniffer can apply one of a set of coding standards to a project and generate a report, telling you just how bad your style is.

That might sound like a massive pain in the rear end. In fact, it can be just that. But there are sensible nonpassive aggressive uses for a tool like this. I’ll get to these, but first I’ll put the tool through its paces. Installation first:

$ sudo pear install PHP_CodeSniffer

Now I’m going to apply the Zend coding standard to my code:

$ phpcs --standard=Zend build/userthing/persist/UserStore.php

FILE: ...userthing/build/userthing/persist/UserStore.php
--------------------------------------------------------------------------------
 
FOUND 9 ERROR(S) AFFECTING 8 LINE(S)
--------------------------------------------------------------------------------
  6 | ERROR | Opening brace of a class must be on the line after the definition
  7 | ERROR | Private member variable "users" must contain a leading underscore
  9 | ERROR | Opening brace should be on a new line
 13 | ERROR | Closing parenthesis of a multi-line function call must be on a
    |       | line by itself
 ...

Clearly, I’d have to adjust my style to submit code to Zend!

It makes sense however, for a team to define coding guidelines. In fact, the decision as to which set of rules you choose is probably less important than the decision to abide by a common standard in the first place. If a codebase is consistent, then it’s easier to read, and therefore easier to work with. Naming conventions, for example, can help to clarify the purpose of variables or properties.

Coding conventions can play a role in reducing risky or bug-prone code as well.

This is a dangerous area, though. Some style decisions are highly subjective, and people can be disproportionately defensive about their way of doing things. CodeSniffer allows you to define your own rules, so I suggest that you get buy-in from your team on a set of rules so that no one feels that their coding life has become a coding nightmare.

Another benefit of an automated tool is its impersonal nature. If your team does decide to impose a set of coding conventions, it’s arguably better having a humorless script correcting your style than a humorless coworker doing the same thing.

As you might expect by now, I would like to add a CodeSniffer target to my build file:

<target name="sniff" depends="build">
        <mkdir dir="${build}/reports" />
        <phpcodesniffer standard="Zend">
            <fileset dir="${build}/userthing">
                <include name="**/*.php"/>
            </fileset>
            <formatter type="checkstyle" outfile="${build}/reports/checkstyle.xml"/>
            <formatter type="default" usefile="false"/>
        </phpcodesniffer>
    </target>

The phpcodesniffer task will do this job for me. I use the standard attribute to specify Zend rules. I define the files to check using the nested fileset element. I define two formatter elements. The first, with a checkstyle type attribute, will generate an XML file in the reports directory. The second, with  a type of default, outputs summary information to the commandline.

Building a package

Having tested the codebase, and run various reports on it, I should also check that I can build a package.

In this case I will generate a PEAR package using the pearpkg2 task:

<target name="makepackagefile" depends="build">
    <pearpkg2 name="userthing" dir="${build}">
       <option name="packagefile" value="userthing_package.xml"/>
       <option name="packagedirectory" value="${build}"/>
       <option name="baseinstalldir" value="/"/>
       <option name="channel" value="pear.php.net"/>
       <option name="summary" value="blah blah"/>
       <option name="description" value="blah blah blah"/>
       <option name="apiversion" value="1.1.0"/>
       <option name="apistability" value="beta"/>
       <option name="releaseversion" value="${version}"/>
       <option name="releasestability" value="beta"/>
       <option name="license" value="none"/>
       <option name="phpdep" value="5.4.0"/>
       <option name="pearinstallerdep" value="1.4.6"/>
       <option name="packagetype" value="php"/>
       <option name="notes" value="notes notes notes"/>
       <mapping name="maintainers">
           <element>
               <element key="handle" value="mattz"/>
               <element key="name" value="matt"/>
               <element key="email" value="[email protected]"/>
               <element key="role" value="lead"/>
           </element>
       </mapping>
       <fileset dir="${build}">
       <include name="userthing/**" />
       </fileset>
    </pearpkg2>
</target>

The various elements in this should be pretty self-explanatory, especially with reference to the package file structure that you encountered in Chapter 15. Note that I use the version property that I created at the head of the build file to set the value of the releaseversion option element. Also note that I specify the files to include in the package using a fileset element.

Thanks to the packagefile option, the makepackagefile target will generate a file named userthing_package.xml. In order to transform this into an actual package I need to resort to exec again:

<target name="buildpackage" depends="makepackagefile">
    <exec dir="build" checkreturn="true"  executable="/usr/bin/pear">
        <arg value="package" />
        <arg value="userthing_package.xml" />
    </exec>
</target>

This use of exec is equivalent to typing:

pear package userthing_package.xml

from within the build directory. The result is an archive named userthing-1.1.1.tgz.

Notice that I used an attribute named checkreturn within the exec element. This directive causes Phing to alert you if the command you are calling fails to return a successful status code. Without this attribute, exec could fail silently if, for example, the pear executable is not found in the path I specified. Let’s run the buildpackage target:

$ phing buildpackage

Buildfile: /home/mattz/work/userthing/build.xml
 
userthing > build:
 
userthing > makepackagefile:
 
 [pearpkg2] Creating [default] package.xml file in base directory.
Analyzing userthing/domain/User.php
Analyzing userthing/persist/UserStore.php
Analyzing userthing/util/Validator.php
userthing > buildpackage:
 
BUILD FINISHED
 
Total time: 1.7685 second

Let’s test the package by deploying it:

$ pear install --force build/userthing-1.1.1.tgz

install ok: channel://pear.php.net/userthing-1.1.1

So I have a lot of useful tools that I can use to monitor my project. Of course, left to myself I’d soon lose interest in running them, even with my useful Phing build file. In fact, I’d probably revert to the old idea of an integration phase, and pull out the tools only when I’m close to a release, by which time their effectiveness as early warning systems will be irrelevant. What I need is a CI server to run the tools for me.

Test Failures

So far everything seems to be going well, even if userthing won’t be finding its way into the Zend codebase any time soon. But tests succeed when they fail, so I’d better break something to make sure that Jenkins reports on it.

Here is a part of a class named Validate in the namespace userthingutil:

public function validateUser( $mail, $pass ) {
    // make it always fail!
    return false;
 
    $user = $this->store->getUser( $mail );
    if ( is_null( $user ) ) {
        return null;
    }
 
    if ( $user->getPass() == $pass ) {
        return true;
    }
    $this->store->notifyPasswordFailure( $mail );
    return false;
}

See how I’ve sabotaged the method? As it now stands, validateUser() will always return false.

Here’s the test that should choke on that. It’s in test/ValidatorTest.php:

public function testValidate_CorrectPass() {
    $this->assertTrue(
        $this->validator->validateUser( "[email protected]", "12345" ),
        "Expecting successful validation"
         );
}

Having made my change, all I need do is commit, and wait. Sure enough, before long, the project status shows a build marked by an alarming red icon. Once I click on the build link, I find more details. You can see the screen in Figure 20-12.

I also receive a mail with the subject line Build failed in Jenkins: userthing #8. This provides information about my latest commit, the build process, and my test failures.