Running Cucumber in Continuous Integration
Many teams set up their continuous integration[55] (CI) server to run Cucumber every time someone shares changes with the rest of the team. Since it’s a command-line tool, there is nothing fancy you need to do—just plop the cucumber command into the CI project’s configuration.
You may want to use slightly different command-line options for CI than you use on your own workstation. You can do this by defining a special profile for the CI build (for example ci).
Being Strict
It’s common to have missing or pending steps while we’re working on a new scenario. Some teams strive to keep the mainline (trunk) of the source repository pristine, without any pending or missing steps.
CI systems detect failure by inspecting the exit status of the processes they are running, and by default Cucumber only exits with an error status (a nonzero value) if there are one or more failing steps. If you are running Cucumber in a CI environment, you may want to set it up to fail if anyone checks in missing or pending steps.
If we pass the --strict option to cucumber, it will exit with a nonzero value if there are any missing or pending steps as well as failing ones. You can inspect the exit status yourself after running cucumber:
| | # OS X or Linux |
| | $ echo $? |
| | |
| | # Windows |
| | $ echo %ERRORLEVEL% |
Sharing Reports
Earlier in this chapter we saw how you can use the --format option to change Cucumber’s output. If you pass --out JUNIT_DIR --format junit to Cucumber in your ci profile, you can configure your CI server to pick up those reports from JUNIT_DIR and analyze them. Some CI servers can generate a trend chart showing you and your team how healthy your build stays over time. This is a great way to get an indicator of whether the project is improving or decaying.
You can also pass the --out HTML_FILE --format html to get a full HTML report of the results. This is particularly handy if you are using a browser automation tool, because you can embed screenshots of the browser right into the HTML report. You can learn more about this in Taking Screenshots.