The coverage tool can generate an XML coverage report in Cobertura format (http://cobertura.sourceforge.net/). This is useful if we want to process the coverage information in another tool. In this recipe, we will see how to use the coverage command-line tool, and then view the XML report by hand.
It's important to understand that reading a coverage report without reading the source code is not very useful. It may be tempting to compare two different projects based on the coverage percentages. But unless the actual code is analyzed, this type of comparison can lead to faulty conclusions about the quality of the software.
For example, a project with 85 percent coverage may appear, on the surface, to be better tested than one with 60 percent. However, if the 60 percent application has much more thoroughly exhaustive scenarios – as they are only covering the core parts of the system that are in heavy use – then it may be much more stable than the 85 percent application.
With these steps, we will discover how to create an XML report using the coverage tool, consumable by other tools:
- Generate coverage metrics by following the steps in Installing and running coverage on your test suite recipe (mentioned in Chapter 1, Using Unittest to Develop Basic Tests) and only running the first test suite (which has resulted in less than 100 percent coverage).
- Generate an XML report by typing:
coverage xml. - Open
coverage.xmlusing your favorite text or XML editor. The format of the XML is the same as Cobertura—a Java code coverage analyzer. This means that many tools, like Jenkins, can parse the results.
The coverage tool has a built-in feature to generate an XML report. This provides a powerful way to parse the output using some type of external tool.
Tip
In the previous screenshot, I opened it using SpringSource Tool Suite (you can download it from http://www.springsource.com/developer/sts), partly because I happen to use STS every day, but you can use any text or XML editor you like.
XML is not the best way to communicate coverage information to users. Generating an HTML report with coverage is a more practical recipe when it comes to human users.
What if we want to capture a coverage report and publish it inside a continuous integration system like Jenkins? All we need to do is install the Cobertura plugin (refer https://wiki.jenkins-ci.org/display/JENKINS/Cobertura+Plugin), and this report becomes traceable. Jenkins can nicely monitor trends in coverage and give us more feedback as we develop our system.