Install the coverage tool and run it against your test suite. Then you can view a report showing what lines were covered by the test suite.
With the following steps, we will build some unit tests and then run them through the coverage tool.
- Create a new file called
recipe52.pyto contain our test code for this recipe. - Write a simple unit test that injects a single, alarming event into the system.
from network import * import unittest from springpython.database.factory import * from springpython.database.core import * class EventCorrelationTest(unittest.TestCase): def setUp(self): db_name = "recipe52.db" factory = Sqlite3ConnectionFactory(db_name) self.correlator = EventCorrelator(factory) dt = DatabaseTemplate(factory) sql = open("network.sql").read().split(";") for statement in sql: dt.execute(statement + ";") def test_process_events(self): evt1 = Event("pyhost1", "serverRestart", 5) stored_event, is_active, updated_services, updated_equipment = self.correlator.process(evt1) print "Stored event: %s" % stored_event if is_active: print "This event was an active event." print "Updated services: %s" % updated_services print "Updated equipment: %s" % updated_equipment print "---------------------------------" if __name__ == "__main__": unittest.main() - Clear out any existing coverage report data using
coverage-e. - Run the test suite using the coverage tool.
gturnquist$ coverage -x recipe52.py Stored event: (ID:1) pyhost1:serverRestart - 5 This event was an active event. Updated services: [{'is_active': True, 'service': {'STATUS': 'Outage', 'ID': 1, 'NAME': u'service-abc'}}, {'is_active': True, 'service': {'STATUS': u'Outage', 'ID': 2, 'NAME': u'service-xyz'}}] Updated equipment: [{'STATUS': 5, 'ID': 1, 'HOST_NAME': u'pyhost1'}] --------------------------------- . ---------------------------------------------------------------------- Ran 1 test in 0.211s OK
- Print out the report captured by the previous command by typing
coverage-r. If the report shows several other modules listed from Python's standard libraries, it's a hint that you have an older version of the coverage tool installed. If so, uninstall the old version by typingpip uninstall coveragefollowed by reinstalling withpip install coverage.
- Create another file called
recipe52b.pyto contain a different test suite. - Write another test suite that generates two faults and then clears them out.
from network import * import unittest from springpython.database.factory import * from springpython.database.core import * class EventCorrelationTest(unittest.TestCase): def setUp(self): db_name = "recipe52b.db" factory = Sqlite3ConnectionFactory(db=db_name) self.correlator = EventCorrelator(factory) dt = DatabaseTemplate(factory) sql = open("network.sql").read().split(";") for statement in sql: dt.execute(statement + ";") def test_process_events(self): evt1 = Event("pyhost1", "serverRestart", 5) evt2 = Event("pyhost2", "lineStatus", 5) evt3 = Event("pyhost2", "lineStatus", 1) evt4 = Event("pyhost1", "serverRestart", 1) for event in [evt1, evt2, evt3, evt4]: stored_event, is_active, updated_services, updated_equipment = self.correlator.process(event) print "Stored event: %s" % stored_event if is_active: print "This event was an active event." print "Updated services: %s" % updated_services print "Updated equipment: %s" % updated_equipment print "---------------------------------" if __name__ == "__main__": unittest.main() - Run this test suite through the coverage tool using
coverage-xrecipe52b.py. - Print out the report by typing
coverage -r.
The first test suite only injects a single alarm. We expect it to cause a service outage as well as taking its related piece of equipment down. Since this would not exercise any of the event clearing logic, we certainly don't expect 100 percent code coverage.
In the report, we can see it scoring network.py as having 65 statements, and having executed 55 of them, resulting in 85 percent coverage. We also see that recipe52.py had 23 statements and executed all of them. This means all of our test code ran.
At this point, we realize that we are only testing the alarming part of the event correlator. To make this more effective, we should inject another alarm followed by a couple of clears to make sure that everything clears out and the services return to operational status. This should result in 100 percent coverage in our simple application.
The second screenshot indeed shows that we have reached full coverage of network.py.
We also see Spring Python reported as well. If we had used any other third-party libraries, then they would also appear. Is this right? It depends. The previous comments seem to indicate that we don't really care about coverage of Spring Python but, in other situations, we may be very interested. And how can the coverage tool know where to draw the line?
In later recipes, we will look into how to be more selective of what to measure so we can filter out noise.
It is true that the unit test isn't adequate with regard to testing the outcome. To draw up this recipe, I visually inspected the output to see whether the network management application was performing as expected. But this is incomplete. A real production grade unit test needs to finish this with a set of assertions so that visual scanning is not needed.
So why didn't we code any? Because the focus of this recipe was on how to generate a coverage report and then use that information to enhance the testing. We covered both of those. By thinking about what was and wasn't tested, we wrote a comprehensive test that shows services going into outage and back to operational status. We just didn't put the finishing touch of confirming this automatically.