Writing a nose extension to pick tests based on regular expressions

Out-of-the-box test tools like nose are very useful. But, eventually, we reach a point where the options don't match our needs. Nose has the powerful ability to code custom plugins, that gives us the ability to fine tune nose to meet our needs. This recipe will help us write a plugin that allows us to selectively choose test methods by matching their method names using a regular expression when we run nosetests.

Getting ready

We need to have easy_install loaded in order to install the nose plugins which we are about to create. If you don't already have it, please visit http://pypi.python.org/pypi/setuptools to download and install the package as indicated at the site.

If you just installed it now, then you will have to:

  • Rebuild your virtualenv used for running code samples in this book
  • Reinstall nose using pip

How to do it...

With the following steps, we will code a nose plugin that picks test methods to run by using a regular expression.

  1. Create a new file called recipe13.py to contain the code for this recipe.
  2. Create a shopping cart application where we can build some tests around it.
    class ShoppingCart(object):
    def __init__(self):
        self.items = []

    def add(self, item, price):
        self.items.append(Item(item, price))
        return self

    def item(self, index):
        return self.items[index-1].item

    def price(self, index):
        return self.items[index-1].price
    def total(self, sales_tax):
        sum_price = sum([item.price for item in self.items])
        return sum_price*(1.0 + sales_tax/100.0)

    def __len__(self):
        return len(self.items)

class Item(object):
    def __init__(self, item, price):
        self.item = item
        self.price = price
  3. Create a test case that contains several test methods, including one that does not start with the word test.
    import unittest

class ShoppingCartTest(unittest.TestCase):
    def setUp(self):
        self.cart = ShoppingCart().add("tuna sandwich", 15.00)

    def length(self):
        self.assertEquals(1, len(self.cart))

    def test_item(self):
        self.assertEquals("tuna sandwich", self.cart.item(1))

    def test_price(self):
        self.assertEquals(15.00, self.cart.price(1))

    def test_total_with_sales_tax(self):
        self.assertAlmostEquals(16.39, 
                                self.cart.total(9.25), 2)
  4. Run the module using nosetests from the command line, with verbosity turned on. How many test methods get run? How many test methods did we define?
    How to do it...
  5. Create a new file called recipe15_plugin.py to write a nose plugin for this recipe.
  6. Capture a handle to sys.stderr to support debugging and verbose output.
    import sys
err = sys.stderr
  7. Create a nose plugin named RegexPicker by subclassing nose.plugins.Plugin.
    import nose
import re
from nose.plugins 
import Plugin

class RegexPicker(Plugin):
    name = "regexpicker"

    def __init__(self):
        Plugin.__init__(self)
        self.verbose = False

    Nose plugin requires a class level name. This is used to define the— with-<name> command-line option.

  8. Override Plugin.options and add an option to provide the pattern on the command line.
        def options(self, parser, env):
        Plugin.options(self, parser, env)
        parser.add_option("--re-pattern",
           dest="pattern", action="store",
           default=env.get("NOSE_REGEX_PATTERN", "test.*"),
           help=("Run test methods that have a method name matching this regular expression"))
  9. Override Plugin.configuration by having it fetch the pattern and verbosity level from the options.
        def configure(self, options, conf):
        Plugin.configure(self, options, conf)
        self.pattern = options.pattern
        if options.verbosity >= 2:
            self.verbose = True
            if self.enabled:
                err.write("Pattern for matching test methods is %s
" % self.pattern)

    When we extend Plugin, we inherit some other features, like self.enabled, which is switched on when –with--<name> is used with nose.

  10. Override Plugin.wantedMethod, so that it accepts test methods that match our regular expression.
        def wantMethod(self, method):
        wanted = 
          re.match(self.pattern, method.func_name) is not None
        if self.verbose and wanted:
            err.write("nose will run %s
" % method.func_name)
        return wanted
    Write a test runner that programmatically tests our plugin by running the same test case that we ran earlier.
if __name__ == "__main__":
    args = ["", "recipe13", "--with-regexpicker", 
            "--re-pattern=test.*|length", "--verbosity=2"]

    print "With verbosity..."
    print "===================="
    nose.run(argv=args, plugin=[RegexPicker()])

    print "Without verbosity..."
    print "===================="
    args = args[:-1]
    nose.run(argv=args, plugin=[RegexPicker()])
  11. Execute the test runner. Looking at the results in the following screenshot, how many test methods run this time?
    How to do it...
  12. Create a setup.py script that allows us to install and register our plugin with nosetests.
    import sys
try:
    import ez_setup
    ez_setup.use_setuptools()
except ImportError:
    pass

from setuptools import setup

setup(
    name="RegexPicker plugin",
    version="0.1",
    author="Greg L. Turnquist",
    author_email="[email protected]",
    description="Pick test methods based on a regular expression",
    license="Apache Server License 2.0",
    py_modules=["recipe13_plugin"],
    entry_points = {
        'nose.plugins': [
            'recipe13_plugin = recipe13_plugin:RegexPicker'
            ]
    }
)
  13. Install our new plugin.
    How to do it...
  14. Run nosetests using --with-regexpicker from the command line.
    How to do it...

How it works...

Writing a nose plugin has some requirements. First of all, we need the class level name attribute. It is used in several places that also includes defining the command-line switch to invoke our plugin, --with-<name>.

Next, we write options. There is no requirement to override Plugin.options but, in this case, we need a way to supply our plugin with the regular expression. To avoid destroying the useful machinery of Plugin.options, we call it first, and then add a line for our extra parameter using parser.add_option.

  • The first, unnamed arguments are string versions of the parameter, and we can specify multiple ones. We could have had -rp and --re-pattern if we wanted to.
  • Dest: This is the name of the attribute that stores the results (see configure).
  • Action: This is specifies what to do with the value of the parameter (store, append, and so on.).
  • Default: This is specifies what value to store when none is provided (notice we use test.* to match standard unittest behavior).
  • Help: Provides help information to print out on the command line.

Nose uses Python's optparse.OptionParser library to define options.

Note

To find out more about Python's optparse.OptionParser please refer to: http://docs.python.org/library/optparse.html.

Then, we write configure. There is also no requirement to override Plugin.configure. Because we had an extra option, --pattern, we need to harvest it. We also want to turn on a flag driven by verbosity, a standard nose option.

There are many things we can do when writing a nose plugin. In our case, we wanted to zero in on test selection. There are several ways to load tests, including by module, and filename. After loading, they are then run through a method where they are voted in or out. These voters are called the want* methods and they include wantModule, wantName, wantFunction, and wantMethod, as well as some others. We implemented wantMethod where we tested if method.func_name matches our pattern using Python's re module. want* methods. These methods have three return value types:

  • True: This test is wanted
  • False: This test is not wanted (and will not be considered by another plugin)
  • None: The plugin does not care. Another plugin (or nose) gets to choose. This can succinctly be achieved by not returning anything from the want* method.

Tip

wantMethod only looks at functions defined inside classes. nosetests is geared to find tests by many different methods and is not confined to just searching subclasses of unittest.TestCase. If tests are found in the module, but not as class methods, then this pattern matching is not utilized. For this plugin to be more robust, we would need a lot of different tests and we would probably need to override the other want* test selectors.

There's more...

This recipe just scratches the surface on plugin functionality. It focuses on the test selection process.

Later in this chapter, we will explore generating a specialized report. This involves using other plugin hooks that gather information after each test is run, as well as generating the report after the test suite is exhausted. Nose provides a robust set of hooks allowing detailed customization to meet our changing needs.

Tip

Plugins should subclass nose.plugins.Plugin

There is a lot of valuable machinery built into Plugin. Subclassing is the recommended means of developing a plugin. If you don't, you may have to add on methods and attributes, which – you didn't realize – were needed by nose and that come for free when you subclass.

It's a good rule of thumb to subclass the parts of the nose API that we are plugging into instead of overriding.

Online documentation of the nose API is a little incomplete. It tends to assume too much knowledge of the reader. If we override and our plugin doesn't work correctly, it may be difficult to debug what is happening.

Tip

Do not subclass nose.plugins.IPluginInterface

This class is used for documentation purposes only. It provides information about each of the hooks our plugin can access. But it is not designed for subclassing real plugins.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.218.212.102