Writing Your Own Plugins

Many third-party plugins contain quite a bit of code. That’s one of the reasons we use them—to save us the time to develop all of that code ourselves. However, for your specific coding domain, you’ll undoubtedly come up with special fixtures and modifications that help you test. Even a handful of fixtures that you want to share between a couple of projects can be shared easily by creating a plugin. You can share those changes with multiple projects—and possibly the rest of the world—by developing and distributing your own plugins. It’s pretty easy to do so. In this section, we’ll develop a small modification to pytest behavior, package it as a plugin, test it, and look into how to distribute it.

Plugins can include hook functions that alter pytest’s behavior. Because pytest was developed with the intent to allow plugins to change quite a bit about the way pytest behaves, a lot of hook functions are available. The hook functions for pytest are specified on the pytest documentation site.^[11]

For our example, we’ll create a plugin that changes the way the test status looks. We’ll also include a command-line option to turn on this new behavior. We’re also going to add some text to the output header. Specifically, we’ll change all of the FAILED status indicators to “OPPORTUNITY for improvement,” change F to O, and add “Thanks for running the tests” to the header. We’ll use the --nice option to turn the behavior on.

To keep the behavior changes separate from the discussion of plugin mechanics, we’ll make our changes in conftest.py before turning it into a distributable plugin. You don’t have to start plugins this way. But frequently, changes you only intended to use on one project will become useful enough to share and grow into a plugin. Therefore, we’ll start by adding functionality to a conftest.py file, then, after we get things working in conftest.py, we’ll move the code to a package.

Let’s go back to the Tasks project. In ​Expecting Exceptions​, we wrote some tests that made sure exceptions were raised if someone called an API function incorrectly. Looks like we missed at least a few possible error conditions.

Here are a couple more tests:

Let’s run them to see if they pass:

Let’s run it again with -v for verbose. Since you’ve already seen the traceback, you can turn that off with --tb=no.

And now let’s focus on the new tests with -k TestAdd, which works because there aren’t any other tests with names that contain “TestAdd.”

We could go off and try to fix this test (and we should later), but now we are focused on trying to make failures more pleasant for developers.

Let’s start by adding the “thank you” message to the header, which you can do with a pytest hook called pytest_report_header().

Obviously, printing a thank-you message is rather silly. However, the ability to add information to the header can be extended to add a username and specify hardware used and versions under test. Really, anything you can convert to a string, you can stuff into the test header.

Next, we’ll change the status reporting for tests to change F to O and FAILED to OPPORTUNITY for improvement. There’s a hook function that allows for this type of shenanigans: pytest_report_teststatus():

And now we have just the output we were looking for. A test session with no --verbose flag shows an O for failures, er, improvement opportunities:

And the -v or --verbose flag will be nicer also:

The last modification we’ll make is to add a command-line option, --nice, to only have our status modifications occur if --nice is passed in:

This is a good place to note that for this plugin, we are using just a couple of hook functions. There are many more, which can be found on the main pytest documentation site.^[12]

We can manually test our plugin just by running it against our example file. First, with no --nice option, to make sure just the username shows up:

Now with --nice:

And with --nice and --verbose:

Great! All of the changes we wanted are done with about a dozen lines of code in a conftest.py file. Next, we’ll move this code into a plugin structure.