In Chapter 4, you explored nose, which is an advanced and better framework for Python testing. Unfortunately, nose has not been under active development for the past several years. That makes it an unsuitable candidate for a test framework when you want to choose something for a long-term project. Moreover, there are many projects that use unittest or nose or a combination of both. You definitely need a framework that has more features than unittest, and unlike nose, it should be under active development. nose2 is more of a test-runner for unittest and an almost defunct tool. You need a unit test framework that’s capable of discovering and running tests written in unittest and nose. It should be advanced and must be actively developed, maintained, and supported. The answer is pytest.
This chapter extensively explores a modern, advanced, and better test automation framework, called pytest. First, you’ll learn how pytest offers traditional xUnit style fixtures and then you will explore the advanced fixtures offered by pytest.
Introduction to pytest
pytest is not a part of Python’s standard library. You have to install it in order to use it, just like you installed nose and nose2. Let’s see how you can install it for Python 3. pytest can be installed conveniently by running the following command in Windows:
pip install pytest
For Linux and macOS, you install it using pip3 as follows:
sudo pip3 install pytest
This installs pytest for Python 3. It might show a warning. There will be a directory name in the warning message. I used a Raspberry Pi with Raspberry Pi OS as the Linux system. It uses bash as the default shell. Add the following line to the .bashrc and .bash_profile files in the home directory.
PATH=$PATH:/home/pi/.local/bin
After adding this line to the files, restart the shell. You can now check the installed version by running the following command:
py.test --version
The output is as follows:
pytest 6.2.5
Simple Test
Before you begin, create a directory called chapter05 in the code directory. Copy the mypackage directory as it is from the chapter04 directory. Create a directory called test in chapter05. Save all the code files for this chapter in the test directory.
Just like when using nose, writing a simple test is very easy. See the code in Listing 5-1 as an example.
def test_case01():
assert 'python'.upper() == 'PYTHON'
Listing 5-1
test_module01.py
Listing 5-1 imports pytest in the first line. test_case01() is the test function. Recall that assert is a Python built-in keyword. Also, just like with nose, you do not need to extend these tests from any class. This helps keep the code uncluttered.
Run the test module with the following command:
python3 -m pytest test_module01.py
The output is as follows:
===================== test session starts ====================
=============== 1 passed in 0.04 seconds ===================
You can also use verbose mode as follows:
py.test test_module01.py -v
The output in verbose mode is as follows:
=================== test session starts =======================
platform linux -- Python 3.4.2, pytest-3.0.4, py-1.4.31, pluggy-0.4.0 --
/usr/bin/python3
cachedir: .cache
rootdir: /home/pi/book/code/chapter05/test,
inifile:
collected 1 items
test_module01.py::test_case01 PASSED
==================== 1 passed in 0.04 seconds =================
For the sake of simplicity and convenience, from now onward, you will use the same method to run these tests for rest of this chapter and the book. You will use pytest in the last chapter to implement a project with the methodology of test-driven development. Also, observe when you run your own tests that the output of test execution is in color by default, although the book shows the results in black and white. You do not have to use any external or third-party plugin for this effect. Figure 5-1 shows a screenshot of an execution sample.
Figure 5-1
Sample pytest execution
Test Class and Test Package in pytest
Like all the previous test automation frameworks, in pytest you can create test classes and test packages. Take a look at the code in Listing 5-2 as an example.
class TestClass01:
def test_case01(self):
assert 'python'.upper() == 'PYTHON'
def test_case02(self):
assert 'PYTHON'.lower() == 'python'
Listing 5-2
test_module02.py
Also create an init.py file, as shown in Listing 5-3.
all = ["test_module01", "test_module02"]
Listing 5-3
_init.py
Now navigate to the chapter05 directory:
cd /home/pi/book/code/chapter05
And run the test package, as follows:
py.test test
You can see the output by running the previous command. You can also use the following command to run a test package in verbose mode.
py.test -v test
You can run a single test module within a package with the following command:
py.test -v test/test_module01.py
You can also run a specific test class as follows:
pytest can discover and automatically run the tests, just like unittest, nose, and nose2 can. Run the following command in the project directory to initiate automated test discovery:
py.test
For verbose mode, run the following command:
py.test -v
xUnit-Style Fixtures
pytest has xUnit-style fixtures. See the code in Listing 5-4 as an example.
def setup_module(module):
print("
In setup_module()...")
def teardown_module(module):
print("
In teardown_module()...")
def setup_function(function):
print("
In setup_function()...")
def teardown_function(function):
print("
In teardown_function()...")
def test_case01():
print("
In test_case01()...")
def test_case02():
print("
In test_case02()...")
class TestClass02:
@classmethod
def setup_class(cls):
print ("
In setup_class()...")
@classmethod
def teardown_class(cls):
print ("
In teardown_class()...")
def setup_method(self, method):
print ("
In setup_method()...")
def teardown_method(self, method):
print ("
In teardown_method()...")
def test_case03(self):
print("
In test_case03()...")
def test_case04(self):
print("
In test_case04()...")
Listing 5-4
test_module03.py
In this code, setup_module() and teardown_module() are module-level fixtures that are invoked before and after anything else in the module. setup_class() and teardown_class() are class-level fixtures and they run before and after anything else in the class. You have to use the @classmethod() decorator with them. setup_method() and teardown_method() are method-level fixtures that run before and after every test method. setup_function() and teardown_function() are function-level fixtures that run before and after every test function in the module. In nose, you need the @with_setup() decorator with the test functions to assign those to the function level-fixtures. In pytest, function-level fixtures are assigned to all test functions by default.
Also, just like with nose, you need to use the -s command-line option to see the detailed log on the command line.
Now run the code with an additional -s option, as follows:
py.test -vs test_module03.py
Next, run the test again with the following command:
py.test -v test_module03.py
Compare the outputs of these modes of execution for a better understanding.
pytest Support for unittest and nose
pytest supports all the tests written in unittest and nose. pytest can automatically discover and run the tests written in unittest and nose. It supports all the xUnit-style fixtures for unittest test classes. It also supports most of the fixtures in nose. Try running py.test -v in the chapter03 and chapter04 directories.
Introduction to pytest Fixtures
Apart from supporting xUnit-style fixtures and unittestfixtures, pytest has its own set of fixtures that are flexible, extensible, and modular. This is one of the core strengths of pytest and why it’s a popular choice of automation testers.
In pytest, you can create a fixture and use it as a resource where it is needed.
In Listing 5-5, fixture01() is the fixture function. It is because you are using the @pytest.fixture() decorator with that. test_case01() is a test function that uses fixture01(). For that, you are passing fixture01 as an argument to test_case01().
Here is the output:
=================== test session starts ======================
platform linux -- Python 3.4.2, pytest-3.0.4, py-1.4.31, pluggy-0.4.0 --
/usr/bin/python3
cachedir: .cache
rootdir: /home/pi/book/code/chapter05/test,
inifile: collected 1 items
test_module04.py::test_case01
In fixture01()...
In test_case01()...
PASSED
================= 1 passed in 0.04 seconds ====================
As you can see, fixture01() is invoked before the test function test_case01(). You could also use the @pytest.mark.usefixtures() decorator, which achieves the same result. The code in Listing 5-6 is implemented with this decorator and it produces the same output as Listing 5-5.
import pytest
@pytest.fixture() def fixture01():
print("
In fixture01()...")
@pytest.mark.usefixtures('fixture01')
def test_case01(fixture01):
print("
In test_case01()...")
Listing 5-6
test_module05.py
The output of Listing 5-6 is exactly the same as the code in Listing 5-5.
You can use the @pytest.mark.usefixtures() decorator for a class, as shown in Listing 5-7.
import pytest
@pytest.fixture()
def fixture01():
print("
In fixture01()...")
@pytest.mark.usefixtures('fixture01')
class TestClass03:
def test_case01(self):
print("I'm the test_case01")
def test_case02(self):
print("I'm the test_case02")
Listing 5-7
test_module06.py
Here is the output:
================== test session starts =======================
platform linux -- Python 3.4.2, pytest-3.0.4, py-1.4.31, pluggy-0.4.0 --
/usr/bin/python3
cachedir: .cache
rootdir: /home/pi/book/code/chapter05/test,
inifile: collected 2 items
test_module06.py::TestClass03::test_case01
In fixture01()...
I'm the test_case01
PASSED
test_module06.py::TestClass03::test_case02
In fixture01()...
I'm the test_case02
PASSED
================ 2 passed in 0.08 seconds ====================
If you want to run a block of code after a test with a fixture has run, you have to add a finalizer function to the fixture. Listing 5-8 demonstrates this idea.
import pytest
@pytest.fixture()
def fixture01(request):
print("
In fixture...")
def fin():
print("
Finalized...")
request.addfinalizer(fin)
@pytest.mark.usefixtures('fixture01')
def test_case01():
print("
I'm the test_case01")
Listing 5-8
test_module07.py
The output is as follows:
================= test session starts ========================
platform linux -- Python 3.4.2, pytest-3.0.4, py-1.4.31, pluggy-0.4.0 --
/usr/bin/python3
cachedir: .cache
rootdir: /home/pi/book/code/chapter05/test,
inifile: collected 1 items
test_module07.py::test_case01
In fixture...
I'm the test_case01
PASSED
Finalized...
============== 1 passed in 0.05 seconds =====================
pytest provides access to the fixture information on the requested object. Listing 5-9 demonstrates this concept.
import pytest
@pytest.fixture()
def fixture01(request):
print("
In fixture...")
print("Fixture Scope: " + str(request.scope))
print("Function Name: " + str(request.function. name ))
print("Class Name: " + str(request.cls))
print("Module Name: " + str(request.module. name ))
============== 1 passed in 0.06 seconds ===================
Scope of pytest Fixtures
pytest provides you with a set of scope variables to define exactly when you want to use the fixture. The default scope of any fixture is the function level. It means that, by default, the fixtures are at the level of function.
The following shows the list of scopes for pytest fixtures:
function: Runs once per test
class: Runs once per class of tests
module: Runs once per module
session: Runs once per session
To use these, define them like this:
@pytest.fixture(scope="class")
Use the function scope if you want the fixture to run after every single test. This is fine for smaller fixtures.
Use the class scope if you want the fixture to run in each class of tests. Typically, you’ll group tests that are alike in a class, so this may be a good idea, depending on how you structure things.
Use the module scope if you want the fixture to run at the start of the current file and then after the file has finished its tests. This can be good if you have a fixture that accesses the database and you set up the database at the beginning of the module and then the finalizer closes the connection.
Use the session scope if you want to run the fixture at the first test and run the finalizer after the last test has run.
There is no scope for packages in pytest. However, you can cleverly use the session scope as a package-level scope by making sure that only a specific test package runs in a single session.
pytest.raises( )
In unittest, you have assertRaises() to check if any test raises an exception. There is a similar method in pytest. It is implemented as pytest.raises() and is useful for automating negative test scenarios.
In Listing 5-10, the line with pytest.raises(Exception) checks if an exception is raised in the code. If an exception is raised in the block of code that include the exception, the test passes; otherwise, it fails.
============== 1 failed, 1 passed in 0.21 seconds =============
In test_case01(), an exception is raised, so it passes. test_case02() does not raise an exception, so it fails. As mentioned earlier, this is extremely useful for testing negative scenarios.
Important pytest Command-Line Options
Some of pytest’s more important command-line options are discussed in the following sections.
Help
For help, run py.test -h. It will display a list with uses of various command-line options.
Stopping After the First (or N) Failures
You can stop the execution of tests after the first failure using py.test -x. In the same way, you can use py.test --maxfail=5 to stop execution after five failures. You can also change the argument provided to --maxfail.
Profiling Test Execution Duration
Profiling means assessing execution of programs for factors like time, space, and memory. Profiling is primarily done to improve programs so that they consume fewer resources while executing. The test modules and suites you write are basically programs that test other programs. You can find the slowest tests with pytest. You can use the py.test --durations=10 command to show the slowest tests. You can change the argument provided to --duration. Try running this command on the chapter05 directory as an example.
JUnit-Style Logs
Frameworks like JUnit (a unit test automation framework for Java) produce the logs for execution in XML format. You can generate JUnit-style XML log files for your tests by running the following command:
py.test --junitxml=result.xml
The XML file will be generated in the current directory.
Conclusion
The following are the reasons I use pytest and recommend that all Python enthusiasts and professionals use it:
It is better than unittest. The resulting code is cleaner and simpler.
Unlike with nose, pytest is still under active development.
It has great features for controlling test execution.
It can generate XML results without an additional plugin.
It can run unittest tests.
It has its own set of advanced fixtures that are modular in nature.
If you are working on a project that uses unittest, nose, or doctest as the test framework for Python, I recommend migrating your tests to pytest.
