One of the tenets of the Test-Driven Development methodology is that the developer should write the test code first, run it with the expectation that they will fail (because the actual code doesn't exist yet), write the actual code, and then re-run the tests with the expectation that they will succeed. I've never really been able to get my brain around coding in this fashion... although in the agile development world, writing tests first makes sense, particularly if you're coding without any written specification. Writing tests before writing code forces you to design what you're going to write. In spite of this, I still don't think writing tests first really replaces the need for a spec up front... but to each their own. Personally, I don't start coding until I have a specification and I write my tests after I write the code, but that doesn't make it the right approach 100% of the time. Use whatever techniques you find help you test your code effectively.

Writing tests is all about gauging expectations. When the code you've written runs, what do you expect to happen? At the most basic, here is how you can approach writing tests:

That three-step process is a little esoteric. Let's consider a very simple case to test a customer coming to our site and logging in. We first expect that the user will not be authenticated, so we check their session and make sure that it doesn't contain any authentication data. Then, we emulate the event of the customer logging into the site. Finally, we check that the customer's session now has valid authentication data in it.

While tests can get much more complicated than this, and you'll find yourself testing multiple things in the "before" and "after" stages of each action, the basic strategy remains the same: to determine that your application is responding to inputs the way you expect.

The Django test runner should not run tests on your development or production database. Often, you'll want to run tests that perform creation, manipulation, or deletion of large amounts of data in your database, and you don't want all of these changes to persist after your test suite has run. For this reason, Django creates a separate database on which to run your project's tests. This database is given the same name as your development database, prefaced with test_.

When the Django test runner tries to access to the test database, it will use the same credentials stored for your production database. Right now, the MySQL user in our settings.py file doesn't have permission to create a new database named test_ecomstore, let alone the permissions to create tables and start mucking around with the data. For this to work, we need to first make sure that the MySQL login our project uses has permission to create and manipulate the test database. Inside of MySQL, run the following two commands:

mysql> CREATE DATABASE test_ecomstore;
mysql> GRANT ALL ON test_ecomstore.* TO 'ecomstore'@'localhost';

Now the test database is created, and our project will be allowed to access it when tests are run.

To aid in testing, Python and Django offer default test methods that allow you to check the conditions of your application while the tests are running. These allow you to assert that certain things are true at any given point, or to cause the test to fail if the condition you're testing for is not met.

Some of the more common and useful test methods available on the basic Python unittest.TestCase instance are listed in Table 14-1.

Some of these are functionally equivalent. For instance, using failUnless() and assertTrue() are both testing that the argument you've passed in evaluates to True. Any Python value that evaluates to False will cause the test to fail. When there is more than one option available to you, use the one that's most logical for the given situation. As one simple guideline, try to avoid double negatives. For example, instead of this:

self.failUnless(not logged_in)

just reverse the logic of the process. If you really expect the logged_in variable to have a value of False:

self.assertFalse(logged_in)

This will greatly enhance the readability of your testing code. All of these methods take an optional string parameter after the required arguments, which is a message that will be displayed when the expression fails. For example:

self.assertFalse(logged_in, 'User is logged in!')

There are also available test methods on the enhanced Django TestCase class, which inherits from the Python unittest.TestCase class. These extra methods are designed to help you use the Python unittest base class in order to write tests that check conditions specific to the web environment, such as the flow between pages, form errors given invalid input, and templates used to render view responses:

The good news is that if you're a Python developer and you've been using the unittest base class for writing unit tests, starting to use the Django TestCase subclass for web testing will be a very easy transition for you. If the Django TestCase methods still don't make much sense to you at this point, keep on reading. We're going to use all of them in the tests we're going to write in this chapter.

So when you create your test classes, where should you put them? If you're using at least Django 1.1, the easiest thing to do is simply to put them in the tests.py file that Django creates inside every app in your project. If you're using an earlier version of Django, you can easily create tests.py files for your apps manually. When you run the test runner, it will look in each of your INSTALLED_APP directories and load any test classes that it finds inside your tests.py files.

Inside these files, you define test classes that inherit from the Django TestCase class, and create methods inside each test class. When you run the test runner, each class is instantiated and each of the test methods inside the class runs. Here is how an example test class you define might look:

class MyTestClass(TestCase):
    def test_something(self):
        pass

    def test_something_else(self):
        pass

Test methods are prefaced with test_. Test classes can contain other helper methods that don't start with test_ that can be called by test methods, but these are not called by the test runner directly. In the context of test methods, the pass placeholder keyword for empty Python blocks takes on a logical, more literal meaning.

When you write your tests, it helps immensely if you give them descriptive names that describe exactly what part of your application their testing, as well as the nature of the response that you're expecting. That way, when you run your tests and one of the test methods encounters a failure, you'll have some context about the problem that was encountered just by reading the name of the test method that failed.

First, let's consider the simplest of all cases: a new user comes to the site, not logged in, and loads the catalog home page. What kinds of things can we expect to happen in this case? Well, we expect that the home page will load without any errors, so we can check the status code of the response and make sure that it's the 200 that's indicative of a success. Open the tests.py file in your catalog app. If there is any placeholder code in there already that Django added when it created the app, you can remove all of it and replace it with the following:

from django.test import TestCase, Client
from django.core import urlresolvers

import httplib

class NewUserTestCase(TestCase):
    def test_view_homepage(self):
        client = Client()
        home_url = urlresolvers.reverse('catalog_home')
        response = client.get(home_url)
        # check that we did get a response
        self.failUnless(response)
        # check that status code of response was success
        # (httplib.OK = 200)
        self.assertEqual(response.status_code, httplib.OK)

Inside this new test class, we create a new instance of the Django client test class, Client, which is designed to help simulate the actions of a user. We make our new Client instance request our home page, using the get() method on our test client instance. There are two methods you can call on the test client instance in order to make requests to pages in your test methods: get() and post(). Each makes the request using its corresponding HTTP verb, and both take the URL path of a page on your site. Mostly, we're going to use get() to emulate catalog browsing, but later we'll use post() to test form submissions.

We check to see that a response was returned, and that the status code indicates a success. Notice that we're using the urlresolvers module in order to get the URL path of the homepage before we navigate to it. This is very useful and, just like in our actual web application, makes our code much less prone to 404 Page Not Found errors due to simple typos. Also, if we ever choose to update our URL entries to be something different, the tests won't break because of it.

Also notice that instead of hard-coding the value of 200 to check for the status code, we're falling back on the httplib module to provide the numerical HTTP status codes for us, looking them up by their enumerated names. Table 14-3 shows the names of the status codes that you'll be using most frequently in testing your application.

In order to run these tests and ensure that users can, in fact, load our home page, let's take the test runner for a spin right now. To run the tests for a single app in your project, you can use the syntax manage.py test [app name]. Run the following in your shell:

$ python manage.py test catalog

You should see a lot of code output that's creating all of the tables and indexes for the test database, and then some output about how your tests succeeded. If there were any errors in the syntax of your code, the test runner will fail to start up. You cannot test a project that won't run under normal circumstances.

You can run tests for your entire project by merely omitting the name of the app from this call in your shell. If you run this, however, it may take a little while, as all the tests bundled with Django will be run in addition to your own tests:

$ python manage.py test

Before we go much further, I should point out one very important test method that the Python and Django test cases provide: the setUp() method. When you run your tests, the Django test runner will call the setUp() method on each test class instance before it runs any of the test methods. This is useful in cases where you want to create variables that will be used across multiple test methods.

Take our test Client class: as we start adding test methods, we'll find ourselves creating an instance of this class in every method. Instead of all this repetition, we can simply instantiate the Client class once, in the setUp() method of the class, assign the instance as a variable on the test class, and use that client instance on the class in our test methods. So, to refactor the work we just did:

class NewUserTestCase(TestCase):
    def setUp(self):
        self.client = Client()

    def test_view_homepage(self):
       home_url = urlresolvers.reverse('catalog_home')
        response = self.client.get(home_url)
        # check that we did get a response
        self.failUnless(response)
       # check that status code of response was success
       self.assertEqual(response.status_code, httplib.OK)

The setUp() method can contain assertions and tests for failures itself. As a practical means of demonstrating this point, let's add code that tests to ensure that the user is not authenticated. We can check this by looking at their session data. Authenticated users will have an entry named _auth_user_id in their session dictionary, while anonymous users won't have this yet. We can retrieve this from the Django auth module, by grabbing the SESSION_KEY constant:

from django.contrib.auth import SESSION_KEY

class NewUserTestCase(TestCase):
    def setUp(self):
        self.client = Client()
        logged_in = self.client.session.has_key(SESSION_KEY)
       self.assertFalse(logged_in)

The setUp() method is run before each test method in the class. As an illustration of this, the following test class will run successfully without any failures:

class ExampleTestCase(TestCase):
    def setUp(self):
        self.my_value = 1

    def test_setup_method(self):
        self.my_value = 0

    def test_setup_method_2(self):
        self.assertEqual(self.my_value, 1)
       self.failIfEqual(self.my_value, 0)

If, in the course of running the setUp() method, the test runner encounters any failure or exceptions in the code, the test will return an error response and it will be assumed that the test has failed. There is also a corresponding tearDown() method, which runs after all of the test methods, and is useful for performing any cleanup operations on data that you don't need.

Go ahead and re-run the tests for the catalog app a second time, to make sure that these few additions correctly assert themselves without any unexpected problems.

Now let's test some of the other pages, which are bound to be a bit more complex than the home page. Let's add a single test method for navigating to an active category page. Add this method inside your tests.py file:

from ecomstore.catalog.models import Category

class NewUserTestCase(TestCase):
    ... code omitted here ...

     def test_view_category(self):
         category = Category.active.all()[0]
         category_url = category.get_absolute_url()
         # test loading of category page
         response = self.client.get(category_url)
         # test that we got a response
         self.failUnless(response)
         # test that the HTTP status code was "OK"
         self.assertEqual(response.status_code, httplib.OK)

Now if you go back and run the tests for the catalog app, you should get some interesting results. While the test database is still created and the tests run, you should see a message about a failure we encountered when the test suite was trying to load one of the categories. This is because our test database is empty by default. While the categories table does exist, it doesn't contain any category data. Because we're trying to get the zero-index on an empty result set, an IndexError is raised trying to get the category.

Testing a data-driven dynamic web site without any data won't be sufficient for our purposes. In order to populate your test database schema with some actual data you can reference in your test methods, you could create new objects from scratch right in your setUp() method, using the standard Django ORM methods. However, this is a lot of extra typing you need to do, and clutters up your test classes with all kinds of code that does little more than create some basic classes.

A much superior approach to loading initial data is to use fixtures. Fixtures are test files that reside in your project's apps folders, containing JSON or YAML-formatted data that conforms to the schema of your database tables.

Django provides a means for you to create fixtures from your test database, by simply dumping the data from the database you've been using for development all along. If, at this point, your product database already contains thousands of products, you might not want to run this, since it might take a while to run. If that's the case, simply choose another app that has a small amount of data in it, and run this command:

$ python manage.py dumpdata catalog --indent=2

This produces some nicely formatted JSON text that we can use as fixtures in our test cases. The dumpdata command dumps JSON data by default, but you can also choose YAML as the outputted format if you'd prefer. Inside your catalog app, create a new subdirectory called fixtures. Then, go back and run this command to dump the data into that directory, into a file named initial_data.json:

$ python manage.py dumpdata catalog --indent=2 > catalog/fixtures/initial_data.json

This will create a fixtures file that contains all of the categories, products, and product reviews currently in our development database. The file name initial_data (followed by any extension for a data format that Django supports for fixtures, such as .json or .yaml) means that the test runner will load this data into the test database after it creates the test database and before it runs any test methods.

You're free to edit this file by hand. As a matter of fact, in order to test that our ActiveProductManager is working, we should make sure that at least one of the products in our fixtures file has its is_active field set to False. Open the new initial_data.json file, choose one of the catalog.product model records, and make sure that at least one has is_active set to 0, which is False in JSON. (At this point, I'm assuming you have more than one product in your database.) When you run Django tests, any fixtures found in your apps is loaded into the database. Fixtures are loaded before each test is run, so you are free to wipe out all existing data in one of your test methods, even though other subsequent test methods might depend on this data. For example, given an initial_data.json file that contains fixtures with Product model data in them, the following two tests succeed:

from ecomstore.catalog.models import Product

class MyTestCase(TestCase):
    def test_delete_all(self):
        for p in Product.objects.all():
            p.delete()
        # check that the data
        self.assertEqual(Product.objects.all().count(),0)

    def test_products_exist(self):
        self.assertTrue(Product.objects.all().count() > 0)

Fixtures can also be specified in each test class, by passing in a list of fixture file names to a property of the test class called, aptly enough, fixtures. The following will scan the project's fixtures directories for any fixture files with these names:

class MyTestCase(TestCase):
    fixtures = ['products', 'categories']

    def test_something(self):
       # test stuff here

Including the extension of the fixture file is optional. In this case, the test runner will look for any files that have valid fixture extensions, such as .json or .yaml, and load them into the test database before running any test methods for that class.

So, now that we have some fixtures in place, let's get back to the category testing that we were trying to do before. Inside tests.py in your catalog app, add the following lines to the test class we created in the last section:

class NewUserTestCase(TestCase):
    ... code omitted here ...

    def test_view_category(self):
        category = Category.active.all()[0]
        category_url = category.get_absolute_url()
        # test loading of category page
        response = self.client.get(category_url)
        # test that we got a response
        self.failUnless(response)
        # test that the HTTP status code was "OK"
        self.assertEqual(response.status_code, httplib.OK)
       # test that we used the category.html template in response
       self.assertTemplateUsed(response, "catalog/category.html")

Let's have a quick look at template testing. One of the test methods we've been using is assertTemplateUsed(), which takes a response and the path to a template file in our templates directory as an argument. The method checks that the template file specified was used by the view function in generating a response. So far, we've just been putting the name of the template file into the function to check it.

While this works, it's less than ideal because if we ever change the name of the template that the URL is using, it will break the test, even if we correctly update the URL entry and the view function with the view template name. A much better solution is to look up the name of the template being used to render the response by retrieving the value of the template_name keyword argument using the resolve() method on the URL. You can test this inside a Python shell in your project:

>>> from django.core import urlresolvers
>>> urlresolvers.resolve('/product/something/')
(<function show_product at 0x11e0488>, (),
   {'template_name': 'catalog/product.html', 'product_slug': 'some-product'})
>>> urlresolvers.resolve('/product/something/')[2]['template_name']
'catalog/product.html'

So, we can refactor our existing test method to look up the template_name variable on the URL entry and use that value in the line that checks the template used:

from django.core import urlresolvers

class NewUserTestCase(TestCase):
    # other test methods here

    def test_view_category(self):
        category = Category.active.all()[0]
        category_url = category.get_absolute_url()

# get the template_name arg from URL entry
url_entry = urlresolvers.resolve(category_url)
template_name = url_entry[2]['template_name']
  # test loading of category page
  response = self.client.get(category_url)
  # test that we got a response
  self.failUnless(response)
  # test that the HTTP status code was "OK"
  self.assertEqual(response.status_code, httplib.OK)
  # test that we used the category.html template in response
 self.assertTemplateUsed(response, template_name)

This test method is pretty good, but at the moment, it's really only testing that Django is successfully generating and returning the response with the correct template file. There's nothing here to test that the content on the page is what we expect it to be. This is what the assertContains() method is used for. This method takes two required arguments: a response object and a string that you expect to occur in the document body of the response. If the test method finds the string in the content of the response, the test succeeds. We can put this to good use by testing that the category page contains the category name and the category description:

def test_view_category(self):
        category = Category.active.all()[0]
        category_url = category.get_absolute_url()
        # get the template_name arg from URL entry
        url_entry = urlresolvers.resolve(category_url)
        template_name = url_entry[2]['template_name']
        # test loading of category page
        response = self.client.get(category_url)
        # test that we got a response
        self.failUnless(response)
        # test that the HTTP status code was "OK"
        self.assertEqual(response.status_code, httplib.OK)
        # test that we used the category.html template in response
        self.assertTemplateUsed(response, template_name)
       # test that category page contains category information
       self.assertContains(response, category.name)
       self.assertContains(response, category.description)

Now that we've got all the basic methods used in test methods covered, let's use all of them
to write a brief test method to ensure that our product view is working correctly. Put the
following method inside your test class:

    def test_view_product(self):
        """ test product view loads """
        product = Product.active.all()[0]
        product_url = product.get_absolute_url()
        url_entry = urlresolvers.resolve(product_url)
        template_name = url_entry[2]['template_name']
        response = self.client.get(product_url)

self.failUnless(response)
  self.assertEqual(response.status_code, httplib.OK)
  self.assertTemplateUsed(response, template_name)
  self.assertContains(response, product.name)
  self.assertContains(response, product.description)

Don't forget to add the Product model to the import statement or else this won't work!

One other issue that you'll encounter is an error with the logging of the product page view. The ProductView model we created in Chapter 9 for tracking the pages that customers are viewing requires a valid IP Address... something that the Django test runner does not simulate on the Client instance. So, in order to make our tests succeed, we need to add a little bit of additional logic to the log_product_view() function in our stats.py file:

v.ip_address = request.META.get('REMOTE_ADDR')
if not request.META.get('REMOTE_ADDR'):
    v.ip_address = '127.0.0.1'
 v.user = None

Test classes can also contain other methods that exist only to act as helper methods for your test methods. Take a look at the lines of code that are getting the name of the template file given the URL; the code to do this might be repeated across several of your test methods. You can create such a method by simply defining a method that doesn't start with test_, and called from your test methods using the self keyword:

def get_template_name_for_url(self, url):
    """ get template_name kwarg for URL """
    url_entry = urlresolvers.resolve(url)
    return url_entry[2]['template_name']

Helper methods on test classes can also contain assertions, and they will cause the test to fail if the assertion fails. However, they won't be called by the test runner by default. You must call them explicitly from within either setUp() or one of your test methods for them to run. For this reason, it's generally best practice to leave your assertions in your test methods and only use helper methods to reduce repetition of common tasks.

The response object also comes equipped with a context property. This is useful for testing the existence of variables that you expect to appear in the returned response. For example, our product view function should be returning a variable called form with each product page response. We can test that this variable is included in the response by asserting that it exists in the response context, and checking that it is an instance of the correct form class:

from ecomstore.catalog.forms import ProductAddToCartForm

# check for cart form in product page response
cart_form = response.context[0]['form']
self.failUnless(cart_form)
# check that the cart form is instance of correct form class
self.failUnless(isinstance(cart_form, ProductAddToCartForm))

There are other variables returned in the product page view that we could also test for in the response context. For example, each product page is supposed to return a variable called product_reviews. Testing for this variable is a little trickier. For instance, the following code will not work:

product_reviews = response.context[0]['product_reviews']
self.failUnless(product_reviews)

This will work if we are certain that the product we've selected for our test has at least one product review. However, if the product has no reviews, then the variable product_reviews will be an empty list, which will cause the failUnless() method to fail in the preceding case. On top of this, if the variable product_reviews is missing from the dictionary, then the first line of the preceding code will raise a KeyError exception.

A much better approach for testing the existence of variables in response contexts is to check that they exist by using the get() function available on Python dictionaries, and returning the Python null value None if the object doesn't exist. Then, check that the return value is not None:

product_reviews = response.context[0].get('product_reviews',None)
self.failIfEqual(product_reviews, None)

One of the assumptions we've made about our code is that inactive categories and products will not be returned to the user interface. We can test this simply by trying to load an inactive product and checking that Django returns a 404 Page Not Found error.

Add the following class to the test file and re-run your tests. Remember to make sure that at least one product in the initial_data.json fixtures file you created earlier is set to inactive:

class ActiveProductManagerTestCase(TestCase):
    def setUp(self):
        self.client = Client()

    def test_inactive_product_returns_404(self):
        """ test that inactive product returns a 404 error """
        inactive_product = Product.objects.filter(is_active=False)[0]
        inactive_product_url = inactive_product.get_absolute_url()
        # load the template file used to render the product page
        url_entry = urlresolvers.resolve(inactive_product_url)
        template_name = url_entry[2]['template_name']
        # load the name of the default django 404 template file
        django_404_template = page_not_found.func_defaults[0]
        response = self.client.get(inactive_product_url)

self.assertTemplateUsed(response, django_404_template)
self.assertTemplateNotUsed(response, template_name)

One thing to notice here is that we cannot test the HTTP status code of the 404 page, because when Django serves up the custom Page Not Found page we created, it actually uses an HTTP status code of 200 and not 404. So, in order to test that the inactive product is not being loaded, we're using the template assertion methods to check the behavior.