The mappings mentioned previously are stored in Python files called URLconfs. These files must expose a urlpatterns object, which should be the result of a patterns function defined by Django. The patterns function, when called, consists of the following:

Here’s an example to make sense of this, using an extended version of the URLconf from our blog application in Chapter 2, “Django for the Impatient: Building a Blog”:

from django.conf.urls.defaults import *

urlpatterns = patterns('myproject.myapp.views',
    (r'^$', 'index'),
    (r'^archives/(?P<year>d{4})/(?P<month>d{2})/(?P<day>d{2})/$', 'archive'),
)

Clearly, the regexes (first introduced in Chapter 1, “Practical Python for Django”) are the star of the show here. Aside from the lack of leading slashes (which are omitted due to their omnipresence in URLs), the primary thing to look for is the regex characters denoting the beginning and end of a string, ^ and $, respectively.

In practice, the ^ is nearly always required to remove ambiguity about what to match. The URL /foo/bar/ is not the same as /bar/, but the URL-matching regex r'bar/' would match both; r'^bar/' is more specific and matches only the latter.

The $ character is also used more often than not, for the same reason. It ensures the regex only matches the end of a URL and not the middle. However, $ is omitted for URL items designed to include other URLconf files because a URL leading into an include is not the final URL, but only part of one.

Another aspect of these regexes is they can use a part of regex syntax called symbolic groups (the parentheses-clad expressions beginning with ?P<identifier>) to capture parts of the URL that vary. This feature provides the power necessary to define dynamic URLs. In the previous example, we have an archives section of a blog, addressing individual entries based on their date. The fundamental information being accessed by the URL—a blog entry—does not change; only the date being accessed. As we see later, the values captured here are passed to the specified view function, which can use them in a database query or however else it sees fit.

Finally, once you’ve defined the regex, you simply need to note the function it’s linking to and possibly some extra keyword parameters (stored here as a dict). The first argument to patterns, if not empty, is prepended to the function string. Going back to the example for this section, you note the prefix string is 'myproject.myapp.views'; as a result, the full Python module paths go from 'index' and 'archive' to 'myproject.myapp.views.index' and 'myproject.myapp.views.archive', respectively.

A relatively late addition to Django’s URL dispatching mechanism is the url method, which is designed to replace the tuples outlined previously while remaining nearly identical in structure. It takes the same three “arguments”—a regex, a view string/function, and optional argument dict—and adds another optional, named argument: name.name is simply a string, which should be unique among all your URLs; then, you can use it elsewhere to refer back to this specific URL.

Let’s rewrite the previous example using url.

from django.conf.urls.defaults import *

urlpatterns = patterns('myproject.myapp.views',
    url(r'^$', 'index', name='index'),
    url(r'^archives/(?P<year>d{4})/(?P<month>d{2})/(?P<day>d{2})/$', 'archive',
        name='archive'),
)

As you can see, it’s a simple drop-in replacement for the older tuple syntax. Because it’s an actual function and not just a tuple, it enforces what used to be merely a convention. The first two arguments are required and have no name, although the argument dict is now an optional named argument, kwargs, along with the new optional named argument, name.

We’re presenting this url approach after the tuple-based syntax because it’s newer; even by the time you read this, there are still likely to be more Django URLconfs out in the wild utilizing tuples than using url. However, we strongly encourage you to use url in your own code; we have endeavored to set a good example by using it in the rest of this book because it offers more power and flexibility than the tuple approach.

Finally, see the example applications in Part III, “Django Applications by Example,” for more information on the name argument and how it can be used to reference back to your URLs from other parts of the code.

One trick commonly used by Django developers is that of refactoring their URLconfs into multiple patterns calls per URL file, at least for files which have a nontrivial number of entries. This is possible because the return type of patterns is an internal Django object type that can be appended to as if it were a list or other container type. As such, it’s easy to concatenate multiple such objects together, and thus it’s possible and desirable to segregate them based on the prefix string. Here’s a semi-abstract example representing a top-level URL tying together multiple apps.

from django.conf.urls.defaults import *

urlpatterns = patterns('myproject.blog.views',
    url(r'^$', 'index'),
    url(r'^blog/new/$', 'new_post'),
    url(r'^blog/topics/(?P<topic_name>w+)/new/$', 'new_post'),
)

urlpatterns += patterns('myproject.guestbook.views',
    url(r'^guestbook/$', 'index'),
    url(r'^guestbook/add/$', 'new_entry'),
)

urlpatterns += patterns('myproject.catalog.views',
    url(r'^catalog/$', 'index'),
)

Note the use of the += operator for the second and third calls to patterns. By the end of the file, urlpatterns contains a conglomerate of all six defined URLs, each with their own distinct mappings thanks to the different prefix arguments. Of course, astute readers notice this still isn’t the height of refactoring. The “blog,” “guestbook” and “catalog” sections of the URL definitions are themselves slightly repetitive. Next, we cover how to streamline even further by including other URLconfs.

The refactoring mindset seen in the previous section can be further applied, by breaking up URLconf files into multiple such files. This is most commonly seen in projects consisting of multiple apps, where there can be a “base” app defining an index page or other site-wide features such as authentication. The base application’s URLconf then defines the subsections filled in by other apps and uses a special include function to pass off further URL dispatching to said apps, as seen here in an update to the previous example.

## urls.py

from django.conf.urls.defaults import *

urlpatterns = patterns('myproject.blog.views',
    url(r'^$', 'index'),
    url(r'^blog/', include('myproject.blog.urls'),
)

urlpatterns += patterns('',
    url(r'^guestbook/', include('myproject.guestbook.urls')),
)

urlpatterns += patterns('',
    url(r'^catalog/', include('myproject.catalog.urls')),
)

## blog/urls.py

urlpatterns = patterns('myproject.blog.views',
    url(r'^new/$', 'new_post'),
    url(r'^topics/(?P<topic_name>w+)/new/$', 'new_post'),
)

## guestbook/urls.py

urlpatterns += patterns('myproject.guestbook.views',
    url(r'^$', 'index'),
    url(r'^add/$', 'new_entry'),
)

## catalog/urls.py

urlpatterns += patterns('myproject.catalog.views',
    url(r'^$', 'index'),
)

This example is actually a bit larger than the previous one, but hopefully you can see the benefits for a realistic Web site with dozens of potential URLs for each section; in each scenario, we save a decent amount of typing and repetition of the “blog,” “guestbook,” and “catalog” parts of the URL definitions. Specifically, we now have a multi-app Web site delegating most URLs to its subapplications, with the exception of the index page, which lives in the blog application (although you can make a base or similar application for such things—it’s entirely up to you).

URLconf including can be valuable even within single applications—there’s no hard limit on when to use multiple apps versus individual ones, so it’s entirely possible to have a Django app with hundreds of URLs. Most developers, in such a situation, would quickly start organizing things into modules, and URLconf including supports this as well. In general, the organization of your site is up to you, and the URLconf mechanisms have been set up to be as flexible as possible, in which includes play a large part.

Throughout this section, we’ve been using strings to denote the Python module path to the view functions to which our URLs link. However, that’s not the only way to go about it; Django also enables you to pass in a callable object in place of a string, such as:

from django.conf.urls.defaults import *
from myproject.myapp import views

urlpatterns = patterns('', # Don't need the prefix anymore
    url(r'^$', views.index),
    url(r'^blog/', include('myproject.blog.urls')),
)

This opens up the door to a lot of functionality, such as using decorator functions to wrap generic views or even creating your own callable objects to do more complex delegation to different views. See Chapter 11, “Advanced Django Programming,” for more on decorators and other tricks usable with callable views.

Once you have your URLconfs set up, you need to define what behavior the URLs exhibit. We go over the details of view methods in a short while, but for now we show you the layout of the HTTP request and response objects, which those views are dealing with. All view functions accept a “request” parameter, which is an HttpRequest object, a nicely packaged set of attributes representing the raw HTTP request handed down from the Web server.

def userlist(request):
    return paginated_userlist_page(page=request.GET['page'])

At this point, you’ve read about the information that is passed into our view function; now we examine what it’s responsible for returning, namely a response. From our point of view, responses are simpler than requests—their primary data point is the body text, stored in the content attribute. It’s usually a large string of HTML, and it’s so central to HttpResponse objects there are a couple of ways of setting it.

The most common method is via the act of creating the response object—HttpResponse takes a string as a constructor argument, which is then stored in content.

response = HttpResponse("<html>This is a tiny Web page!</html>")

Just with that, you’ve got a fully functioning response object, one worthy of being returned farther up the stack to the Web server and forwarded to the user’s browser. However, it’s sometimes useful to build the response content piece-by-piece; to support this, HttpResponse objects implement a partial file-like behavior, notably the write method.

response = HttpResponse()
response.write("<html>")
response.write("This is a tiny Web page!")
response.write("</html>")

Of course, this means you can use an HttpResponse for any code that expects a file-like object—for example, the csv module’s CSV writing utilities—that adds a lot of flexibility to the process of generating the information your code returns to end users.

Another key aspect of response objects is the capability to set HTTP headers by treating the HttpResponse object like a dictionary.

response = HttpResponse()
response["Content-Type"] = "text/csv"
response["Content-Length"] = 256

Finally, Django provides a number of HttpRequest subclasses for many common response types, such as HttpResponseForbidden (which uses the HTTP 403 status code) and HttpResponseServerError (similar but for HTTP 500 or internal server errors).

Although the basic flow of a Django application is fairly simple—take in request, find appropriate view function, return a response—extra layers are available that can be leveraged to add a lot of power and flexibility. One of these extra layers is middleware—Python functions executed at various points in the previous process that can alter the effective input (by modifying the request before it reaches the view) or output (modifying the response created by the view) of the entire application.

A middleware component in Django is simply a Python class which implements a certain interface, namely it defines one of a number of methods with names such as process_request or process_view. (We examine the most commonly used ones in the following subsections.) When listed in the MIDDLEWARE_CLASSES tuple in your settings.py file, Django introspects the middleware class and calls its method at the appropriate time. The order of classes listed in your settings file determines the order in which they are executed.

Django comes with a handful of built-in middleware, some of which are generally useful and others which are required for specific “contrib” applications such as the authentication framework. See the official Django documentation for more on these.

from some_exterior_auth_lib import get_user

class ExteriorAuthMiddleware(object):
    def process_request(self, request):
        token = request.COOKIES.get('auth_token')
        if token is None and not request.path.startswith('/login'):
            return HttpResponseRedirect('/login/')
        request.exterior_user = get_user(token)

class TextFilterMiddleware(object):
    def process_response(self, request, response):
        response.content = response.content.replace('foo', 'bar')

At the heart of it, Django views are Python functions, plain and simple. The only restriction on view functions is they must take an HttpRequest object and return an HttpResponse object, which is described previously. Also previously mentioned are the regex patterns from the URLconfs, in which you can define named groups. Combined with an optional dictionary parameter, they provide the arguments to the view function, as in the following example (slightly altered from its earlier incarnation):

urlpatterns = patterns('myproject.myapp.views',
    url(r'^archives/(?P<year>d{4})/(?P<month>d{2})/(?P<day>d{2})/$', 'archive',
        {'show_private': True}),
)

Combined with the HttpRequest object, the archive view function referenced by the previous URL could have a signature such as:

from django.http import HttpResponse

def archive(request, year, month, day, show_private):
    return HttpResponse()

As long as it returns an HttpResponse of some kind, the inner guts of the method are inconsequential; what we’re seeing here is essentially an API. As with any API, you can use prewritten code implementing it or write your own from scratch. We examine your options in that order.

Possibly the most-touted aspect of Django, and Web frameworks in general, is the capability to use predefined code for the so-called CRUD operations that make up most of the average Web application. CRUD stands for Create, Read (or Retrieve), Update, and Delete, the most common actions taken in a database-backed application. Showing a list of items or a detail page for a single object? That’s Retrieve. Displaying an edit form and altering the database when it’s submitted? That’s Update or Create, depending on your application and the form in question. Delete should need no explanation.

These tasks and their variants are all provided for by Django’s set of generic views. As previously shown, they are simply Python functions, but ones that are highly abstracted and parameterized to achieve maximum flexibility in their defined role. Because they handle the logic, framework users simply need to refer to them in their URLconf files, pass the appropriate parameters, and make sure a template exists for the view to render and return.

For example, the object_detail generic view is intended to facilitate display of a single object and takes its parameters from both the URL regex and the argument dictionary to do so:

from django.views.generic.list_detail import object_detail
from django.conf.urls.defaults import *

from myproject.myapp.models import Person

urlpatterns = patterns('',
    url(r'^people/(?P<object_id>d+)/$', object_detail, {
        'queryset': Person.objects.all()
    })
)

In the previous example, we’ve defined a regex that matches URLs such as /people/25/, where 25 is the database ID of the Person record we want to display. The object_detail generic view needs both an object_id argument and a QuerySet it can filter to find the object identified by that ID. In this case, we provide object_id via the URL and the queryset via the argument dictionary.

Generic views often expose a handful of options. Some are specific to that view; although others are global, such as a template_name argument enabling the user to override the default location of the view’s template or an extra_context dict which enables the user to pass extra information into the template’s context. (See Chapter 6, “Templates and Form Processing,” for more on templates and contexts.) You can see the official Django documentation for details on all the generic views and their arguments; we go over some of the more commonly used ones next. Note generic views are organized in a two-level module hierarchy for neatness’ sake.

Generic views are both a blessing and a curse. The blessing aspect should be obvious; they save a lot of time and can be used to cut out almost all the work involved in simple or moderately complex views. Their usefulness is further expanded by wrapping them within custom views, as we outline next. However, generic views’ usefulness can make it difficult to accept that sometimes; you just have to write your own completely custom view from scratch, even if the generic view closest to your vision would get you 90 percent of the way there. Knowing when to throw in the towel and go the custom route is a valuable skill, which, like many aspects of software development, can only truly be picked up with experience.

There are times when generic views on their own, called straight from a URLconf file, do not suffice. Often, this requires a completely custom view function to be written, but equally often, a generic view can still be leveraged to do the grunt work depending on the logic required.

The most common use of such “semi-generic” views, in our experience, has been to work around an inherent limitation in the URLconf itself. You can’t perform logic with the captured URL parameters until the regex has been parsed. This limitation exists due to the way URLconfs are designed, and it’s easy to work around it. Consider the following snippet combining portions of a URLconf file and a view file:

## urls.py

from django.conf.urls.defaults import *

urlpatterns = patterns('myproject.myapp.views',
    url(r'^people/by_lastname/(?P<last_name>w+)/$', 'last_name_search'),
)

## views.py

from django.views.generic.list_detail import object_list
from myproject.myapp.models import Person

def last_name_search(request, last_name):
    return object_list(request,
        queryset=Person.objects.filter(last__istartswith=last_name)
    )

As you can see, although our function takes the last_name argument defined as a named group in the URL regex, we’re still delegating 99 percent of the actual work to the generic view. This is possible because generic views are normal Python functions and can be imported and called as such. It’s easy to fall into the trap of thinking about the framework as its own language, but as we’ve emphasized before, it’s all just Python, and this sort of trick shows why that’s a good thing.

Finally, as mentioned earlier, it’s sometimes the case you can’t use generic views at all, which brings us back to the beginning of this section; the view functions as a blank slate, conforming only to a simple API, waiting for you, the programmer, to fill it however you want. We share a couple of observations based on our own experience and point out some convenient shortcut functions supplied by the framework; however, in general, this is an area in which your own skills and experiences determine what you do next.

from django.shortcuts import render_to_response
from django.http import Http404
from myproject.myapp.models import Person

def person_detail(request, id):
    try:
        person = Person.objects.get(pk=id)
    except Person.DoesNotExist:
        raise Http404

    return render_to_response("person/detail.html", {"person": person})

from django.shortcuts import render_to_response, get_object_or_404
from myproject.myapp.models import Person

def person_detail(request, id):
    person = get_object_or_404(Person, pk=id)

    return render_to_response("person/detail.html", {"person": person})

def myview(*args, **kwargs):
    # Here we can refer to e.g. args[0] or kwargs['object_id']