Pylons is one of the older WSGI-based Python web frameworks, dating back to September 2005. Pylons reached version 1.0 in 2010 and is considered very stable at this point. In fact, not much development is planned for Pylons 1.x any more; all new development is happening in Pyramid (see Pyramid and MongoDB for details). The Pylons philosophy is the precise opposite of “one-size-fits-all.” Application developers are free to choose from the various database, templating, session store options available. This kind of framework is excellent when you aren’t exactly sure what pieces you will need when you are starting work on your application. If it turns out you need to use an XML-based templating system, you are free to do so.

The existence of Pyramid aside, Pylons 1.x is a very capable and stable framework. As Pylons is so modular, it is easy to add MongoDB support to it.

First you need to create a virtual environment for your project. These instructions assume you have the virtualenv tool installed on your system. Install instructions for the virtualenv tool are provided in the first chapter of this book.

To create the virtual environment and install Pylons along with its dependencies, run the following commands:

virtualenv --no-site-packages myenv
cd myenv
source bin/activate
easy_install pylons

Now we have Pylons installed in a virtual environment. Create another directory named whatever you like in which to create your Pylons 1.x project, change your working directory to it, then execute:

paster create -t pylons

You will be prompted to enter a name for your project, along with which template engine you want to use and whether or not you want the SQLAlchemy Object-Relational Mapper (ORM). The defaults (“mako” for templating engine, False to SQLAlchemy) are fine for our purposes—not least since we are demonstrating a NoSQL database!

After I ran the paster create command, a “pylonsfoo” directory (I chose “pylonsfoo” as my project name) was created with the following contents:

MANIFEST.in
README.txt
development.ini
docs
ez_setup.py
pylonsfoo
pylonsfoo.egg-info
setup.cfg
setup.py
test.ini

Next you need to add the PyMongo driver as a dependency for your project. Change your working directory to the just-created directory named after your project. Open the setup.py file present in it with your favourite editor. Change the install_requires list to include the string pymongo. Your file should look something like this:

try:
    from setuptools import setup, find_packages
except ImportError:
    from ez_setup import use_setuptools
    use_setuptools()
    from setuptools import setup, find_packages

setup(
    name='pylonsfoo',
    version='0.1',
    description='',
    author='',
    author_email='',
    url='',
    install_requires=[
        "Pylons>=1.0", "pymongo",
    ],
    setup_requires=["PasteScript>=1.6.3"],
    packages=find_packages(exclude=['ez_setup']),
    include_package_data=True,
    test_suite='nose.collector',
    package_data={'pylonsfoo': ['i18n/*/LC_MESSAGES/*.mo']},
    #message_extractors={'pylonsfoo': [
    #        ('**.py', 'python', None),
    #        ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
    #        ('public/**', 'ignore', None)]},
    zip_safe=False,
    paster_plugins=['PasteScript', 'Pylons'],
    entry_points="""
    [paste.app_factory]
    main = pylonsfoo.config.middleware:make_app

    [paste.app_install]
    main = pylons.util:PylonsInstaller
    """,
)

Now you need to fetch the PyMongo driver into your virtual environment. It is easy to do this by executing:

python setup.py develop

Your Pylons app is now ready to be configured with a MongoDB connection. First, we shall create a config file for development

cp development.ini.sample development.ini

Next open the file development.ini in your favourite editor. Underneath the section [app:main] add the following two variables, changing the URI and database names to whatever works for your set up:

mongodb.url = mongodb://localhost
mongodb.db_name = mydb

You can now try starting your project with the following command:

paster serve --reload development.ini

You should see the following output:

Starting subprocess with file monitor
Starting server in PID 82946.
serving on http://127.0.0.1:5000

If you open the URL http://localhost:5000/ in a web browser, you should see the default Pylons page. This means that you have correctly set up your project. However, we do not yet have a way to talk to MongoDB.

Now that the configuration is in place, we can tell Pylons how to connect to MongoDB and where to make the PyMongo connection available to our application. Pylons provides a convenient place for this in <project_name>/lib/app_globals.py. Edit this file and change the contents to the following:

from beaker.cache import CacheManager
from beaker.util import parse_cache_config_options
from pymongo import Connection
from pylons import config

class Globals(object):
    """Globals acts as a container for objects available throughout the
    life of the application

    """

    def __init__(self, config):
        """One instance of Globals is created during application
        initialization and is available during requests via the
        'app_globals' variable

        """
        mongodb_conn = Connection(config['mongodb.url'])
        self.mongodb = mongodb_conn[config['mongodb.db_name']]
        self.cache = CacheManager(**parse_cache_config_options(config))

Once this has been set up, a PyMongo Database instance will be available to your Pylons controller actions through the globals object. To demonstrate, we will create a new controller named “mongodb” with the following command:

paster controller mongodb

You should see a file named mongodb.py in the <project_name>/controllers directory. For demonstration purposes, we shall modify it to increment a counter document in MongoDB every time the controller action is run.

Open this file with your editor. Modify it to look like the following (remembering to change the from pylonsfoo import line into whatever you named your project):

import logging

from pylons import app_globals as g, request, response, session, tmpl_context as c, url
from pylons.controllers.util import abort, redirect

from pylonsfoo.lib.base import BaseController, render

log = logging.getLogger(__name__)

class MongodbController(BaseController):

    def index(self):
        new_doc = g.mongodb.counters.find_and_modify({"counter_name":"test_counter"},
            {"$inc":{"counter_value":1}}, new=True, upsert=True , safe=True)
        return "MongoDB Counter Value: %s" % new_doc["counter_value"]

Once you have saved these changes, in a web browser open the URL http://localhost:5000/mongodb/index. Each time you load this page, you should see a document in the counters collection be updated with its counter_value property incremented by 1.

Pyramid is an unopinionated web framework which resulted from the merge of the repoz.bfg framework into the Pylons umbrella project (not to be confused with Pylons 1.x, the web framework). Pyramid can be considered to be a bit like a Pylons 2.0; it is a clean break, a completely new codebase with no code-level backwards compatibility with Pylons 1.x.

However, many of the concepts are very similar to the older Pylons 1.x. Pyramid is where all the new development is happening, and it has fantastic code test coverage and documentation. This section is only intended to be a brief introduction to setting up a Pyramid project with a MongoDB connection. To learn more, refer to the excellent Pyramid book and other resources available free online at http://docs.pylonsproject.org/.

On its own, Pyramid is just a framework, a set of libraries you can use. Projects are most easily started from a what is known as a scaffold. A scaffold is like a project skeleton which sets up plumbing and placeholders for your code.

A number of different scaffolds are included with Pyramid, offering different persistence options, URL mappers and session implementations. Conveniently enough, there is a scaffold called pyramid_mongodb which will build out a skeleton project with MongoDB support for you. pyramid_mongodb eliminates the need for you to worry about writing the glue code to make a MongoDB connection available for request processing in Pyramid.

As with Pylons 1.x, to start using Pyramid you first need to create a virtual environment for your project. These instructions assume you have the virtualenv tool installed on your system. Install instructions for the virtualenv tool are provided in the first chapter of this book.

To create the virtual environment and install Pyramid and its dependencies, run the following commands:

virtualenv --no-site-packages myenv
cd myenv
source bin/activate
easy_install pyramid

Take note of the line sourcing the bin/activate script. It is important to remember to do this once in every shell to make the virtual environment active. Without this step, your default system Python install will be invoked, which does not have Pyramid installed.

Now your virtual environment has Pyramid and all its dependencies installed. However, you still need pyramid_mongodb and its dependencies like PyMongo etc. Run the following command to install pyramid_mongodb in your virtual environment:

easy_install pyramid_mongodb

With Pyamid and pyramid_mongodb installed in your virtual environment, you are ready to create a Pyramid project with MongoDB support. Decide upon a project directory and a project name. From that project directory execute in the shell:

paster create -t pyramid_mongodb <project_name>

After I ran the paster create command, a “mongofoo” directory (I chose “mongofoo” as my project name) was created with the following contents:

README.txt
development.ini
mongofoo
mongofoo.egg-info
production.ini
setup.cfg
setup.py

The default configuration files tell Pyramid to connect to a MongoDB server on localhost, and a database called “mydb”. If you need to change that, simply edit the mongodb.url and mongodb.db_name settings in the INI-files. Note that if you do not have a MongoDB server running at the address configured in the INI-file, your Pyramid project will fail to start.

Before you can run or test your app, you need to execute:

python setup.py develop

This will ensure any additional dependencies are installed. To run your project in debug mode, simply execute:

paster serve --reload development.ini

If all went well, you should see output like the following:

Starting subprocess with file monitor
Starting server in PID 54019.
serving on 0.0.0.0:6543 view at http://127.0.0.1:6543

You can now open http://localhost:6543/ in a web browser and see your Pyramid project, with the default template. If you made it this far, Pyramid is correctly installed and pyramid_mongodb was able to successfully connect to the configured MongoDB server.

The pyramid_mongodb scaffold sets up your Pyramid project in such a way that there is a PyMongo Database object attached to each request object. To demonstrate how to use this, open the file <project_name>/views.py in your favourite editor. There should be a skeletal Python function named my_view:

def my_view(request):
    return {'project':'mongofoo'}

This is a very simple Pyramid view callable. Pyramid view callables are similar to controller actions in Pylons 1.x, and are where much of the application-defined request processing occurs. Since view callables are passed an instance of a request object, which in turn has a property containing the PyMongo Database object, this is an ideal place to interact with MongoDB.

Imagine a somewhat contrived example whereby we wish to insert a document into a collection called “page_hits” each time the my_view view callable is executed. We could do the following:

import datetime
def my_view(request):
    new_page_hit = {"timestamp":datetime.datetime.utcnow(), "url":request.url}
    request.db.page_hits.insert(new_page_hit, safe=True)
    return {"project":"mongofoo"}

If you now reload the web page at http://localhost:6543 you should see a collection called “page_hits” in the MongoDB database you configured in your INI-file. In this collection there should be a single document for each time the view has been called.

From here, you should be well on your way to building web applications with Pyramid and MongoDB.

Django is proabably the most widely-used Python web framework. It has an excellent community and many plugins and extension modules. The Django philosophy is the opposite of Pylons or Pyramid; it offers one well-integrated package including its own database and ORM layer, templating system, URL mapper, admin interface and so on.

There are a number of options for running Django with MongoDB. Since the Django ORM is such an integral part of Django, there is a project known as Django MongoDB Engine which attempts to provide a MongoDB backend for the Django ORM. However, this approach heavily abstracts the underlying query language and data model, along with many of the low-level details discussed in the course of the book. If you are already familiar with the Django ORM, enjoy using it, and are willing to use a fork of Django, Django MongoDB Engine is worth a look. You can find more information at the website http://django-mongodb.org/.

Our recommended approach for now is to use the PyMongo driver directly with Django. Be aware, however, that with this method, the Django components which depend on the Django ORM (admin interface, session store etc) will not work with MongoDB. There is another project called Mango which attempts to provide MongoDB-backed session and authentication support for Django. You can find Mango at https://github.com/vpulim/mango.

10gen have made a sample Django app with PyMongo integration available. This sample app can be found at https://github.com/mdirolf/DjanMon. We shall step through running the sample Django + MongoDB app on your local machine, and examine how it sets up the MongoDB connection.

First, download the sample Django project. If you already have the git command line tools installed, you can run git clone https://github.com/mdirolf/DjanMon.git. Otherwise, simply click the “Download” button at https://github.com/mdirolf/DjanMon.

In order to successfully run the sample app, you will need to build a Python virtual environment with Django, pymongo and PIL installed. As with Pylons and Pyramid, you will first need to have the virtualenv tool installed on your system—details on how to do this are covered in the first chapter of this book. Once you have virtualenv installed, chose a directory in which to store virtual env, then execute the following shell commands in it:

virtualenv --no-site-packages djangoenv
cd djangoenv
source bin/activate
pip install django pymongo PIL

This will create your virtual environment, activate it and then install Django, the PyMongo driver and the PIL image manipulation library (required by the demo app) into it. Assuming this all succeeded, you are ready to start the sample app development server. Note that the sample app expects a MongoDB server to be running on localhost.

Now we can run 10gen’s Django demonstration app. Change your current working directory to your copy of the “DjanMon” project. There should be a file called manage.py in the current working directory. The app can be run with the Django development server with the command:

python manage.py runserver

You should see output on the console like the following:

Validating models...

0 errors found
Django version 1.3, using settings 'DjanMon.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Now you can open a web browser and visit http://localhost:8000/ and see the demonstration app! The app lets you create simple messages (optionally with attached images) which are persisted in MongoDB.

Let us examine how the sample app works. Take a look at the file status/view.py. This is where the MongoDB connection is created, and where most of the application logic is stored. In their Django + MongoDB integration example, 10gen take a different approach from the others outlined in this chapter. They create a PyMongo Database in the global scope of the views module, rather than attaching it to request objects as in Pyramid or making it a framework-wide global as in Pylons 1.x:

import datetime
import string
import random
import mimetypes
import cStringIO as StringIO

from PIL import Image
from django.http import HttpResponse
from django.http import HttpResponseRedirect
from django.shortcuts import render_to_response
from pymongo.connection import Connection
from pymongo import DESCENDING
import gridfs

db = Connection().sms

This approach is simple and works fine for a demo. However, in larger Django projects with multiple installed applications (in this sample, there is a single installed app—it is named “status”) this would require a separate PyMongo connection pool to be maintained for each app. This results in wasted MongoDB connections and duplicated code. Instead, it would be recommended to create the connection in a single place and import it in any other modules which need access.

This should be enough information to get you started building your Django MongoDB application.

In this book we have tried to give you a solid grasp of how to leverage MongoDB in real-world applications. You should have a decent understanding of how to go about modeling your data, writing effective queries and avoiding concurrency problems such as race conditions and deadlocks. There are a number of other advanced topics which we didn’t have space for in this book but are nonetheless worth looking into as you build your application. Notably, map-reduce enables computing aggregates efficiently. Sharding permits you to scale your application beyond the available memory of a single machine. GridFS allows you to store binary data in MongoDB. Capped Collections are a special type of collection, which look like a circular buffer and are great for log data. With these features at your disposal, Python and MongoDB are extremely powerful tools to have in your toolbox when developing an application.