Web development, at anything but the smallest scale, is software development. If your site takes input from users in some form and does something with that input, it’s a Web application, and its development is—or should be—similar to the development of other kinds of software.
If you came to Web development from the software side, it’s probably obvious to you to use established software development tools and techniques such as version control, bug tracking, and powerful development environments. You can just skim this appendix to make sure you’re not forgetting anything that could make your life easier.
If you’re arriving from the design side of things, some of the things presented can be new to you. The good news is, we guarantee any effort you invest in learning and using them pays off many times over in increased productivity, flexibility, and peace of mind.
If you’re developing any kind of software at all, and you’re not using version control, you’re missing out. Version control systems keep a complete revision history of your project, enabling you to rewind your code to any point in time (for example, to the hour before you made that innocent-looking change that broke something you didn’t notice for a week).
Older version control systems such as SCCS (Source Code Control System) and RCS (Revision Control System) were simple, either maintaining the original versions of files plus deltas (minor changes to files between versions) or vice versa—keeping the latest editions of files and applying “backward deltas.” One limitation of such systems was the controlled files lived on and were modified on the same server.
As software development has progressed, especially in the open source community, it became clear the existing systems were awkward for distributed group development, hence leading to more modern version control systems such as CVS (Concurrent Versions System), an improved and distributed offshoot of RCS, and later, the Subversion project, which was meant to be “a better CVS” and a compelling replacement for it.
Version control systems generally follow a tree analogy, where the original or primary line of development is called the trunk, of which copies can be made—becoming distinct entities in their own right—and then which tend to go off in their own direction. These copies are known as branches, and the decisions about what work belongs in branches and what belongs in the trunk varies from project to project.
One methodology keeps all feature development—development that can break backward compatibility or be otherwise unstable—in the trunk and uses branches to represent releases of the software, which only receive bug-fixes after they diverge from the trunk. Another “opposite” approach keeps the trunk stable and puts all new feature work into branches, which has the benefit of enabling more than one big, earth-shattering feature to be worked on at the same time.
Django itself uses a methodology that exists somewhere in-between these two approaches: It keeps both release branches and feature branches, and the trunk is in a middle ground—neither completely stable nor terribly unstable. Changes that have a large effect on the stability of the framework get their own branches with simpler, less disruptive alterations being performed directly on the trunk.
Having branches diverge from the trunk is useful for segregating copies of the codebase, but it wouldn’t be very useful without a way to get those changes back into the trunk! This is where the tree analogy breaks down a bit: The other main concept in source control is that of merging changes from one branch into another branch.
For example, Django’s forms framework received a major overhaul recently and because it consisted of a large series of changes, it got its own branch. Work went on for some time, resulting in a much-improved version of the framework. During that time period, the trunk kept receiving its own updates in various areas, so not only did the “newforms” branch have its own list of changes compared to the point it diverged at, but there was another list of changes that occurred on the trunk between that point and the present.
There’s a lot of theory surrounding how to reconcile nontrivial sets of changes such as these, but the basic gist is version control tools provide commands to merge such changes together and apply them to one side or the other. In Django’s case, the maintainers ran a couple of commands to update the trunk with the changes from the “newforms” branch, manually dealt with a few places where the version control program was unable to reconcile things, and they were all done.
Now that you’ve got a general idea of what version control entails, let’s explore the two main paradigms in today’s source control techniques, including a brief look at one or two specific version control systems from each.
The big names in open source centralized version control are CVS and Subversion—with the latter steadily supplanting the former. Older systems you can see mentioned include RCS and SCCS. Commercial alternatives in this category include Perforce, IBM Rational ClearCase, and Microsoft Visual Studio Team System.
Subversion (http://subversion.tigris.org/) is the closest thing we have to an industry standard version control system. It’s open source, available for many platforms, has good performance and stability, and is well proven in the real world.
Subversion operates on a centralized model with one master repository that all users connect to. Your checkout of the code can be worked on without a network connection—as in, you can edit the files—but recording your changes to or receiving changes from the repository requires you to be online. Subversion can keep track of what you’ve changed since the last time you fetched an update from the central server, but it doesn’t give you any way to record multiple sets of changes without a connection to the server.
Subversion is the version control system that is currently used for the main Django code repository at http://code.djangoproject.com, as well as by the many Django-related projects hosted at Google Code and elsewhere. Subversion also has excellent integration with the Trac wiki and issue-tracker (see the following for more on Trac).
Decentralized version control is the future. A decentralized system can do everything that a centralized one can, but with the powerful additional feature that every “checkout” of a project is also a full-fledged repository. With distributed version control, you don’t need a connection to a central server to record sets of changes. You simply record them in your local repository. Later, if someone else wants the changes, they can be pushed or pulled across the network.
Open source decentralized version control systems include Git, Mercurial, Darcs, Bazaar, SVK, and Monotone. Commercial systems include BitKeeper and TeamWare.
Mercurial (http://www.selenic.com/mercurial/) is one of the most popular distributed systems in use today. Mercurial is largely written in Python with a bit of C in performance-critical portions. Performance is something that the Mercurial developers take very seriously, which has led to the adoption of Mercurial by some very large projects, including the Mozilla Web browser and Sun’s OpenSolaris effort.
Something of a rival to Mercurial, in that both tools vie with Bazaar for open-source DVC mindshare, is Git (http://git.or.cz). Written in C, by Linus Torvalds and other Linux kernel team members, Git is a Unix-inspired tool and was originally created to help deal with the extremely complex source-control needs of the Linux kernel project, which uses it today. Git has also been embraced by the Ruby on Rails project and many other projects in Rails’ sphere of influence, in addition to the WINE project, X.org (the Linux graphics system), the Fedora Linux distribution, and others.
Here’s a basic walkthrough for the uninitiated of using version control on a Django project. In this example we use Mercurial (the hg command).
We start with a skeletal Django project.
$ django-admin.py startproject stuff_dev_site $ cd stuff_dev_site $ ./manage.py startapp stuff_app $ ls stuff_app __init__.py manage.py settings.pyc urls.py __init__.pyc settings.py stuff_app
Now we turn this working directory into a repository.
$ hg init
And make a basic .hgignore file (either with echo, as the following, or with a text editor) that tells Mercurial to skip over those .pyc bytecode files because they are automatically generated by the Python bytecode compiler and not something we need to separately track.
$ echo ".pyc$" > .hgignore
By default, patterns in an .hgignore file are regular expressions. Now we can go ahead and add files:
$ hg add adding .hgignore adding __init__.py adding manage.py adding settings.py adding stuff_app/__init__.py adding stuff_app/models.py adding stuff_app/views.py adding urls.py
Then, we commit our changes with hg commit and verify that Mercurial recorded them with hg log.
$ hg commit -m "Initial version of my project" No username found, using '[email protected]' instead $ hg log changeset: 0:e991df3d3205 tag: tip user: [email protected] date: Sun Oct 07 13:49:14 2008 -0400 summary: Initial version of my project
Every time you commit to a Mercurial repository, Mercurial records a changeset identified with two different numbers: an incremental integer that is only valid in this repository and a hexadecimal hash that uniquely identifies this changeset no matter where it is pushed or pulled to.
Let’s make a simple change, inspect it, and commit it.
$ echo "I_LIKE_CANDY = True" >> settings.py $ hg status M settings.py $ hg commit -m "Apparently someone likes candy." No username found, using '[email protected]' instead $ hg log changeset: 1:65e7cda9f64b tag: tip user: [email protected] date: Sun Oct 07 13:57:53 2008 -0400 summary: Apparently someone likes candy. changeset: 0:e991df3d3205 user: [email protected] date: Sun Oct 07 13:49:14 2008 -0400 summary: Initial version of my project
Once you have recorded a changeset, it remains available as a “snapshot” of your project from that point forward. If you decide that everything you did after changeset 0 was a terrible mistake, then the command hg revert --rev 0 --all rewinds your working directory to that point.
So at this point you are off and running with your project. You write code, you test with the development server, and you record your changes with concise yet meaningful commit messages. Soon it’s time to deploy your project.
You could just tar all the files into an archive, copy that archive to the server where your live application is deployed, and extract the files from the archive. That wouldn’t be so bad—if you never made any changes to your application ever again. But assuming you are actually developing software rather than creating marble busts, it’s likely that you’ll want to correct and enhance your handiwork, and for repeated rounds of updates the tar-copy-untar cycle is a pain.
It would be much better if you could make a quick copy of the repository, but have Mercurial remember where the copy came from so you could incrementally fetch any updates that were made to the original. That’s exactly what the hg clone command does.
For simplicity in this example, we assume the deployed copy of the Web site lives on the same server as the development copy. This means that making the clone is as easy as changing our working directory to the spot we would like to create the clone, and then typing hg clone and the path to the original (development) directory. For example, if we were still inside the original directory and wanted the deployment version to be a copy on the same level as that original directory, our commands would be like this:
$ cd .. $ hg clone stuff_dev_site stuff_live_site 8 files updated, 0 files merged, 0 files removed, 0 files unresolved
This gives us a perfect copy (“clone”) of our original repository and working directory.
$ ls stuff_live_site/ __init__.py manage.py settings.py stuff_app urls.py
Wait, where are our .pyc files? Well, we told Mercurial to ignore them, so they didn’t get added to the repository, and they don’t show up in this list.
Now let’s make some changes to the original (development) directory.
$ cd stuff_dev_site $ echo "I_LIKE_DOGS = True" >> settings.py $ hg commit -m "Also, dogs are liked."
When we’ve sufficiently tested those changes and are ready to deploy, we switch to the “live” branch and pull them in:
$ cd ../stuff_live_site $ hg pull -u pulling from /stuff_dev_site searching for changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
We can use the same hg revert command to undo changes to the live site—for example, we can do that if the change we just pulled turned out to cause unexpected problems on the live site despite our testing on the dev site. Every clone of the project has the same content and the same history.
That’s just the surface, but hopefully it has at least whetted your appetite. There is, of course, much more to using version control in general, and Mercurial in particular, than we have hinted at here. For more, including links to excellent free manuals for Subversion, Mercurial, and other systems, see withdjango.com.
Version controlling your source code is useful, and some developers make do with a version control system and nothing more. However, many others use a breed of Web applications typically designed to “partner” with a version control system, providing not only a Web interface to the source repository and its history, but issue or todo-item tracking, documentation, and so forth.
There are a number of such packages in existence; many are hosted services, such as Google Code, SourceForge, Launchpad, and Basecamp. Others are stand-alone applications you can host yourself; the one favored by the majority of open-source developers is named Trac, which is—not too surprisingly—written in Python.
Trac is an open source wiki, issue-tracking, source code, and project management system, maintained by Edgewall Software. We admit a bias right now: Trac is our favorite in the field of project management software. It works wonderfully out of the box, especially its excellent integration with Subversion and other version control systems. It also has pervasive wiki markup that can be used to easily link code revisions, bug tickets, and pages of documentation and notes.
If that’s not testimonial enough, you should know that the Django code repository itself at http://code.djangoproject.com/ is running on Trac. (You just might not have recognized it due to the lovely style customizations.)
We should also mention that although Trac ships with Subversion support as the default, it is possible to use other systems (such as the ones we described previously) with Trac via plugins. You can download Trac and read the full documentation at http://trac.edgewall.org/. If you find yourself wanting to extend Trac, make sure you check out http://trac-hacks.org/ for extra plugins and other “hacks.”
You don’t need any special software to work on a Django project. Any coder’s text editor will do. Here are some tips for some popular editors.
The most important piece of equipment you need to be a happy and productive Django coder using Emacs is python-mode, which enables syntax highlighting, intelligent indentation, and various other niceties that streamline the production and editing of Python code. Emacs versions 22 and newer come with a built-in version of python-mode. For older versions, or Emacs variants such as XEmacs, see the Emacs page on the official Python Web site (http://www.python.org/emacs/).
There’s also a user-contributed mode for Django templates, which you can find on the Django wiki (http://code.djangoproject.com/wiki/Emacs).
Vim is a text editor based on the old vi Unix tool and provides an incredible number of enhancements and improvements to the original tool. You can read more about it and download it at the main vim Web site at http://www.vim.org/. As with Emacs, there is a Python syntax mode. A comprehensive page of tips on using the vim editor (and its variants) with Python can be found on the Django wiki (http://code.djangoproject.com/wiki/UsingVimWithDjango).
TextMate is a popular commercial editor for OS X that has excellent support for Django. TextMate organizes its support for particular languages and syntaxes into “bundles” and ships with a Python bundle that speeds up working with Python code and makes it more readable through syntax coloring. In addition, in TextMate’s public bundle repository, there are two special bundles for Django: one for Python code and one for Django template code. More information can be found via the Django wiki (http://code.djangoproject.com/wiki/TextMate).
The Eclipse IDE offers a powerful Python development module called PyDev. You can fetch the code and learn more about PyDev via its SourceForge page (http://pydev.sourceforge.net/).