So far, we've dealt with a simple deployment situation—a single production machine running Intranet as a Mongrel instance, backed by a MySQL database. This will be able to cope with a fair amount of traffic, and it may be that in your situation, there will be no need for anything more complex.
Having said this, one issue that will definitely arise is how to roll out new versions of the application. This introduces several challenges, such as ensuring the application is compatible with new versions of Rails, upgrading the database, and so on. Capistrano, a tool for simplifying and automating deployment of applications, was designed to ease multiple deployments, and we'll be seeing how it can help us meet these challenges. We'll also cover common deployment issues and their solutions, plus other housekeeping that an application requires, such as clearing out stale sessions and managing log files.
If you have a lot of users connecting to the application concurrently, you may find that things start to slow down. In this chapter, we'll cover how to keep up your application's speed using some simple techniques, such as caching. If things are still too slow, the next option may be to add more hardware to cope with the load. We'll see how to scale a Rails application using Apache, as well as by adding more servers.
This chapter is slightly trickier to follow than previous ones, as we're now considering two machines—one for development and a second for production (called the development machine and production server respectively in this chapter). Intranet is being deployed from the former to the latter. You can still follow the tutorial on a single machine by treating it as both the development machine and production server: simply ensure that you can login to it via SSH. If you want to test the Apache deployment tips, ensure you have Apache installed (see Chapter 6 for some instructions). The section on large-scale deployments is harder to simulate with a single machine, so we'll just be covering some of the concepts to give you an idea of the issues and solutions.
Capistrano is designed to make your life easier by automating the repetitive tasks associated with application deployment. Typically, to get a Rails application into production, we would log in to the production server using SSH, and perform the following actions:
In fact, this is how we did our first production deployment. However, if we have to repeat deployment several times, we need to remember each step and the order in which they should be applied every time, which can be tedious and error-prone.
An alternative is to use Capistrano, which eases the deployment process by automating the above tasks for us. When we trigger it, it will log in to the server, check out the latest code from the Subversion repository, update the database, and restart Mongrel, all using a handful of simple commands. We don't even have to leave our development machine. Even better, it will also make it easy for us to roll back to a previous version of the application if something goes wrong with the deployment. This can of course be done manually, but this requires a precise sequence of steps to be followed—something at which people are inherently bad, but at which computers excel.
In the following sections, we'll see how to set up the deployment environment and deploy Intranet onto the production Linux server. For more details on installation of each component, refer to Chapter 3 Laying the Foundations.
The first step in using Capistrano is to apply it to a Rails application. For example, if the application were in /home/demo/Intranet
you would do:
$ cap --apply-to /home/demo/Intranet
This command adds two files to the application:
lib/tasks/capistrano.rake:
Applying Capistrano to your application makes some extra Rake tasks available (see Running a Migration in Chapter 4). However, these tasks are now deprecated, and Capistrano has its own command-line tool (cap, as used above) to run tasks, so the file can be ignored or deleted.config/deploy.rb:
This file contains the configuration for Capistrano and should not be deleted. It specifies where your Subversion repository is, where your servers (web, application, database) are, how to log in to them via SSH, which directory to deploy the application to, etc. It is also the place where you put custom tasks specific to your application; we'll see some of these later.The next step is to customize the configuration file (config/deploy.rb) to tell Capistrano where the production server is and how to log in to it. Here are the lines that need to be configured, along with some sample data:
set :application, "Intranet" role :web, "192.168.13.129" role :app, "192.168.13.129" role :db, "192.168.13.129", :primary => true
Capistrano can cope with setups where an application is to be deployed to multiple servers, with each server having a different role. The roles available by default in Capistrano (you can define your own on top of these) are:
app:
An application server, running the Rails application. Typically, this will be a machine running Rails under one or more Mongrel processes.web:
A web server serving static files. Some applications serve JavaScripts, stylesheets, and static HTML files from a web server like Apache, running on a completely separate machine from the Rails processes proper.db:
A database server, storing the back-end data for the application. You need to set the :primary => true
option for a role if you want to be able to run migrations automatically through Capistrano.In our case, we're using a single server, which acts as the web server, application server, and database server, as well as the Subversion server. That's why we removed this line from the generated Capistrano file:
role :db, "db02.example.com", "db03.example.com"
You only need multiple role :db
lines if you are using multiple database servers. The default deploy.rb
file also contains multiple specifications for both :web
and :app
servers. As we only have a single server, we can trim those settings down to leave a single host specification 192.168.13.129
(the IP address of our server), as in the example on the previous page. You could use a domain name instead of an IP address here; so, if the IP address 192.168.13.129
were registered with the domain name server.company.local
, we could have configured deploy.rb
with:
set :application, "Intranet" role :web, "server.company.local" role :app, "server.company.local" role :db, "server.company.local", :primary => true
When you ask Capistrano to deploy your application, it will attempt to log in via SSH to the :web, :app
and :db
servers to do the necessary code check out and command-line actions (e.g. run migrations, restart Mongrel) to get your application running. Our recommendation would be to create one user account on each server specifically for Capistrano. The home directory of this account can be used as the deployment location for Rails applications, and the same user account can be used to check out code from the local Subversion repository.
Log in to the server and set up an account specifically for deploying Rails applications (e.g. I called mine captain)
. On a Linux server, you can do this as follows:
$ sudo groupadd captain
$ sudo useradd --create-home -g captain captain
$ sudo passwd captain
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
$ sudo mkdir /home/captain/apps
$ sudo chown captain.users /home/captain/apps
The final command creates a directory called apps
inside captain's
home directory. This is where we will deploy our Rails applications to.
Once you have a Capistrano user on the production server, you need to add its username and password to the deployment recipe (deploy.rb):
set :user, "captain" set :password, "police73"
If you have the expertise, you can set up SSH public key authentication for logins from developer machines to the server instead of using usernames and passwords. This is more secure, as it means you don't need to put usernames and passwords in your deployment recipes. See http://sial.org/howto/openssh/publickey-auth/ for some instructions.
Once Capistrano has logged in via SSH, it will need to check out the code from the Subversion repository. As we are using Subversion over an SSH tunnel (via an svn+ssh
URL), we specify the username as part of the repository URL:
svn+ssh://[email protected]/repository/#{application}/trunk
Each time we deploy the application, we will be prompted for the password of the captain
user. This is because Capistrano has no mechanism for caching passwords between SSH sessions, or for allowing you to specify an svn+ssh
password in the configuration file.
If you get annoyed with continually being prompted by Subversion for passwords, investigate an SSH password-caching tool like Pageant for Windows (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html), or ssh-agent for *nix and Mac systems (included with the OpenSSH tools). Alternatively, you can use the even simpler method of setting up a password-less public key on the client, and registering that key with the server. See the previous tip box for more information.
Another alternative is to enable access to the Subversion repository over HTTP for checkouts. If you're using this configuration, you can add the Subversion credentials directly to deploy.rb:
set :svn_username, "captain" set :svn_password, "police73"
Capistrano will then be able to automatically log in to the repository over HTTP and check out the code, without requiring you to enter a password.
Finally, set the directory on the server into which the Rails application code should be checked out. This will depend on how your application server is set up and the permissions of the person who is deploying the application. As we have a captain
account specifically for our Rails applications, we can use the apps
directory inside that user's home directory as our deployment destination:
set :deploy_to, "/home/captain/apps/#{application}"
This references the application
variable set earlier in the recipe ("Intranet"), specifying that the application is deployed to /home/captain/apps/Intranet
.
The final deployment recipe looks like this:
ip_address = "192.168.13.129" set :application, "Intranet" role :web, ip_address role :app, ip_address role :db, ip_address, :primary => true set :user, "captain" set :password, "police73" set :repository, "svn+ssh://captain@#{ip_address}/repository/#{application}/trunk" set :deploy_to, "/home/captain/apps/#{application}"
We've created a variable ip_address
to store the IP address of our single server, to help prevent typos. We've then referenced it in several places in the recipe.
We are also going to need a production database on the server. Creating a database and user was discussed in Chapter 4 Creating a Database and System Account, and configuring the production environment in the database.yml
file was covered in Chapter 6 The Production Database.
As a quick reminder: in the case of MySQL, we can create the database and user on the server with the mysql
command-line client:
$ mysql -uroot -p
Once into the client, do:
mysql> CREATE DATABASE intranet_production;
mysql> GRANT ALL ON intranet_production.* TO intranet@localhost IDENTIFIED BY 'police73';
mysql> FLUSH PRIVILEGES;
(Replace police73
with the password you want to give to the intranet
user.)
Then we can configure the database.yml
file for the production database like this:
production: adapter: mysql database: 'intranet_production' username: intranet password: police73 host: localhost
With the configuration file edited to the deployment environment, we are ready to set up the required directories on the server. Connect into the RAILS_ROOT
for the Intranet application and run the cap setup
command:
$ cap setup
* executing task setup
* executing "umask 02 &&
mkdir -p /home/captain/apps/Intranet /home/captain/apps/Intranet/releases /home/captain/apps/Intranet/shared /home/captain/apps/Intranet/shared/system &&
mkdir -p /home/captain/apps/Intranet/shared/log &&
mkdir -p /home/captain/apps/Intranet/shared/pids"
servers: ["192.168.13.129"]
[192.168.13.129] executing command
command finished
As you can see, this logs into the application server(s) and creates several directories, which will store releases of the application and related temporary files (logs, pids). The directories (inside /home/captain/apps)
are:
Intranet:
The main directory for the application.Intranet/releases:
Individual releases of the application end up in here. Each time you deploy the application via Capistrano, you add a new directory for that release inside this directory.Intranet/shared:
Each release of the application shares the files in here, which means you have a shared set of log
files, a single location for Mongrel pid
files, etc.Intranet/shared/log:
All releases of the application put their logs into this directory.Intranet/shared/pids:
When the application is running under Mongrel, the Mongrel pid
file should go in this directory.Intranet/shared/system:
This can be used to house files and directories that need to remain constant across releases. We'll be using it shortly as a place to store file uploads (see the previous chapter, where we added this functionality).We can now deploy the application into this directory structure:
$ cap cold_deploy
You should get feedback on the task's progress like this:
* executing task deploy
* executing task update
** transaction: start
* executing task update_code
* querying latest revision...
[email protected]'s password:
...
* executing task spinner
* executing "sudo -u app /home/captain/apps/Intranet/current/ script/spin"
servers: ["192.168.13.129"]
[192.168.13.129] executing command
** [out :: 192.168.13.129] sudo:
** [out :: 192.168.13.129] no passwd entry for app!
command finished
command "sudo -u app /home/captain/apps/Intranet/current/script/spin" failed on 192.168.13.129
You will be prompted for captain's
password at some point; this will be used to log in to the Subversion repository and check out the code.
This almost looks OK. However, an error occurred towards the end of the command's execution, during the spinner
task:
no passwd entry for app!
This is because Capistrano logged in as captain
and tried to run a script inside the deployed Rails application (script/spin) as the :app
user; but the :app
user doesn't exist on the production server. It turns out, this error is irrelevant for our case, as we don't want to start the server using script/spin
; we want to override this behavior to start Mongrel instead (see the section Managing Mongrel from Capistrano). So, we can ignore this error for now. The important thing is that our code has been deployed to the production server. Where is it?
Look inside the apps/Intranet
directory and you'll see that some new files and directories have been created:
current:
This is a symlink (a shortcut in Windows parlance) to the latest release in the releases
directory.revisions.log:
This file contains a record of when releases were made and who made them.releases/yyyymmddhhnnss:
Each sub-directory of the releases
directory contains a version of our application. When a new release of the application is deployed, a new sub-directory is added under the releases
directory. Note that each sub-directory is timestamped with the date and time when the release was deployed.While the releases are independent of each other, each contains more symlinks that point up to the shared directory (see the previous section). This ensures that different releases all share a single set of log files, pid files, etc.
The cold_deploy
task doesn't run the database migrations. So, while our code is deployed, there is no database for it to work with. We can apply our migrations to the production database with another Capistrano task, migrate:
$ cap migrate
* executing task migrate
* executing "cd /home/captain/apps/Intranet/current && rake RAILS_ENV=production db:migrate"
servers: ["192.168.13.129"]
[192.168.13.129] executing command
** [out :: 192.168.13.129] (in /home/captain/apps/Intranet/releases/20070413171324)
** [out :: 192.168.13.129] == CreatePeople: migrating ====================================================
** [out :: 192.168.13.129] -- create_table(:people)
** [out :: 192.168.13.129] -> 0.0831s
** [out :: 192.168.13.129] == CreatePeople: migrated (0.0837s) ===========================================
...
If this works, you should see something like the output on the previous page (truncated for brevity). Note that the production database was used as the target for migrations (Capistrano's default environment).
There is no default Capistrano task for starting Mongrel (but we'll be writing one shortly). However, we can run any arbitrary task on the server using cap invoke
. The invoke
command logs into the server using the username and password set in deploy.rb
; then, on the remote machine, it executes the command passed to it. For example, to start our application, we can do:
$ export COMMAND="cd /home/captain/apps/Intranet/current;
mongrel_rails start -d -e production -p 4000
-P /home/captain/apps/Intranet/shared/pids/mongrel.pid"
$ export ROLES=app
$ cap invoke
First, we set up the command we want to execute by specifying a COMMAND
environment variable. Here, the command we're setting up will run mongrel_rails
to start a Mongrel instance to serve a Rails application; next, we specify the group of machines where we want to run the command using a ROLES
environment variable. To specify multiple roles, place commas between them (e.g. ROLES=web,app)
. Finally we invoke COMMAND
on the remote machines.
Hopefully, you'll see the mongrel_rails
command being executed:
* executing task invoke
* executing "mongrel_rails start -d -e production -p 4000 -P /home/captain/apps/Intranet/shared/pids/mongrel.pid"
servers: ["192.168.13.129"]
[192.168.13.129] executing command
command finished
With the database migrated and Mongrel started, the application is now in production. Use a browser to confirm it is up and running; if not, see the section later in this chapter for some hints about how to troubleshoot your deployment.
In the next section, we'll write a new Capistrano task that encapsulates the manual Mongrel command above.
In the previous section we saw how to do a cold deployment of our code to the server; we also saw that the command threw an error, as Capistrano tried to run the script/spin
command to start the application. This is because Capistrano is running its spinner
task as a sub-task of the cold_deploy
task, which in turn calls script/spin
inside the application.
However, we can override any of Capistrano's default tasks with our own by defining appropriate methods in the deployment recipe. In our case, we want to override the spinner
task, which will be used to cold start a Mongrel instance for the application, and we'll also need to override the restart
task, which stops the application and starts it again (so that any changes to the code are reflected in the production instances). For good measure, we can add our own stop
task (sometimes it's useful to be able to just stop Mongrel) and add an alias, start
, for the spinner task. To define these tasks, add the following into the section commented with TASKS
in config/deploy.rb:
# Where to store Mongrel PID file mongrel_pid = "#{shared_path}/pids/mongrel.pid" # Port to run on port = 4000 desc "Override the spinner task with one which starts Mongrel" task :spinner, :roles => :app do run <<-CMD mongrel_rails start -e production -p #{port} -P #{mongrel_pid} -c #{current_path} -d CMD end desc "Alias for spinner" task :start do spinner end desc "Override the restart task with one which restarts Mongrel" task :restart, :roles => :app do run "mongrel_rails restart -P #{mongrel_pid}" end desc "Stop Mongrel" task :stop, :roles => :app do run "mongrel_rails stop -P #{mongrel_pid}" end
We now have spinner
and restart
tasks that override the defaults provided by Capistrano, plus a start
alias for the spinner
task, and a new stop
task. A few points to note:
run
method can be used to execute a command-line string on the server; here we're executing our custom mongrel_rails
command (see previous section), passing the environment to run in, the port to use, where to place the pid
file, and which application to run. You can use run
to execute any valid command on a deployment server. release_path
and shared_path
. These refer to the absolute path to the latest release of the application and to the shared directory respectively. :roles => :app
option, meaning, we only run them on the application server, where Mongrel is running; if you need to specify multiple roles, pass an array of role names into this option, e.g. :roles => [:app, :web]
.<<-CMD ... CMD
syntax: this is known as heredoc, and can be used to create multi-line strings without the necessity of inserting newline characters and escaping quote marks.Adding these tasks means that Capistrano will call the correct commands when we cold deploy or deploy a new version of the application. We can also run our tasks manually to get "remote control" of our Mongrel servers with:
$ cap start
$ cap stop
$ cap restart
One other issue that isn't immediately obvious is that Intranet's file upload functionality complicates the picture. Currently, uploaded files are stored in the public/files
directory. However, when we upgrade the application, we effectively start from a clean slate again: we get a new public/files
directory which is empty. What we really want is to keep file uploads in a central location available to every release—each time we upgrade, we still reference the same file uploads directory.
Capistrano provides a shared/system
directory for exactly this scenario. Data that is part of the back-end store for the application is used by every release (similar to how we have a single database instances for all releases). In our case, we'll create a files
sub-directory inside shared/system
; then we'll create a symbolic link (like a Windows shortcut) from the public/files
directory of the application to the shared/system/files
directory.
First we define a task in config/deploy.rb
, which will set up the required directories and symlinks. The task creates the directory shared/system/files
with the appropriate permissions (read, write, execute for the captain
user; read and execute for everyone else), removes the existing public/files
directory in the newly deployed release, and creates a symlink from public/files
to shared/system/files:
task :symlink_for_file_uploads, :roles => :app do run <<-CMD mkdir -p -m 775 #{shared_path}/system/files && rm -Rf #{release_path}/public/files && ln -s #{shared_path}/system/files #{release_path}/public/files CMD end
We can now make use of Capistrano's callback mechanism to hook into the execution of tasks, before or after they run. This enables us to layer our tasks over the existing Capistrano default tasks, adding functionality to them without having to fiddle with Capistrano's core code.
To hook into a task, we provide a callback handler. This is a custom task which has a special name, significant to Capistrano. The name should consist of before_
or after_
, followed by the name of the task we want to attach our handler to. In this case, we want to create the directories and symlinks after the new version of the application has been retrieved from Subversion. The task that updates the application code from Subversion is called update_code
; therefore, our callback handler should be called after_update_code
. It is defined like this:
task :after_update_code do symlink_for_file_uploads end
Now when you deploy the application again, you should see the symlink_for_file_uploads
task being executed after the code for the release has been updated from Subversion, e.g.:
$ cap deploy
...
* executing task after_update_code
* executing task symlink_for_file_uploads
* executing "mkdir -p -m 775 /home/captain/apps/Intranet/shared/system/files &&
rm -Rf /home/captain/apps/Intranet/releases/20070428155358/public/files &&
ln -s /home/captain/apps/Intranet/shared/system/files /home/captain/apps/Intranet/releases/20070428155358/public/files"
...
You can verify that the symlink has been created correctly by connecting to the current release directory, then going to the public
directory and listing the directory contents (notice the italicised files
entry in the listing below):
$ ls -go
total 44
-rw-rw-r-- 1 235 2007-04-28 16:50 404.html
-rw-rw-r-- 1 309 2007-04-28 16:50 500.html
-rwxrwxr-x 1 477 2007-04-28 16:50 dispatch.cgi
-rwxrwxr-x 1 859 2007-04-28 16:50 dispatch.fcgi
-rwxrwxr-x 1 476 2007-04-28 16:50 dispatch.rb
-rw-rw-r-- 1 0 2007-04-28 16:50 favicon.ico
lrwxrwxrwx 1 47 2007-04-28 16:50 files -> /home/captain/apps/Intranet/shared/system/files
drwxrwxr-x 3 4096 2007-04-28 16:43 images
-rw-rw-r-- 1 7552 2007-04-28 16:50 index.html
drwxrwxr-x 3 4096 2007-04-28 16:43 javascripts
-rw-rw-r-- 1 99 2007-04-28 16:50 robots.txt
drwxrwxr-x 3 4096 2007-04-28 16:43 stylesheets
lrwxrwxrwx 1 41 2007-04-28 16:50 system -> /home/captain/apps/Intranet/shared/system
Capistrano really shows its character when you want to easily upgrade an application: with a single command, deploy a new version from the repository and migrate the database, far more easily than the old-fashioned way of doing each step in the deployment manually. To see this in action, we'll add a new migration to the application, which will set up a default administrator account. First create the skeleton for the migration (on the development machine):
$ ruby script/generate migration default_admin_user
Then edit db/migrate/008_default_admin_user.rb
and add this content:
class DefaultAdminUser < ActiveRecord::Migration
def self.up
u = User.new(:username => 'admin', :passwd => 'admin').save
end
def self.down
User.find_by_username('admin').destroy
end
end
When the migration is applied, a default administrative user with username admin and password admin is added to the users
table. (The highlighted section of the code shows where the credentials are set.). Ensure that the migration is added to the Subversion repository, so it is available when you next deploy to the production server.
The command to check out the latest version of the code and apply any new migrations is:
$ cap deploy_with_migrations
This will also restart the Mongrel instance, and you can now test that the new admin account has been correctly added to the database.
After you've deployed several new releases, your production server will get cluttered with obsolete entries in the releases
directory. To clear out all of the releases except for the most recent five, do:
$ cap cleanup
Note that by default this Capistrano task will attempt to use sudo
when deleting the obsolete files and directories. However, if the account Capistrano uses to log in (in our case, captain)
is unable to use sudo
on the production server, you'll get this error message when you run the task:
$ cap cleanup
...
* executing "sudo rm -rf /home/captain/apps/Intranet/releases/20070428153649" servers: ["192.168.13.129"]
[192.168.13.129] executing command
** [out :: 192.168.13.129] captain is not in the sudoers file. This incident will be reported.
...
To turn off this behavior, you have to tell Capistrano that it doesn't need to use sudo
to perform tasks on the production server. Add this line near the top of config/deploy.rb:
set :use_sudo, false
This should fix the problem and allow the cleanup
task to run correctly.
In our case, it's safe to turn off sudo
, as the captain
user owns the directory into which we're deploying the application, and is also the owner of the Mongrel process that is running the application. In situations where this is not the case (for example, if you are running Mongrel under one user account and the application files are owned by a different account), you may not have the option to turn off sudo
. In this case, you may need to add the Capistrano user to the sudoers
file instead. How to do this is beyond the scope of this book; but the sudo
website (http://www.gratisoft.us/sudo/) has plenty of documentation that should help.
The final situation that we need to deal with is what happens when deployment of a new version of the application goes wrong: perhaps once it's on the production server, some unexpected fault brings the whole site down. In this situation, you need to be able to go back to the last known good version of the application, which will usually be the previous release. Capistrano provides a task to do this:
$ cap rollback
This command actually rolls back the application code, shifting the current
symlink to point at the previous version of the application in the releases
directory. However, it doesn't automatically roll back the database: potentially you could roll back to version 1 of your application while retaining a database structure that only works with version 2.
Capistrano can be used to work around this issue, and run migrations up or down to bring the database back in line with the application version. Currently this has to be done manually after you've run cap rollback
.
For example, say we roll back to the version of the application before the migration (of the previous section) was added. We also want to roll back to the version of the database before the migration was applied. To find the latest migration in the application, take a look in the current/db/migrate
directory (this is the "rolled back" application directory). In our case, the highest numbered migration in that directory is 007_create_file_attachments.rb
; so, we need to migrate the database down to version 7. The command for doing this is:
$ cap -s migrate_env="VERSION=7" migrate
* executing task migrate
* executing "cd /home/captain/apps/Intranet/current && rake RAILS_ENV=production VERSION=7 db:migrate"
servers: ["192.168.13.129"]
[192.168.13.129] executing command
** [out :: 192.168.13.129] (in /home/captain/apps/Intranet/releases/20070502221404)
** [out :: 192.168.13.129] == DefaultAdminUser: reverting ===============================================
** [out :: 192.168.13.129] == DefaultAdminUser: reverted (0.2761s) ======================================
command finished
(Replace the number 7 in the command with the appropriate version to migrate to.)
3.145.179.35