Rails deployment is complex, involving Subversion checkouts, web, database and application servers, multiple user accounts, and a variety of permissions. With so many variables in the mix, things can go wrong and often do. On top of that, Capistrano is a complex beast, doing a complex job, which makes things even worse. Consequently, you will run across deployment errors. To help you work out what's happening when these occur, here are some examples of the kinds of error you might have to face, and fixes for them.
If the version of Rails on the production server mismatches the one required by the application, you may get this error message when you run cap deploy_with_migrations:
** [out :: 192.168.13.129] Cannot find gem for Rails ~>1.2.3.0:
** [out :: 192.168.13.129] Install the missing gem with 'gem install -v=1.2.3 rails', or
** [out :: 192.168.13.129] change environment.rb to define RAILS_GEM_VERSION with your desired version.
The error is being thrown because of this line in config/environment.rb:
RAILS_GEM_VERSION = '1.2.3' unless defined? RAILS_GEM_VERSION
This states the version of Rails required by the application. As the version of Rails on the production server and the version on the development server are different, Rails will refuse to start the application. There are three ways to fix this:
- Remove the offending line from environment.rb: This is the simplest fix, but it means that your Rails application won't check the version of Rails that it is running under. If the application relies on specific Rails features that are absent from old versions of Rails (for example), and you deploy to a server lacking those features, the application may not work correctly.
- Freeze gems into the application: Rails provides a facility that enables you to take Rails with the application. This technique is known as "freezing" Rails, and is accomplished by running the following
raketask:rake rails:freeze:gems- What this does is copy the current Rails gems (rails, activerecord, activesupport, actionpack, actionmailer, actionwebservice) into the vendors/rails directory inside the application. Rails will now use the gems in that directory instead of the centrally-installed gems when running the application. Note that this technique is only useful if the version of Rails you need isn't installed on the production server.
If you'd rather freeze a specific Rails version into your application (rather than the one installed on the development machine) you can do:
rake rails:freeze:edge TAG=rel_1-2-3- where rel_1-2-3 is a Subversion tag representing release 1.2.3 of Rails (see Chapter 3 for coverage of what Subversion tags are). You can get a full list of Rails tags by visiting http://svn.rubyonrails.org/rails/tags/.
The down side to using the rake tasks to freeze Rails gems is that they export the specified version of Rails into the vendor/rails directory. This means that you will need to store the whole of Rails in your application's Subversion repository too, which isn't ideal. A better approach is to manually create a link in your application that references the Rails Subversion repository using the svn:externals property (discussed in Chapter 8 in the context of plugins). You can do this by editing this property via Eclipse (editing Subversion properties is covered in the section Ignoring Temporary Files in Chapter 4) and setting its value to one of the Rails version tags, e.g.:
vendor/rails http://svn.rubyonrails.org/rails/tags/rel_1-2-3/
- If you now do an svn up to update your application, Rails will be fetched into vendor/rails; however, you no longer need to store Rails in your repository. Instead it will be automatically fetched from the Rails repository proper each time your application is checked out, as your application now just references the external repository.
The only other issue with this approach is that you are reliant on the Rails Subversion repository being available to perform your deployment. If you don't want this dependency, don't use the svn:externals approach.
- What this does is copy the current Rails gems (rails, activerecord, activesupport, actionpack, actionmailer, actionwebservice) into the vendors/rails directory inside the application. Rails will now use the gems in that directory instead of the centrally-installed gems when running the application. Note that this technique is only useful if the version of Rails you need isn't installed on the production server.
- Upgrade Rails on the server: Ensure that the production server is using the same version of Rails as the application by ensuring the correct version is installed, e.g.
gem install rails -v 1.2.3
One other more obscure situation you may run across is where the version of Ruby on the server is different from the one on the development machine(s). For example, imagine your application uses the libxml-ruby library (an alternative Ruby XML library, which is faster than Ruby's default REXML library). libxml-ruby is installed on the developer machines but not on the production server; the gem is pulled into the application in environment.rb with:
require 'libxml-ruby'
On the production server, the libxml-ruby gem is not available. When you try to start Mongrel (e.g. with cap spinner), the command fails; however, no indication of this is given by Capistrano, and when you try to browse to the application, it is unavailable.
Mongrel logs its activity into a file inside the application's log directory; in the case of an application deployed under Capistrano, this file is shared/log/mongrel.log; it should hopefully give you more insights into any errors that occur while Mongrel is starting, e.g.:
** Starting Mongrel listening at 0.0.0.0:4000
** Starting Rails with production environment...
/usr/local/lib/site_ruby/1.8/rubygems.rb:251:in 'report_activate_error': Could not find RubyGem libxml-ruby (>= 0.0.0) (Gem::LoadError)
from /usr/local/lib/site_ruby/1.8/rubygems.rb:188:in 'activate'
from /usr/local/lib/site_ruby/1.8/rubygems.rb:66:in 'active_gem_with_options'
...
In any situation where Capistrano doesn't report any errors but your application has failed to start, check this log first.
In cases where Capistrano is trying to check out code from the Subversion repository and you type in the wrong password, you may see this error message:
[email protected]'s password:
Permission denied, please try again.
Run the task again and type in the correct password.
If Capistrano tries to log in to a server using the username and password credentials you supplied, and those credentials are incorrect, you may see this error message:
** [update_code] exception while rolling back: Net::SSH::AuthenticationFailed, captain
authentication failed for 'captain'
Fix the user and password settings in config/deploy.rb.
If you get the IP address or domain name of an application or web server wrong in config/deploy.rb, you are likely to see this error message:
** [update_code] exception while rolling back: Errno::EHOSTUNREACH, No route to host - connect(2)
/opt/lampp/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/transport/session.rb:88:in 'initialize': No route to host - connect(2) (Errno::EHOSTUNREACH)
This indicates that Capistrano is unable to log in to the server. Fix the IP addresses and/or host names assigned to any role settings in config/deploy.rb.
If you spell the database host name incorrectly, you will get an Unknown MySQL server error when running cap migrate (see below, where the host name is "localhosti" instead of "localhost").
$ cap migrate
* executing task migrate
* executing "cd /home/captain/apps/Intranet/current && rake RAILS_
ENV=production db:migrate"
servers: ["192.168.13.129"]
[192.168.13.129] executing command
** [out :: 192.168.13.129] (in /home/captain/apps/Intranet/releases/20070413171324)
** [out :: 192.168.13.129] rake aborted!
** [out :: 192.168.13.129] Unknown MySQL server host 'localhosti' (1)
** [out :: 192.168.13.129] (See full trace by running task with --trace)
command finished
command "cd /home/captain/apps/Intranet/current && rake RAILS_ENV=production db:migrate" failed on 192.168.13.129
Fix the host property in config/database.yml to make this go away.
When everything else fails, you may need to do some more significant debugging. In Chapter 6: Errors in Production, we covered some of the common errors and fixes. Other approaches:
- Check the log files:
mongrel.logandproduction.logshould give you some clues; the MySQL log files might also help. - Run the application in the development environment on the production server. This should give you more immediate error reporting in the browser, which makes life easier when you are trying to unravel knotty problems.
- Run the test suite on the production server. This is usually worth doing anyway, but can be particularly useful when trying to track down obscure errors.
If you get really stuck, the only thing left to do may be to completely wipe the application off the production server and start from scratch. With any luck, this shouldn't be necessary too often. Here are some instructions for completely rebuilding your application on the production server:
- Log onto the production server.
- Back up the production database using
mysqldump(see Back Up Rails in Chapter 6). - Copy the
shared/systemdirectory somewhere safe. - Kill all the Mongrel processes on the server. To manually stop Mongrel, do:
$ mongrel_rails stop -P apps/Intranet/shared/pids/mongrel.pid(passing the path to your Mongrel PID file as the-Poption) - Check all the Mongrel processes are dead:
$ killall mongrel_rails
- Rewind the database back to version 0:
$ rake db:migrate VERSION=0
- Remove the directory containing all deployed versions of the application. (In the Acme case, this is the entire
Intranetdirectory).
This puts you back to a clean slate. You can now go back to the development machine and run:
$ cap setup
$ cap cold_deploy
$ cap migrate
Once this finishes, the code and database are back to the latest version. You can now import the MySQL backup into the production database, and move the contents of shared/system back into the appropriate directory inside the deployed application.