As much as all of us hate to do it, keeping a backup of your important applications and data is a crucial part of maintaining a Tiger system. Backups generally fall into two categories: full and incremental. A full backup is an exact duplicate of everything within your filesystem (or a branch thereof, such as /Users or /usr/local). From a full backup, you can quickly restore the state of all the backed-up files as they existed at the time the backup was made. Full backups are time-consuming (all files must be copied each time the backup is run) and cannot efficiently be used to store multiple versions of files over an extended period of time. If, for example, you want to have a copy of each day's updates to your file server for the period of a year, you would need to make 365 full copies of each of the files—this could quickly add up in terms of storage media. Full backups are usually reserved for mostly static information that can be copied and stored.
An incremental backup, on the other hand, is used to archive filesystems that aren't static. You start with a single full backup and then periodically back up the files that have changed since the full backup took place. Incremental backups can take place at multiple levels, with each level backing up only the files that have changed since the preceding level was backed up.
This section of the chapter looks at software available for backing up files and volumes.
For most people, backing up the contents of the /Users/<username> home directories will suffice for protecting personal information. This, of course, doesn't help much with applications and Unix system services that have been installed, nor does it cover additional accounts for other family members or friends.
We'll take a look at what tools are available, and what they're appropriate for. Finally, we'll review several command-line tools that can be employed to back up both Carbon/Classic applications and generic data files.
Retrospect is the most well-known and accepted backup solution for the Mac, featuring the capability to back up and restore Windows, Red Hat, and Macintosh systems, store backups on Internet FTP servers, and more. Figure 29.4 shows Retrospect Backup.
Retrospect works based on a client/server model. After the Retrospect client has been installed on the systems to backup, a Retrospect Server administrator can locate the client computers, select the files and volumes to back up, and store them on a variety of media including CDRW, DVD, DAT, and Internet storage sites. Backups can be scripted for automatic execution, or run at any time with the click of a mouse. Restoring files from a backup is a simple point-and-click operation that automatically copies files from the source media back to the client.
Mixed-platform networks will be happy to find that Retrospect clients are available for older Mac systems as well as Windows. Unfortunately, Retrospect has a history of annoying “gotchas” that can affect users with advanced setups. I strongly urge that you read the Dantz forums to figure out whether there are any known issues with your setup.
Retrospect 6.0 comes in several different versions, ranging from a single-desktop backup solution to workgroup and server editions. Find out more about Retrospect from http://www.dantz.com/.
Another enterprise-ready backup solution is BRU. Like Retrospect, BRU can work on a client/server model and comes in a variety of versions from single machine to server editions. BRU covers a much larger range of client platforms—from Windows NT 4.0, Server 2003, and XP to just about every Unix and Linux distribution available.
BRU has been a popular backup solution for Unix systems for quite some time, and should be definitely be considered for deploying a disaster recovery system. The entire application is scriptable and can be run from the command line. To appeal to traditional Mac users, TOLIS has also introduced a GUI for BRU, as shown in Figure 29.5. At present, the GUI isn't quite as well developed as that of Retrospect, but given the stability and performance of the platform as a whole, this might not be an issue for you. Visit http://www.tolisgroup.com/ for more information.
For the home user who just wants to make sure that his critical data is backed up to CD, DVD, or iDisk, Apple's aptly named Backup could be the right answer. Shown in Figure 29.6, Backup is a simple piece of software that is capable of selecting common file types (such as Microsoft Word documents), system information (such as Safari preferences), or arbitrary files and folders and backing them up to your Mac's optical drive or .Mac iDisk. Backup does not currently offer a command-line interface, incremental backups, or a way of performing unattended backups.
Unfortunately, besides the entry-level capabilities, there's another catch to the software: Backup is exclusive those with a .Mac membership. This means that to download and use the tool, you'll need to pay the $100 entrance fee (http://www.mac.com/).
NOTE
On a positive note, .Mac membership also buys you a copy of Virex virus scanning software, along with the mac.com email, iDisk, and iSync capabilities. For Mac users who aren't running servers or using Unix tools, this might be a wise investment. For others, however, the money might be better spent on a third-party backup solution.
Data Backup X from ProSoft Engineering (http://www.prosoftengineering.com/products/data_backup.php) is a complete personal backup system that picks up where Apple's tool left off. It offers advanced features such as scheduling, compression, mirroring, synchronization, incremental backups, and an “evolutive” mode that preserves different versions of files as they change across backups. For personal workstations, Data Backup X is hard to beat. If you can afford the $50 expense, Data Backup X is one of the most feature-filled personal backup software currently available.
The Synk application is an open source Cocoa-based utility that provides synchronization and backup services between folders and volumes. Synk allows the creation of AppleScript Run documents that can be used to launch a specific backup configuration with a simple double-click from the desktop. When used to synchronize storage locations, Synk even offers the capability to archive files that would normally be replaced or deleted during a synchronization process. Download Synk from http://www.decimus.net/synk/.
An entirely different backup approach is to mirror your system and data disks exactly. With Tiger, this is a simple process of using the ditto command, which we'll look at in the section “Using Command-Line Backup Tools.” Although this feature is built into the operating system, a number of graphic utilities have sprung up around it to provide a more Mac-like experience.
The most popular of these utilities is Carbon Copy Cloner (http://www.bombich.com/software/ccc.html). Written in AppleScript Studio, Carbon Copy Cloner adds a user-friendly interface to the ditto command (as shown in Figure 29.7), and enables quick, easy, and even scheduled drive mirroring. Copied volumes are bootable and are created with all file permissions and ACLs intact.
NOTE
Carbon Copy Cloner can also prepare ASR-ready (Apple System Restore) disk images. ASR, a command-line utility that you'll learn about shortly, can restore entire volumes over a network.
Although not an efficient solution for those who just want to back up changed data files, Carbon Copy Cloner, along with an external FireWire drive, makes a fantastic complete-system backup for servers and other machines with complex configurations.
TIP
Carbon Copy Cloner (or ditto) can be used to replicate an existing partition onto a different partition or volume. This is a no-cost way to resize a partition. If you resize partitions frequently, however, VolumeWorks from SubRosaSoft (http://www.subrosasoft.com/thestore/product_info.php?products_id=431) is the only way to go.
Traditional Mac users are obviously the target audience of most of the GUI backup software. Unfortunately, this has led to a plethora of tools that can only be controlled or configured if you are sitting in front of your computer. Remote command-line administration of Retrospect, for example, is virtually impossible.
The Unix side of Tiger provides some command-line backup tools that you've seen earlier (tar/cp), but aside from basic HFS+ compatibility in Tiger, they aren't necessarily the best tools for the job. Let's take a look at a few other utilities that are a bit more capable.
The ditto function (for those who don't remember the addictively good-smelling purple copies from grade school that share the name) creates a duplicate of a file or folder structure on your system. ditto is capable of dealing with the HFS+ filesystem and can quickly and easily replicate entire bootable system volumes.
The basic syntax of ditto is
ditto [-<options>] <source file/folder> <destination file/folder>
ditto can be used to copy multiple files or folders to a folder, or a single file to another file. In the former case, the destination directory will be created if it doesn't already exist.
By default, ditto preserves all the resource fork and extended attributes/metadata of the files it is copying—making it safe to copy files with resource forks and other Mac OS–specific information.
For example, to verbosely copy the entire /Applications folder (and subfolders) to another location, /Volume/NewDisk, I'd use
brezup:jray jray $ sudo ditto -V /Applications /Volumes/NewDisk
>>> Copying /Applications
copying file Applications/.DS_Store .. 15364 bytes
copying file Applications/.localized .. 0 bytes
copying file Applications/Acrobat Reader 7.0/Info-macos.plist .. 3043 bytes
copying file Applications/Acrobat Reader 7.0//ACELiteCarbonLib .. 417634 bytes
copying file Applications/Acrobat Reader 7.0/Acrobat Reader
5.0 .. 4285090 bytes
...
and so on.
The ditto command can also be used to create a zipped archive (/tmp/myhomedirectory.zip) of my home directory (/Users/jray) using the -c and -k options (archive and zip, respectively):
brezup:jray jray $ ditto -c -k -V /Users/jray /tmp/myhomedirectory.zip
>>> Copying /Users/jray
copying file ./.bash_history ... 1375 bytes
copying file ./.CFUserTextEncoding ... 3 bytes
copying file ./.DS_Store ... 6148 bytes
copying file ./.lpoptions ... 17 bytes
copying file ./Desktop/.DS_Store ... 6148 bytes
copying file ./Desktop/.localized ... 0 bytes
This archive can then be restored with -x (extract) and -k (zip), reversing the process.
brezup:jray jray $ ditto -x -k -V /tmp/myfile.zip /tmp/MyHome
>>> Copying /tmp/myfile.zip
copying file .bash_history ... 1375 bytes
copying file .CFUserTextEncoding ... 3 bytes
copying file .DS_Store ... 6148 bytes
copying file ._.DS_Store ... 82 bytes
copying file .lpoptions ... 17 bytes
copying file Desktop/.DS_Store ... 6148 bytes
copying file Desktop/._.DS_Store ... 82 bytes
...
Another interesting use for ditto is to copy only files that are contained within a given BOM (bill of materials). Using the -bom option followed by a valid BOM file limits the files copied to those contained within a given BOM file. For more information on BOM files, read about the lsbom command, earlier in this chapter. Table 29.3 provides Apple's options for ditto.
Having a backup of critical files is the best way to avert disaster. But restoring a backup can be time-consuming and creating an exact mirror on a day-to-day basis is often impractical. In many cases, the best solution is to synchronize files between two or more machines as changes are made. Starting in Tiger, Apple has included an HFS+-compatible rsync, an open source utility designed to make folder synchronization fast, scriptable, and painless.
rsync can operate locally to synchronize files between directories on a single machine, or can work over a network. An especially nice feature of the software is its capability to use several different transport mechanisms for network transfers. The rsync utility, for example, can operate as a server and be used to host rsync-accessible directories on remote machines. But if setting up an additional dedicated server process isn't desirable, you don't have to. An alternative (and often overlooked) method of providing rsync access is through a remote shell such as SSH. If your machines are running SSH, they are ready to use rsync immediately, without any additional software or configuration.
The basic rsync syntax is simple: rsync [options] <source> <destination>. For example, to synchronize the contents of the folder /Users/jray/source with /Users/jray/destination, I would use the following:
brezup:jray jray $ rsync -va source/ destination
rsync: building file list...
rsync: 6 files to consider.
./
Icon
apache_pb.gif
index.html
macosxlogo.gif
web_share.gif
wrote 25079 bytes read 100 bytes 50358.00 bytes/sec
total size is 24711 speedup is 0.98
In this example, the -v (verbose) and -a (archive) options are included so that rsync displays files being copied and includes all attributes (permissions, owner, and so on) of the original files. Running the command again produces slightly different output because rsync doesn't need to recopy the files:
brezup:jray jray $ rsync -va source/ destination
rsync: building file list...
rsync: 6 files to consider.
wrote 172 bytes read 20 bytes 128.00 bytes/sec
total size is 24711 speedup is 128.70
Here, no files were copied because the directories are already in sync.
NOTE
Trailing slashes mean something to rsync. If a path is given with a trailing slash, it refers to the contents of the named directory. If the slash is excluded, it means the directory itself.
Although synchronizing local directories might be useful, the true power of rsync is revealed when it is used over a remote network connection. To do this, all that is needed is a running SSH daemon on the remote side and an account with access to the directory you want to sync. If you've ever used scp, you'll recognize the syntax for an SSH-tunneled rsync immediately; for example, assume that I want to synchronize the directory /Users/jray/Tools located on a remote server www.poisontooth.com with a local directory Tools—and, at the same time, compress the data as it is sent:
brezup:jray jray $ rsync -vaze ssh [email protected]:/Users/jray/Tools/ Tools [email protected]'s password: ****** receiving file list .. done ./ htdigsearch/ adduser authuser.pl backupdisdain.pl cgiinput.pl deluser footer.html ... runindex.pl setupconfig.pl ticker.tar wrote 448 bytes read 216031 bytes 18824.26 bytes/sec total size is 1262138 speedup is 5.83
This example introduces the use of -e ssh to specify that ssh should be used as the remote shell that rsync connects through and -z, which compresses data in real-time to increase the transfer rate. A number of additional switches can be used with rsync to change its behavior. Table 29.4 documents many of the useful options.
Make sure to read through the rsync man page for additional information on command use. The include and exclude options can be used to generate complex file selection rulesets beyond simple wildcards. Also, if you want to run a dedicated rsync server rather than use an SSH, you can view a tutorial on the setup of an rsync daemon at http://www.macosxhints.com/article.php?story=20021023063424701.
A fast and easy-to-install alternative to rsync is psync, a Perl script based on the MacOSX::File module, which handles reading and writing data to and from the HFS+ filesystem. psync does not require any special server daemons to be running; it simply works across whatever local or remote volumes you have mounted on your system. The psync home page is located at http://www.dan.co.jp/cases/macosx/psync.html.
To install psync, all that is required is installation of the MacOSX::File module—psync is included. To install the module, use the same CPAN (Comprehensive Perl Archive Network) steps covered in Chapter 18, “Using the Perl and Python Scripting Languages.”
First, invoke the interaction Perl CPAN shell:
brezup:jray jray $ sudo cpan
cpan shell -- CPAN exploration and modules installation (v1.70)
ReadLine support enabled
Next, use install MacOSX::File to download, compile, and install the required Perl module. (Note: You must have the current developer tools installed for this to be successful.)
cpan> install MacOSX::File
CPAN: Net::FTP loaded ok
Fetching with Net::FTP:
Scanning cache /Users/jray/.cpan/build for sizes
MacOSX-File-0.66
MacOSX-File-0.66/bin
MacOSX-File-0.66/bin/pcpmac
MacOSX-File-0.66/bin/pgetfinfo
...
MacOSX-File-0.66/t/spec.t
CPAN.pm: Going to build D/DA/DANKOGAI/MacOSX-File-0.66.tar.gz
Checking if your kit is complete...
Looks good
Writing Makefile for MacOSX::File::Catalog
Writing Makefile for MacOSX::File::Copy
Writing Makefile for MacOSX::File::Info
Writing Makefile for MacOSX::File::Spec
Writing Makefile for MacOSX::File
...
Installing /usr/local/bin/pgetfinfo
Installing /usr/local/bin/pmvmac
Installing /usr/local/bin/psetfinfo
Installing /usr/local/bin/psync
Writing /Library/Perl/darwin/auto/MacOSX/File/.packlist
Appending installation info to /System/Library/Perl/darwin/perllocal.pod
/usr/bin/make install -- OK
The psync syntax is psync [options] <source> <destination>. For example, to copy my Sites directory (/Users/jray/Sites) to /Volume/backup, I'd use psync /Users/jray/Sites /Volumes/backup. All file attributes are retained during the copy, including permissions and HFS+ resource forks.
brezup:jray jray $ sudo psync /Users/jray/Sites /Volumes/backup
Scanning Destination Directory /Volumes/backup ...
0:.
65 items found.
Scanning Source Item /Users/jray/Sites ...
0:..
178 items found.
0 items to delete,
68 items unchanged,
117 items to copy.
copying items ...
+f /Users/jray/Sites/imagefolder/Classic Aqua Blue.jpg
+f /Users/jray/Sites/imagefolder/Classic Aqua Graphite.jpg
...
f /Users/jray/Sites/joestuff/feed.sql
+d /Users/jray/Sites/joestuff/files
+f /Users/jray/Sites/joestuff/files/1/left.htm
+f /Users/jray/Sites/webdav.key
fixing directory attributes ...
00777,jray,staff /Volumes/backup/joestuff/images
00777,jray,staff /Volumes/backup/joestuff/files/6
00777,jray,staff /Volumes/backup/joestuff/files/5
...
Subsequent executions of psync to synchronize the same directory do not result in any files being copied:
brezup:jray jray $ sudo psync /Users/jray/Sites /Volumes/backup
Scanning Destination Directory /Volumes/backup ...
0:..
178 items found.
Scanning Source Item /Users/jray/Sites ...
0:..
178 items found.
0 items to delete,
185 items unchanged,
0 items to copy.
copying items ...
fixing directory attributes ...
psync can create incremental backups across Samba, NFS, and AppleTalk volumes in addition to other HFS volumes. If you're backing up to a remote filesystem, use the -r option to store permission information in the file .psync.db on the remote system. If psync finds a .psync.db file in the source volume, it uses the file to restore the permissions to the destination. Although -r is usually the only psync option you'll need, several more options that you might find somewhat useful are documented in Table 29.5.
psync can be combined with a cron to create an effective (and free) incremental backup solution.
TIP
If you want to add a GUI front end to psync, psyncX can be downloaded from http://sourceforge.net/projects/psyncx/.
Apple has built a useful backup and restore tool into Mac OS X, but, to date, its use is a bit confusing. Specially prepared disk images that are created from a volume or folder can be used to restore a drive to a known state. The images can even be hosted on web servers and restored from a remote server. Chapter 5, “Configuring Tiger Hardware Support and Preferences,” discusses how Disk Utility can be used to create and restore images from the GUI. Because we're mentioning it here, you might guess that these actions can also be performed from the command line. You'd be right.
To create a disk image from the command line, you can use the tool hdiutil. hdiutil provides access to all operations that can be performed on disk images and has been mentioned throughout the book. For our purposes, we're interested in using the following syntax: hdiutil create -format UDRO -srcfolder <source directory> <destination disk image>. This will create a read-only disk image based on the source folder you specify. Be sure to save the image to a different directory or volume from what you are using as the source! For example, to create an image of my Tiger home directory, I could use this:
brezup:jray jray $ hdiutil create -format UDRO -srcfolder /Users/jray/tmp/myhome.dmg
Initializing...
Creating...
Initialized /dev/rdisk1s2 as a 215 MB HFS Plus volume with a 8192k journal
Copying...
........................................................
After the image has been created it must be scanned to generate the appropriate checksum information to use with Apple Software Restore. To do this, we have to change gears and use the asr command-line interface utility with the syntax asr -imagescan <image to prepare>. For example:
brezup:jray jray $ asr -imagescan /tmp/myhome.dmg
Checksumming partition of size 62 blocks...done
Block checksum: ....10....20....30....40....50....60....70....80....90....100
asr: successfully scanned image "/tmp/myhome.dmg"
After asr has completed its run, the image file is ready for use as a restore image directly within the Disk Utility application or with the asr utility itself. To restore a disk image to a volume with asr, use the command asr -source <path or http URL to prepared restore image> -target <Path to target volume> -erase. This simple command will prepare the target volume to match the source image and will automatically bless any appropriate folders found on the source.
TIP
ASR can also be used to do simple volume to volume cloning like this: sudo asr -source <path to source volume> -target <path to destination volume> -erase.
For years, Unix systems have relied on the functionality of the dump and restore commands for performing tape backups. Although Tiger includes these utilities, they are of limited value unless you are using the UFS filesystem. Since the initial release of Mac OS X five years ago, we have covered these tools in Mac OS X Unleashed with the hope that they would eventually work well with HFS+ and tape drives. Unfortunately this has yet to happen, so, for now, we will be making the existing documentation available at http://www.macosxunleashed.com/downloads/dumprestore.pdf.
TIP
Even if you don't plan on using dump or restore, you might want to download this additional text for information about the planning process for incremental backups.
18.191.17.12