One of the most important hardware decisions you’ll need to make is where all your “buckets” (discussed in Chapter 4) are going to live. When determining which storage configuration to use for your archive structure, consider these recommendations:

To me, the clearly superior storage solution is the live and local hard drive with two backup copies. Live indicates that your archive is not frozen by existing only on write-once media (such as DVD or Blu-ray). Local means that it is currently accessible by your computer—that is, stored on an internal drive (or drives), an external drive (or drives), or a local network file server.

KEYWORDS: Sigur Ros, Music, Performance, Concert, Iceland

Storing images on local hard drives has a number of significant benefits:

This live and local strategy provides an orderly framework for your primary archive, with only one each of the master original file and the master derivative file. It is an accessible, confirmable, migratable, and updatable system.

Figure 5-1 shows the inside of a hard disk drive. While the disk platter looks like a mirror, it’s actually composed of up to trillions of tiny magnets standing on end, arrayed in concentric circles. The polarity of the each magnet can be “up” or “down,” which indicates whether the bit is a 1 or a 0. The read/write head is like a record tonearm, which can flip the polarity of the magnet when it’s writing data, or read the polarity when it’s reading data.

The read/write head of the drive must skim very close to the disk platter but never touch it. Because the disk is spinning so fast (between 4800 and 15,000 RPMs), a cushion of air is squeezed between the platter and the head to keep the head just off the surface. If it ever touches the surface, it will ruin the part it touches and kick up enough debris to ruin every other part of the disk in very short order. This is called a head crash, which will generally be accompanied by a screeching sound.

Figure 5-1. The inside of a hard disk drive. The data is stored on the reflective platter. Opening a drive up like this will introduce dust and ruin it forever.

The magnets in a hard disk are organized in concentric circles—as many as 250,000 rings on a 3.5-inch platter. The head skims back and forth at up to 10 meters/second, and must stop on a line 1/10 the width of a human hair, and then correctly read the polarity of each bit. It’s amazing that this is even possible, and even more amazing that it’s affordable.

A hard disk drive also has electronics to control the mechanism, to translate the data to a format that can be written to the disk, and to do error correction and analysis. Hard drives have a power connector that provides juice for the motor that spins the drive and for the controller circuitry. Each drive also has a data interface: IDE/ATA or SATA for desktop drives, and SCSI, Serial Attached SCSI (SAS) or Fibre Channel for enterprise drives.

Hard drives come in two basic physical sizes: 2.5-inch and 3.5-inch, as shown in Figure 5-2. These sizes refer to the size of the data platters, not the size of the hard drive mechanism. Traditionally, 2.5-inch drives are used for laptops, and 3.5-inch drives are used for desktop computers. Some compact desktops also use the smaller drives to enable a smaller form factor for the computer. 2.5-inch drives generally spin slower, which means that they have slower data throughput. They also have a smaller data capacity and are more expensive per gigabyte.

The smaller drives do have several advantages, depending on the use. They are physically smaller, so they can fit in laptops and small portable enclosures. They need less power to spin, so they can generally be bus-powered, meaning they can draw power from a laptop without the use of an external power supply. And since they are designed to be portable, most of them do a better job of “parking the heads” than full-size drives do. This means they are better able to survive being shipped around or used in a moving environment.

Recent developments in 2.5-inch drives are changing how the small drives are used. A new class of 2.5-inch high-speed drives has emerged that can be used in enterprise and server environments. At the moment these drives are very expensive per gigabyte.

Figure 5-2. Drives come in two physcal sizes, 3.5-inch and 2.5-inch, named for the size of the internal platter.

A new kind of storage device for computers is just beginning to show up in the marketplace. Instead of spinning disks, flash memory (like the stuff used in your camera’s memory card) is being used as primary storage. This type of device, called a solid-state drive (SSD), shown in Figure 5-3, offers some important advantages, including extra speed, shock resistance, and greatly reduced power draw. At the time of this writing, SSDs are only available in very limited capacity (less than half that of spinning disk) and they command a relatively high price— more than four times the cost-per-gigabyte of 2.5-inch disk.

This is an area of fast growth, and due to its advantages I expect that SSD will be a pretty commonplace device before too long in any level of portable computing. They will also be commonly used for boot drives and scratch disk in desktop environments, where greater speed makes a big difference.

The capacity of the drive is the amount of data it can hold. These days, capacity is measured in gigabytes or terabytes. Due to marketing reasons, the capacities listed on drive specifications are not calculated in the same way that your operating system calculates data sizes. So a drive sold as “500 GB” actually only contains 465 GB (actually, the 500 number is Gibibytes, and the 465 number is Gigabytes. Aren’t you glad you asked?).

Figure 5-3. Solid state drives will likely become the drives of choice for portable computing due to speed and low power draw.

I generally suggest that it’s better to get the largest capacity you’re likely to need, at least for the next 6–12 months (if you’re on a RAID system, you’ll want a longer time frame—maybe two years—more on that later). Running fewer drives saves on space, power draw, and heat generation. It’s also easier to manage your drives if there are fewer of them.

Each hard drive has, as part of its specification, a speed at which the platter rotates, measured in RPMs. The faster the drive, the faster the throughput, since the head reads and writes the bits at a faster rate. 2.5-inch consumer drives spin at 4200, 5400, and 7200 RPMs. 7200 RPM drives are a good choice at the moment, but sometimes 7200 RPM drives have too large a power draw or generate too much heat for the portable devices they are contained in. The enterprise-class 2.5-inch drives currently spin at 10,000 or 15,000 RPMs.

3.5-inch drives generally come in 7200, 10,000, and 15,000 RPM models. The 7200 RPM models are good all-purpose drives and have the largest capacities. The faster drives are generally used for system or scratch disks, where fast disk-swapping speeds up the performance of programs like Photoshop, which often have to work with large files.

Hard drives come with one of several different connectors built in. When you buy a drive, it will specify which one is built into the drive. The five types are ATA/IDE and SATA for consumer-level drives, and SCSI, Serial Attached SCSI (SAS), and Fibre Channel for enterprise-class drives.

ATA/IDE

For many years, Advanced Technology Attachment (ATA) connections, shown in Figure 5-4, were the favored internal drive connection in PCs. Apple adopted ATA with the Blue and White G3 models. ATA drives must be configured as either a master or a slave when connecting. This is usually accomplished by the use of a hardware jumper or, more recently, through the use of a cable that can tell the drive to act as either a master or slave (Figure 5-5).

Figure 5-4. An ATA connector on a hard drive.

Figure 5-5. Most recent ATA drives have a setting for Cable Select that enables a special ATA cable to set the drive as a master or slave. These cables have a black connector for the master drive and a gray one for the slave. The blue one plugs in to the logic board of the computer.

ATA also goes by the name ATAPI, IDE, EIDE, and now, PATA, which stands for Parallel ATA. ATA is still in use in many computers today, but most drive manufacturers are switching over to SATA (Serial ATA). If you’re buying new enclosures, I’d suggest staying away from PATA in favor of SATA.

SATA

As of 2007, most new computers (Macs and PCs, laptops and desktops) use the newer SATA interface, shown in Figure 5-6. It has a number of advantages, including longer cables, faster throughput, multidrive support through port multiplier technology, and easier configuration. SATA drives can also be used with eSATA hardware (discussed later) to enable fast, inexpensive configuration as an external drive. Most people investing in new hard drive enclosures for photo storage should be using SATA drives.

Figure 5-6. The onboard SATA connectors on a 3.5-inch drive. The smaller connector is for data and the larger one is for power. This is actually the SATA II connector, which is the only SATA you should be considering.

SCSI/SAS and Fibre Channel

SCSI, SAS, and Fibre Channel drives are rare in desktop computers, and are typically found in expensive enterprise-level storage systems. You can also find SAS drives (along with the necessary SAS controller cards) in video editing systems where maximum throughput is needed. Some of the faster drives, such as Western Digital Raptors, come with SAS connectors, so be aware when you mail-order one.

Digging Deeper

SMART Status

Computer and drive manufacturers have collaborated on a number of diagnostic tools that might be able to tell you that a drive is having trouble and is about to fail. The Self Monitoring, Analysis, and Reporting Technology (SMART) keeps track of some important statistics and can alert you to problems with the drive. It’s nowhere near foolproof, but certain factors are highly likely to indicate a dying drive. Reallocated sector count (or almost any error with “sector” in the name) is one of the most important indicators of impending failure, as are nonzero read error rates and spin retry rates.

The initial SMART implementations were very cautious and used a low threshold to determine problems with drives. That caused a lot of consumers to return drives, and manufacturers have since raised the threshold for what constitutes failing, which means that now problems can be missed. Figure 5-7 shows a report from Apple’s disk utility, which reports a drive as having passed the SMART check. A second utility, SMART Reporter, looked at the same drive and saw a failing unit, as shown in Figure 5-8. The manufacturer (Seagate, in this case) recommended sending the drive in for replacement.

Figure 5-7. In the bottom of the Apple Disk Utility panel, you can see a SMART status readout. Sometimes this will read as verified when other tools will show the drive as going bad. If this readout tells you the drive is failing, replace the drive immediately.

Digging Deeper

Figure 5-8. This screenshot from SMART Reporter shows a drive with a high reallocated sector count. That means there were a lot bad sectors on the drive. Since it was a new drive, it was returned before ever being put into service.

If you’re on PC, you can get a similar experience with SMART. The built-in utilities will do some monitoring, but the threshold for passing is pretty low, so passing does not prove anything. If, however, Vista alerts you to a SMART failure, you should take care of that drive immediately. You can add more thorough SMART reporting by using third-party utilities like Active Hard Disk Monitor from LSoft.

SMART reporting is generally available only with PATA and SATA drives directly connected to the computer’s motherboard (or connected through some eSATA ports). Connections made through FireWire or USB (Universal Serial Bus) don’t report the SMART status properly. If you like the idea of this additional information, your best option is to configure your external drives with eSATA connections. Check with the maker of your eSATA expansion card to see if it passes SMART data through.

Now that we’ve gone over some characteristics of hard drive mechanisms, let’s consider where the drive can live. The enclosure for your hard drive can be the computer itself (for an internal drive), a single-drive external case, or a multiple-drive external case. Figure 5-9 illustrates each of these case types.

Multiple-drive Externals

Multiple-drive cases are an excellent solution for a large archive. Although they are larger, there’s less wiring clutter than you would get with several single-drive cases. And once you have bought a big drive box, you can fill it with less-expensive internal drives, which you can swap out for higher capacity drives as additional space is required. This is the arrangement that I currently favor.

Figure 5-9. You can increase your hard drive capacity by adding internal drives, single-drive externals, or multiple-drive externals.

Note

While the connections have ratings for throughput, you probably won’t attain the full speed.

The hard drive mechanism has its internal interface (PATA, SATA, SAS, or Fibre Channel), and the enclosure has one or more external interfaces as well. The external interface determines how the drive enclosure connects to the computer. There are three principal ones in use, and a few additional ones that are used in high-end systems. Figure 5-10 shows a drive that has the three most common connection types.

Figure 5-10. This drive enclosure has all of the principle connections in use for external hard drives—USB, FireWire, and eSATA.

Digging Deeper

Bridge Board Chipsets

SATA, SAS, and Fibre Channel are all internal as well as external device connectors. When you plug one of these drives into a connector on your computer, the drive essentially establishes a direct connection to the computer’s logic board. When you use FireWire or USB, a bridgeboard translates the drive’s SATA or PATA internal connection to FireWire or USB, as shown in Figure 5-11. The quality of this bridgeboard affects the speed and reliability of the external drive.

Figure 5-11. SATA, Fibre Channel, and SAS drives can connect directly to computers, even as external drives. SATA can connect directly through eSATA, or through USB or FireWire. PATA must connect through USB or FireWire.

eSATA

eSATA is a configuration that creates a SATA connection in an external enclosure. It’s a fast and stable connection, offering up to 150 or 300 megabytes per second. eSATA is only beginning to show up on computers as a built-in external interface. You can add eSATA to older computers by means of an expansion card. eSATA does not have the capability to bus-power hard drives, so you must use an external power source.

eSATA is often described as hot-swappable, meaning that you can disconnect and reconnect different drives without restarting the computer, but this is often not the case. The design of the host (the way the eSATA is connected to the logic board) will determine if the connection is really hot-swappable.

Digging Deeper

Whenever I buy an external enclosure for my hard drives, I look for an Oxford 924 (or later) chipset in the bridgeboard (Figure 5-12). This bridge handles FireWire 400 and 800, as well as USB. Retailers who know their products well will be able to say which bridgeboard is in the enclosure. If the retailer can’t tell me what it is, I generally assume that the manufacturer has not bothered to spend the money on a good bridge and has put a generic one in the box.

Figure 5-12. I suggest using a FireWire bridge board with an Oxford chipset. You can read the chipset type on the chip itself, but only after you buy it. Look for a specification when you’re buying a drive enclosure.

The power supply the drive will use depends on the case design. An internal drive added to a tower computer will use the computer’s power supply. This is tidier because you don’t have power cables running all over the place. It does tax the computer’s power supply, however, and that can lead to failure (watch out for an SPF).

The power supply for single-drive external cases is typically a power brick that sits outside of the case. If you are going to use these, try to always buy the same brand so that you have swappable components to test with if there is a problem.

The power supply for a multiple-drive case is usually inside the case, and is a lot like the power supply inside your computer. If it fails, you can often replace it with a generic one from a local computer store.

Bus-powered drives

Portable drives with 2.5-inch disks inside often use the power in USB or FireWire cables to provide electricity to the drive. This is a real convenience for portable devices, but there are a few caveats. Some drives (particularly faster ones) require more current than is supplied by the port. In these cases, the drive will either not fully mount or might disappear when the power draw gets too large. Unfortunately, the only way to see if a drive works with your computer is to hook it up and give it a try.

There’s another note of caution that you should be aware of when using bus-powered drives. Too high a current draw can burn out the port that the drive is connected to. This seems to be typically associated with running multiple drives daisy chained off a laptop’s FireWire port. If you need to run more than one drive off a single port, you should buy one that will accept an external power adapter, as shown in Figure 5-13.

Figure 5-13. Many bus-powered drives can also draw current from an external power supply (the connection at the far right). This is the preferred method for powering daisy chained drives running off a single FireWire port.

When buying hard drives, many mail-order sites offer discounted drives labeled as OEM, or Original Equipment Manufacturer. Many manufacturers originally purchase OEM drives to be included in computers as they leave the factory. Often, manufacturers will end up with a surplus of drives at the end of the process, and these find their way into various retail channels. OEM drives can offer significant cost savings, but there are some things to watch out for.

OEM drives can have custom firmware loaded onto the drive’s controller circuitry. This can include optimizations for a particular brand of computer. Occasionally, this custom firmware can cause real problems when the drive is installed in a different computer.

While OEM drives are shipped to the manufacturer in an appropriately padded box, once they enter the retail channel they are often protected only by an antistatic bag. While this provides protection from electrostatic discharge, it does nothing to protect the drive from impact damage.

Retail drives are those that are intended for sale as individual drives in the retail marketplace. They are prepared for the most common computer configurations and are shipped in static and impact-resistant packaging (Figure 5-14). Thus, they are at least somewhat less likely to give you problems. While I have had trouble-free service from several OEM drives, I have decided against the further purchase of OEM drives for my own use if the drive is available in a retail outlet for a comparable price.

Figure 5-14. When you buy an OEM drive, it comes in minimal packaging (left), which may subject the drive to damage from physical impact. Retail packaging (right) is designed to protect the drive in shipping.

Whether you buy retail or OEM, I strongly suggest buying brand name drives, rather than “private label” or drives from obscure manufacturers. Hard drives are complex and finely tuned mechanisms, and I would not suggest being too cheap.

Hard drives have steadily fallen in price over the last 20 years, keeping to a trend where storage drops to about half its current cost in 18 months or so. Given this trend, it does not make a lot of sense to spend money on drives that you won’t need in the next year. Buying it later will be cheaper than buying it now.

When I first started working with digital images in the early 1990s, hard drive storage cost $2,000 per gigabyte. Today you can purchase it for less than $.10 per gigabyte—a nearly two million percent increase in purchasing power. Storage solutions will continue to get cheaper, faster, physically smaller, more reliable, and higher in capacity in the future. Keep this in mind as you decide how to store your images now and how to configure future storage systems.

This calculation gets a little trickier if you are implementing a RAID system (discussed in the next section). Since you need to dismantle and rebuild the entire RAID system in order to add capacity, you won’t want to upgrade very often. You’ll probably want to set your RAID system up to provide at least two years of storage.

As my former network administrator friend Skip always says, buy the computer you need now, not what you will need in 18 months. I have found this to be very solid advice, ever since he gave it to me more than 18 years ago.

Storing hard drives that are used for backup files also requires some reasonable consideration. You’ll want to protect the drives from ESD and impact, as well as from moisture or heat damage. What you use to protect and store offline drives will depend on a number of factors: budget, how many drives you have, how to get them offsite, whether they are internal or external, and whether they are in caddies or bare.

Drives should be stored in some kind of ESD-protected environment. At the very least you can store them in an antistatic metallic bag (available at major computer stores). You should use this in combination with some kind of padding to protect from impact. Figure 5-18 shows the way I’ve been storing my off-site drives for the last several years. The drives are stored in an antistatic bag inside the clamshell that the drive was purchased in. I then place the drives in a camera bag for added padding.

Figure 5-17. I put a label with some important information on my hard drives. Make sure you don’t cover up any breathing holes on the drive case, or you will suffocate the gnomes that live in the drive mechanism.

Figure 5-18. Different ways that hard drives can be stored. My offsite drives live in this camera bag.

I’ve recently found a product from Wiebetech that I really like for hard drive storage (Figure 5-19). These plastic boxes are fabricated specifically for hard drives, and offer ESD protection, as well as some level of shock protection. I now use these in my off-site storage bag.

Figure 5-19. A dedicated drive storage container from Wiebetech. This case protects against both ESD and, to some extent, impact damage.

When you buy new drives, you should do a little preparation to increase the likelihood of trouble-free service. If you zero all data on the drives, you make sure that every bit on the drive is touched. This will force the drive to do a thorough scan of the drive, finding bad sectors and marking them as “off limits.” If the drive is attached in such a way that you can access the SMART data for the drive, you can double-check the health of the drive mechanism before you transfer your data to it, or before you configure it for RAID.

As a digital photographer, you will be using the external ports for a number of temporary connections, such as your camera, card reader, and backup drives. This will involve a lot of plugging and unplugging during the life of the archive, and mistakes can happen—I find that when I use external drives connected to my workstation, I occasionally unplug or turn off the wrong drive inadvertently. This can lead to total data loss on the drive (particularly on Macintosh).

If you are going to use external drives for your primary archiving, make sure that you configure your computer’s physical space so as to minimize the chances of unplugging the wrong drive. I suggest using color-coded cables.

Also, while externals are ostensibly plug and play, manufacturers have different protocols for plugging and playing. Some require drives to be connected before powering up, while others require just the reverse. Make sure to pay attention to the owner’s manual on this point. You may want to affix a label to the drive indicating the preferred connection sequence.

JBOD is a configuration of multiple disks that retain their identity as individual disks. Each physical disk is a logical disk, as shown in Figure 5-20. It is the simplest arrangement and the easiest to configure, upgrade, and repair.

Figure 5-20. In a JBOD configuration, each disk shows up as an individual drive. This arrangement provides maximum flexibility in configuration and upgrade.

I prefer JBOD as the drive structure for archive files in the bucket system for several reasons:

In a RAID setup, several drives are configured together to act as a single volume. The computer sees multiple drives as a single device, as shown in Figure 5-21. This can be desirable for reasons of speed and/or redundancy. You can control your RAID setup either by using a hardware controller (such as a PCI expansion card) or by using software.

Figure 5-21. In a RAID setup, multiple disks are spanned to create one logical disk to add either speed, redundancy, or both.

You can configure RAID systems so that data is written to multiple drives with redundancy. This means that if a drive fails, all the data can still exist on the device. These mirrored or parity RAID setups can be a good choice to store working files in a mission-critical situation, such as a studio where there is lots of deadline pressure. And this kind of RAID is essential in a corporate environment where 24/7 uptime is necessary.

You can also configure RAID to produce significant gains in throughput, by reading from or writing to multiple drives simultaneously. This is useful in situations where there is a need for lots of throughput, such as in video editing. This can also help a photographer who routinely works with very large image files.

Let’s look at the various flavors of RAID and see what they do.

The previous section might have made you nervous about getting a RAID setup. There is a consumer-level option that you may want to consider if you’re looking for a fault-tolerant storage device—Drobo (Figure 5-23). Drobo’s goal is to offer many of the advantages of RAID with some significant improvements, all in a device that is a simple storage appliance.

Figure 5-23. Drobo is a new kind of RAID that brings drive spanning to your desktop in appliance form.

Drobo acts like a RAID at first glance—it spans up to four drives to make one logical drive. It offers the same kind of parity protection you get with RAID 5, where the data can survive the loss of any single drive (or, if the drives are large enough, the loss of any two drives, like in RAID 6).

However, Drobo offers some things RAID does not. It can intelligently make use of drives of different sizes (unlike RAID, where all drives are chopped down to the size of the smallest drive). When you need to upgrade the capacity of your Drobo system, just take out the smallest drive and replace it with a larger one. The data will be swapped around, making the best use of the new drive space.

Drobo also has some very simple “gas gauge” lights that show you how full it is. As the disks get full, it will show you on the front of the box itself, as well as in the Drobo Dashboard software that comes with the unit (Figure 5-24). It connects to the computer by USB2 or FireWire 800 connection, and offers an add-on Ethernet connection for use as a NAS drive.

Figure 5-24. The Drobo software indicates how much space is left on the drives and how it’s allocated. It also has tools to see how adding larger drives will increase the overall storage capability.

Drobo also has some intelligent power consumption features. It goes into a very low-power-draw sleep when not connected or not used, and wakes up when it’s needed. And because it is a standardized item, the controller card is likely to be available in the future (in the event that the one in your unit dies and you need to get your data off).

There is one important disadvantage, however, compared to something like RAID 5. Drobo is not very fast for reads and writes. The newest FireWire unit adds additional speed over the original Drobo, but you should probably not consider it as primary storage for use in an environment where high throughput is important. I’ve been using one as a working files backup that is accessed in the middle of the night.

Like so much in the computing world, I expect this to be an area of rapid change, and I look forward to seeing fast units produced by the company in the near future.

NAS comes in two flavors: self-contained boxes about the size of a FireWire hard drive, and NAS servers that are effectively small, fully functional servers with connection ports for monitors, keyboards, and external drives. The self-contained drives make me nervous because there is not a transparent operating system—if the disk has a problem, you are dependent on the manufacturer to sort it out for you. The NAS servers provide an easy way to add a server for a low cost if you don’t already have a computer to turn into a server.

Although I generally don’t recommend using NAS as primary storage at the present time, it can be a good solution for someone who wants to keep backup storage online but far away from the primary storage. Since you can connect it by Ethernet cable, you have the flexibility to move it farther away than any other kind of wired storage.

CD, DVD, and Blu-ray disks provide a cost-effective and forward-compatible way to back up your images (Figure 5-25). Because so much content (music and movies) is distributed in this format, it’s likely that players for these disks will be available long into the future—hopefully long past the time when you have migrated your backup storage to some other medium.

While the capability to play optical disks will be available long into the future, the particular disks you burn today may not be so lucky. Data stored on an optical disk can degrade at an unknown rate, leaving you with disks that are partially or totally unreadable. There is a lot of conjecture and bad information floating around about the longevity of optical disks, but the truth is that we don’t know how long any particular one will last.

With proper handling, a quality optical disk that is burned well should be a valuable backup to a hard drive–based archive for 5 to 10 years. I’ve done data validation on the DVDs I’ve used to backup my collection, and have not found evidence of data degradation, even on 6-year-old disks. This is a collection of more than 700 DVDs from many manufacturers, burned with several different optical drives. I have run data validation checks on about 1/4 of the disks.

Figure 5-25. There are three kinds of optical disks: CD, DVD, and Blu-ray. They are all the same physical size, so each successive player also plays the older format(s). This will help keep them from becoming obsolete.

The fail-safe backup of choice for institutions is digital tape, in one of its many forms (Figure 5-28). One reason digital tape is preferred as a secondary medium is because it has a high capacity, and you can add more capacity cheaply once the system is in place. You can also reuse digital tape, which makes it well-suited for use as a rotating offsite backup. Digital tape is much less susceptible to viruses than hard drives are, if older versions of files are kept archived.

Figure 5-28. Digital tape is expensive to implement, but makes the most sense for a very high-volume photographic archive.

Digital tape has a few important drawbacks. The first is that a modern tape system is costly to implement for the single user or small workgroup. The tape deck itself is expensive, as is a tape autoloader, if one is needed. Often, these machines can only connect to computers using relatively expensive add-on cards, like Fibre Channel or SAS. Additionally, there is a confusing array of tape formats to choose from, some of which are proprietary and some of which are open.

Digital tape is most attractive to a studio or workgroup that shoots too much to consider Blu-ray as a secondary medium. If you need to back up hundreds of gigabytes a week, digital tape is probably the right technology. Tape is also the right choice for many workgroups that need to back up the work of multiple users. We’ll discuss more about backup scenarios in the next chapter.

One of the fist choices you need to make is the size of card to buy. As with hard drives, we have a choice between a larger number of smaller cards or a smaller number of larger cards. I come down firmly on the side of using larger cards. Some of the advantages are obvious. With large cards, you can carry lots of storage in a small package. You’re less likely to run out in the middle of a shoot. You’re also less likely to run out in the middle of a shooting sequence and miss pictures because you have to change cards.

In fact, changing cards rapidly is the most common cause of problems I’ve seen. If cards are pulled from the camera before writing is finished, images—and maybe the entire card—can be lost. The same thing applies to cards being disconnected from the computer before being dismounted: the entire file system can be corrupted.

Sometimes there are workflow considerations that make smaller cards more attractive. Those who want to burn each 4 GB card to backup DVD have sometimes become accustomed to the nice size correlation between these two.

In general, I have not found that the premium-priced brands are superior to other “no name” brands. In some cases, premium brands have had significant problems when put into widespread use.

Speed ratings, on the other hand, are important. The speed with which the card can write files and read them out to the computer can determine how fast you can shoot and how long it takes to download. The difference can be significant. Look for cards in a range that is higher than 200x.

Direct Memory Access (DMA) is the traditional transfer protocol used by media cards for the last decade. Ultra DMA (UDMA) started to come into general use in media cards in 2007. Once cameras supported it, the demand grew rapidly for UDMA cards and card readers. UDMA offers a much faster transfer protocol as long as each part of the system can take advantage of it. This means that both card and camera need to be UDMA-compliant in order to get faster write speeds, and both the card and the card reader need to be UDMA-compliant to get the faster read speeds.

There are many different kinds of readers you can use to transfer images from the card to the computer. Some computers have them built in, or you can use readers that connect to your computer via USB, FireWire, or PCMCIA/ExpressCard. They can be DMA or UDMA. Which one should you get?

Unfortunately, there’s no simple answer. After reliability, which is very hard to measure, transfer speed is probably the most important characteristic for users, but this is not easy to specify. Different computer models, system software, port drivers, and bridge chips in the card readers can all affect how fast the files transfer, as can the drive configurations.

The standard technology for wired networks, Ethernet, comes in three speeds—10BASE-T, 100BASE-T, and Gigabit. A fourth speed, 10 Gigabit, is on the way. Each successive version of Ethernet has increased speed by a factor of 10. Gigabit, or 1000BASE-T, became available on personal computers in 2000, and it is capable of transferring about 125 megabytes a second. If you’ve got a hard-wired network, it should be Gigabit Ethernet (at least until 10 Gigabit Ethernet becomes common).

There are also different specifications for wireless networks: 802.11a, 802.11b, 802.11g, and the coming 802.11n. The fastest current specification, 802.11g, cannot even transfer a megabyte a second, which puts it in the neighborhood of 10BASE-T in speed. While this will probably not affect you surfing the Web, it will make large transfers of image data painfully slow.

Both wired and wireless networks can fail to achieve their maximum throughput for any number of reasons. Poor signal and noise in the transfer can reduce throughput, as can collisions in the device negotiating the connection.

Ethernet networks use a hub or switch to control the flow of data between devices on the network. Like the NIC, the hub or switch will have a speed designation. Hubs are simple devices for directing the traffic, and will slow down if there are more than two computers on the network talking to each other. A switch, however, will create a more direct link between any two devices on the network, and can handle larger networks with less slowdown. If you’ve got more than a couple computers swapping a lot of data and you need maximum throughput, you’ll want to use a managed switch for optimal performance.

If you are sharing an internet connection, there needs to be a router on the network. It doles out IP addresses to each device on the network. While you can easily have multiple switches or hubs, you shlould only have one router (unless you know what you’re doing).

Ethernet cable also has its own specification, and the higher the number, the higher the theoretical throughput. The design of the cable—how thick the wire is, how the internal wires are braided, and what kind of shielding is on the wire—all affect how much noise is introduced in data transmission. This becomes particularly important in long cable runs, or when the wires run close to sources of electrical interference, like household electrical lines. Category 6 or 6a cable is current the highest officially specified standard.

If you are going to run some cable for your network, make sure to use the highest category cable you can. If you can, try to keep network cable from running alongside power lines for long stretches. And if you are installing a network in a building you intend to occupy for a long time, you might want to consider running the cable inside a conduit like the smurf tube, shown in Figure 5-31. This will give you the opportunity to upgrade the cable when new technology comes out that can carry even more data. Since you can easily run new cable through the tube, it’s much less expensive to upgrade.

Figure 5-31. A conduit tube (known as a smurf tube for its blue color) can make running future cables, such as fiber optic, a breeze. You can start with Cat 5e or Cat 6 cable and upgrade easily when better cable comes available.

Basically, a server is a computer that makes files available over a network. A server can be dedicated (i.e., it can exist only to make files available to other computers), or it can do other jobs as well. For example, your main Photoshop workstation could make files available to other machines, qualifying it as a file server, even as it performs other tasks. A client computer makes use of files or applications that reside on another computer.

Photographers may decide to switch to client/server networks (rather than directly connecting their drives) for several reasons, including the following:

One economical way to set up a modest server is to take a stable used computer and add some internal or external drives. This is a good use for a proven reliable computer that you’ve replaced with a newer, faster one. Because processor speed is less important for the server than for your editing workstation, an older computer can have a long “second life” as a server. The most desirable qualities for the server are, in order: proven stability, quality manufacturing, Gigabit Ethernet, and expansion options (multiple drive bays, FireWire/USB2/SATA ports, or open PCI slots).

You might also want to consider buying an inexpensive new computer for this task. On the Mac side, an iMac or Mac mini can make a fine server with a pretty small footprint and enough connectivity and processing power to work for an individual or small group. On the PC side, there are many more options, like the Dell Studio Hybrid, that squeeze a lot of computing power into a small and inexpensive package.

While a standard desktop computer running a basic OS can work as a server for a handful of clients, if you’ve got to serve files for more than a half-dozen computers, or if you have a high throughput demand, you should consider getting a machine that was built specifically as a server, running a server OS.

On the high end of the server spectrum are rack-mounted machines such as the Apple Xserve or Dell PowerEdge, with hardware RAID setup. These computers are the kinds of machines that large corporate networks run on. They’re well made and reliable, with modular, redundant parts for quick swap-out. At the moment, rack-mounted RAID servers are very expensive.

As your archive grows, an increasingly large percentage of it will be archive files rather than working files; most of the time, you won’t need time-critical access to most of it. A mission-critical, expensive server like this would be overkill for most photographers’ complete archives, especially as the archives grow in size over the years.

I expect that the falling price of hard drive storage, coupled with the demand for lots of media storage space, will spur the development of many new SOHO servers with lots of capacity. Look for these to become available soon, driven by the needs of people who want to store lots of video from their digital video cameras and television digital video recorder (DVR) devices.

Note that while processing power is not terribly important for file-serving functions, it’s useful for data validation functions, which tend to be more processor-intensive. As you consider what kind of server configuration to create, you might want to keep this in mind. In fact, data validation needs were the reason I upgraded my server from the older G4 computer shown in my last book to the iMac shown in the next section.

The system shown in Figure 5-34 is the cheapest way to get started—simply add a drive to your existing system for the image archive and back up your image files onto an additional drive and to optical media. If you’re using a laptop or all-in-one computer, the dedicated image drive will probably have to be external. With a tower system, it could be an internal drive.

Figure 5-34. This is the basic starter system.

Further considerations:

The configuration shown in Figure 5-35 adds multiuser access, as well as the ability to lock your server in a remote location in your house or studio (make sure the server area is vented to avoid overheating).

Figure 5-35. A basic client/server setup will provide an extra level of security and allow multiple machines to access your files.

Further considerations:

The configuration shown in Figure 5-36 adds a laptop to the system, which has its own backup drive(s).

Figure 5-36. A professional photographer who works on location will generally need a laptop computer for downloading and field work. It can be added to the client/server system easily.

Further considerations:

The configuration shown in Figure 5-37 provides the fastest access times and redundant storage for a no-downtime storage solution. It’s also really expensive.

Further considerations:

Figure 5-37. If you have a very big budget, you might select this configuration, which can support multiple high-performance workstations with seamless “no bucket” backups.

In this section, we’ll look at a couple laptop configurations I have been using—a PC setup and a Mac rig. For me, these laptops are mostly for location work or for business, word processing, and web surfing back in the studio. The PC version is more powerful than most desktop tower computers you’ll find, while the Mac has got less horsepower.

Either of these laptop computers would be a fine workstation for many photographers. The processors that now come in laptops will outperform the best desktop systems of a couple years ago with no problem. If you wanted to use one of these as a main imaging workstation, I’d suggest a couple of improvements. Each of these computers can accept an ExpressCard eSATA attachment. This would provide fast drive access, which would translate into significant improvement in performance for heavy-duty image-editing work.

I’d also strongly consider adding a larger and better monitor for critical color work. Both of these machines have video cards that can drive large monitors, which I have found to provide a significant boost in productivity.

PC laptop

I have a Lenovo W700 (Figure 5-38) for use as both a location processor and a studio workstation. This machine was built for the professional photographer, and includes such features as an integrated color profiling system, Wacom pen tablet, dual internal hard drives, Blu-ray writer, integrated media card readers, two core duo processors, and a video card with 1 GB of video RAM. Currently, it’s configured with:

• 4 GB RAM in the machine. Having lots of RAM will provide the best performance. If I were running Vista 64-bit, I’d have it filled with the full 8 GB.

• Dual 320 GB 7200 RPM drives installed as JBOD (primary storage and a backup).

• External backup drive connected with USB. The portable drives I use are ones I put together myself. They hold SATA drives and have connections for FireWire 400, 800, and USB. They also have a power input. The bridge-board inside the drive is an Oxford 924.

Figure 5-38. The Lenovo W700 is built specifically for digital photographers. It has multiple hard drives, Wacom Tablet, multiple internal drives, and more.

Revised Recommendation

Portable Download Devices

In the first edition of this book, I outlined the use of a self-contained portable download and storage device as part of a location workflow. I’ve discontinued use of these for a number of reasons. First, they slow down the process because they require a second download. And if you just use the device as an external drive, connected by USB or FireWire, you’re generally not getting any added benefit compared to a simple external drive.

Many people like to use these devices, such as the Epson P-5000 photo viewer, when traveling instead of taking a laptop. Because it can read the embedded JPEGs, you can see what you have shot along the road and even view the images on a television. Keep in mind that the vast majority of these devices will only show you JPEG or embedded JPEG previews—almost none of them can decode a raw file. So just because the file displays properly on the portable, that does not mean that there is nothing wrong with the raw image data.

If you use an external device, keep in mind that it is storing the files on a single laptop-sized hard drive—the same kind of devices that routinely fail without warning. If the pictures are important to you, you’ll need to have at least one, and preferably two backups of the files. You can back up the images a couple of ways. You could use the portable storage device to copy all the files to an external drive or you could delay erasing the media cards, or both.

Mac laptop

I also use a Mac, shown in Figure 5-39. This one is a Macbook Pro set up in the following way:

• The RAM is maxed out at 4 gigabytes.

• The drive on this computer is not easily user-replaceable, so I have not upgraded as often. It has a recently upgraded 320 GB 7200 RPM drive that is big enough to hold even a long location shoot.

• I have found the fastest download configuration to be a USB card reader and two FireWire drives, each directly connected to a port on the computer. I can use the same drives that I show in the PC section above.

Figure 5-39. The Mac laptop configuration I use. Since it has only one internal drive, I need at least two external drives to properly back up my works in progress.

For many professional photographers, a dedicated imaging workstation will also be necessary (Figure 5-40). Full-size tower computers offer many advantages, including more processors, better video cards, more RAM slots, multiple drive bays, and PCI card expansion. In this section, I’ll outline one that I have in my studio.

I can boot my Mac Pro Quad-Core tower in either Mac OS or Vista. Because Photoshop cannot run in 64-bit on Mac OS, the fastest way to run Photoshop for very large files is to boot the computer in Vista 64. I’ve been working with image files that are up to 20 GB in size, and running in 32-bit mode on the Mac can be painfully slow.

Figure 5-40. A workstation configuration from my studio.

My Mac workstation is set up as follows:

Figure 5-41 shows the server that I am currently using in my studio. It’s an Intel iMac that is connected by gigabit Ethernet to my workstation (and about seven other computers). While it’s not the perfect setup, I’ve been very happy with its price and performance.

Figure 5-41. Here are the components to my storage system. The iMac works as a file server. I have a four-bay JBOD, a two-bay JBOD, Drobo, Blu-ray burner with empty drive bays, and my backup drives and optical disks.

KEYWORDS: NOC, Network Operating Center, SAN, Network, Servers, IT, Technology

The biggest drawback to the iMac is that it does not have the capability to add eSATA as a direct connection (the drives are connected by FireWire). This means that I don’t have access to SMART data reporting, but I do get a smaller footprint, better processing power, less electrical power draw, lower cost, and less heat generation than I’d get with most tower computers.

Okay, so let’s look at how this machine is set up. It’s the heart of my digital archive (Figure 5-42).

Figure 5-42. This photo shows my server “cubby” with the computer pulled out. The iMac is the server, and the JBOD devices and Drobo sit behind it. (The swapper is outside the cubby.) Make sure you have enough ventilation. I have one fan that pulls air from a cool crawlspace and another that exhausts. There are also cutouts for passive ventilation.