18
Managing and Storing Video Files
AVCHD needs a fast dual- or multi-core computer with at least 4GB of RAM (preferably more), a high-quality graphics card, and a high-speed hard drive or solid-state drive (SSD). In addition, newer editing programs need a 64-bit operating system. If SDHC or SDXC memory cards or MTS files themselves are not compatible with your computer, you may need to update your computer’s operating system, RAM, graphics board, hard drive, drivers, or the computer itself.
For editing, you need to place video files on your computer’s internal or external hard drive. It is usually good practice to house these source files on a 7,200 rpm high-speed drive separate from the editing application and system drive. Reading and writing between the two drives will be a faster process than if both operations happen on a single drive. It is also a good practice to duplicate your source files one more time to an additional backup hard drive stored in some other location and/or archive the files to Blu-ray data discs.
Whether you edit with a desktop or laptop, an external high-speed drive should connect to your computer’s fastest input/output port, which will be FireWire 800, eSATA, USB 3.0, or Thunderbolt. If you have a desktop computer with internal locations for a second, third, or fourth drive, the internal connection (usually Serial ATA 3.0 at 3–6Gbps) may provide your fastest read/write rate.
If you will be offloading your video files directly from the G10/XA10, the connecting cable from the camcorder to the computer is a USB 2.0, which is good enough for transferring or archiving files but not for actually editing them. This is why the video source files need to be relocated from the camcorder onto a high-speed drive that can meet the demands of the editing process.
To transfer directly from the G10/XA10, connect the compact power adapter to the camcorder. Do not do this operation on battery power alone. Turn the camcorder on, toggle the Camera/Playback button to Playback, tap the Index icon in the lower left, choose the memory tab for the location of your video files (Internal, A, or B), and tap the Camera icon. Finally, connect the camcorder to the computer with the supplied mini-to-standard USB 2.0 cable.
When the camcorder acknowledges the computer, the camcorder’s touchscreen may ask you to “Select the memory for connection.” (See Figure 18.1.) Your choices will be All (for PC only), Mem. Card A, Mem. Card B, and Built-in Mem. After you make your choice, it will then say, “USB Connection, Do not disconnect the power source.” The computer screen will display an icon that says “CANON” (or “NO NAME”), which it recognizes as an external storage device.
Figure 18.1 Select the camcorder’s memory location from which to transfer files.
As with transferring still photos, once you have made this connection, keep the following points in mind:
Do not disconnect power from the camcorder.
Do not disconnect the USB cable.
Do not turn off the camcorder or the computer.
Do not open the memory slot cover.
Do not change the camcorder’s operating mode.
When you open the CANON icon on a computer, you’ll find this folder and file structure (see Figure 18.2):
CANON (overall name)
DCIM (folder for still photos)
JPEG files (photo files)
AVCHD (folder for video files and metadata)
BDMV (Blu-ray digital media video folder)
BACKUP (folder)
CLIPINF (info folder)
INDEX.BDM (clip file index)
MOVIEOBJ.BDM (clip description file)
PLAYLIST (folder)
STREAM (video clip folder)
MTS files (your video clips)
CANON (folder for thumbnail clip images)
CV INFO
MY_MUSIC (a resource folder)
MY_PICTURES (a resource folder)
Figure 18.2 The memory card will display these folders on your computer screen.
BDMV, which stands for “Blu-ray disc movie,” is the structure on which AVCHD’s organizational structure is based. Within the BDMV folder, your actual video clips are the MTS files (short for “MPEG-2 Transport Stream files”) in the STREAM sub-folder. After they have been transferred to a computer’s hard drive, you will be able to play these files as fully rendered video with an appropriate player.
The other subfolders under BDMV contain associated index files, clip data, playlists, and backup data, all of which are considered metadata (data about data). When the MTS and its metadata files are together, as they are in the BDMV folder, an editing program can locate and identify files with access to all the related data if it needs it. Some programs will find it difficult or impossible to track the time, date, file name, and file number; link continuous files; or even locate an individual file if the metadata is unavailable.
Copy the video and metadata files from the camcorder’s folders onto your computer’s hard drive (internal or external) by dragging and dropping. The quickest and safest way to offload the files from your memory chip is to simply reproduce all of them, but the entire CANON drive may contain music resources and premade photo mattes for green screen that you do not need to transfer. What you really need is the entire AVCHD folder (and the DCIM folder, if you want still photographs). If your software does not need Canon’s video thumbnails, I recommend that you at least transfer the entire BDMV folder to your computer’s internal or external high-speed hard drive.
Keep your source files and archive copies in their original MTS format along with metadata files. You can edit, transcode, or create master versions in other formats, but whenever you want the highest quality, you should do this from the originals. A transcoded intermediate is simply an interpretation of the information in the original MTS file; it is not the original file itself. You may eventually transcode to ProRes422, DNxHD, MOV, or Cineform format if your editing program works best with one of these. But the algorithms for transcoding these files from MTS keep improving and future editing programs may develop even better codecs. So, archiving your media in its original form makes sense.
You should not rename the files unless you use an editing or file utility that provides a means to do this. If you simply rewrite the name of an MTS file on your own, the file will still play as an individual clip of video, but it may lose connections with its metadata, which could have ramifications for some editing programs.
Disconnecting
After you have completed the transfer, do not physically disconnect the USB cable, remove a memory card, or disrupt the camcorder’s power until you have followed the computer’s protocol to disengage the camcorder from the computer. On a PC, right-click the Canon icon in the System menu then choose Safely Remove Hardware as you would when you eject any other USB memory source, or use any of several alternative methods. Wait for the prompt that says “It’s now safe to remove your hardware.” On a Mac, drag the CANON icon to the Eject symbol. After the CANON icon has disappeared from the computer screen, you may then unplug the USB cable. The “Do not disconnect power” message should disappear from the camcorder’s touchscreen and the camcorder can be turned off. Disconnecting improperly could corrupt or permanently damage files.
Verify and test the transferred files before you erase and reformat the memory card or the camcorder’s internal memory. Always reformat your memory cards in the camcorder, never on a computer, to be certain their formatting is completely compatible with the G10/XA10.
Using a Card Reader
If you recorded to removable memory card A or B, it is possible to eject the card from the camcorder and transfer the files to your computer through a card reader. A USB 3.0 card reader or a card slot built into the computer itself may read SDHC and SDXC cards faster than the connection from the camcorder (which is USB 2.0). Make certain the power on the camcorder is completely off before you open the memory slot door to remove the card. Initially press the card inward to release its latch so that the card pops up in its slot.
SDHC cards and readers have been around for a number of years. To recognize SDXC cards, which are somewhat newer, Mac computers need system 10.6.4 or later, and PCs need Windows 7, Windows XP with Service Pack 3 plus KB955704 update, or Windows Vista with Service Pack 1 or later. Your card reader itself must also be compatible with SDXC; otherwise, the computer may want to reject or reformat the card.
It is very important to slide the write protect tab on the memory card to the side marked “lock” immediately after removing the card from the camcorder and before inserting it in an SDHC or SDXC card reader. The computer will recognize the card as a memory device and may attempt to index or write data to it, which you want to prevent by locking the card. Once the computer acknowledges the memory card, follow the same procedures as before, including the proper protocol for removing or ejecting the card when the transfer is complete.
Class 6 or class 10 SDHC and SDXC cards are delicate and expensive, so you want to store them in a protective case when they are outside of the camcorder. Leave them write protected without erasing and reformatting until you have completed and verified the whole transfer process and know that your files are safe and playable on a computer’s internal or external hard drive.
File Management and Viewing
You should be able to transfer the BDMV folder or full AVCHD folder from the camcorder or memory card to the computer via drag and drop. In some cases, your editing program itself will be able to import directly from the memory card and organize source files into its own locations. There are also dedicated software programs that can streamline the offloading and file-management process. On a PC, you might try Pixela Video Browser, the transfer utility included with the G10/XA10. Pixela opens automatically when it senses the G10/XA10 has been connected; click Video File Import to start the process.
ShotPut Pro ($89–$99), made by Imagine Products, is a highly regarded dedicated application for offloading MTS and metadata files. It is available in separate versions for Mac and PC. One of its most important features is that it can verify transferred data by MD5 or CRC checksums for each file, or byte-by-byte comparisons to ensure error-free clones of your source files. ShotPut Pro automates the transfer process. It can copy files to more than one drive at a time, simultaneously creating a backup drive. It enables you to add prefix names and accumulate numbers to filenames. It visually indicates the progress of all operations and ultimately creates a log of transferred files and a verification report. A demo version is available online at www.imagineproducts.com.
With VLC, a free downloadable application that plays almost every kind of media file and works well on most computers, you can preview and play your MTS files just by tapping the file without having to go into an editing program. It is designed by VideoLan, a non-profit volunteer organization based in France with developers in 20 countries dedicated to producing and promoting free open-source multimedia solutions for the general public. Go to www.videolan.org/vlc.
HD-VU ($99) by Imagine Products is another application that enables you to instantly play or preview MTS clips in their original form without transcoding or entering an editing program. Clips can be played full screen or in a scalable window and can be operated by on-screen or keyboard commands forward, in reverse, and at variable speed. They can be viewed with or without an overlaid timecode that displays minutes, seconds, and frames, and you can view metadata for any clip.
Hard Drives
As discussed, an external drive (see Figure 18.3) holding your editing source files should connect to your laptop or desktop computer’s fastest I/O port, which will be USB 3.0, FireWire 800, eSATA, or Thunderbolt. As a source drive for a desktop computer, the alternative is a second or third Serial ATA 3.0 internal hard drive.
Figure 18.3 LaCie 7,200 rpm Quadra external hard drive with 3TB of memory and FireWire 800, USB 3.0 and eSATA connections.
For editing MTS video files, you will need a high-speed 7,200 rpm (or 10,000 rpm) hard drive with a capacity of 1, 2, or 3TB. The LaCie d2 Quadra v3 hard disk has three types of high-speed I/O interfaces for nearly universal connectivity, or you could choose a comparable model with the single high-speed interface that matches your computer. On the LaCie Quadra drive, FireWire 800 has a burst transfer rate of 85Mbps, eSATA has a transfer rate of 115Mbps, and USB 3.0 has a transfer rate of 130Mbps. In each case, this is many times the rate of USB 2.0. The Quadra series is quiet because its outer shape is a ribbed aluminum heat sink instead of a fan for cooling. The ribbed design is adequate if you are editing in a temperature-regulated area that does not get above the 70s. Otherwise, choose a fan-cooled external drive.
On Macintosh computers manufactured in 2012 or later, Thunderbolt is a very high-speed bidirectional terminal that can transmit and receive data simultaneously. It produces data rates of 8–10Gbps, which is significantly faster than FireWire 800, USB 3.0, and eSATA. The Thunderbolt connector fits all recent Macs. However, Thunderbolt drives and multi-disk RAIDs are currently expensive. The LaCie 1TB Thunderbolt Little Big Disk hard drive sells for $429. To accommodate Thunderbolt’s much higher data rates, the Little Big Disk consists of two 7,200 rpm drives within a single enclosure in a RAID 0 configuration acting as a single drive. (This is discussed further in the next section.) It has a 180Mbps data transfer speed. If you are limited to a Thunderbolt I/O port, third-party vendors sell adapters that let you connect slower FireWire 800 and possibly USB 3.0 drives to the Thunderbolt port. The speed will be limited by the FireWire 800 or USB 3.0 drive.
Editing needs high-speed drives, but additional drives for backup, storage, and archiving can operate with slower than 7,200 rpm speeds. Specifically useful for editing, RAID drives offer the highest read/write rates.
RAID Drives
A RAID drive is a combination of two or more hard drives programmed to operate as a single drive with faster data rates, greater reliability, or both. Preconfigured RAIDs can be purchased, or you may purchase a RAID enclosure that holds multiple drives, which can be striped in various different ways to behave as one drive.
There are various types of RAID:
RAID 0 needs at least two drives. A single MTS file is written or read as fragments across the two drives working together in parallel. This process creates a higher bandwidth with faster read-and-write rates than a single drive. The 1TB LaCie Little Big Disk ($426) uses a RAID 0 configuration so its two drives in a single enclosure collectively handle 180Mbps with input/output through Apple’s ultra high-speed Thunderbolt interface, available on Mac computers manufactured in 2012 or later. Performance speed is great, but RAID 0 does not have error checking. Also, with double the number of drives, there is double the risk of data loss if an error occurs in one of the drives.
RAID 5 needs at least three drives and is typically set up with four. This configuration builds enough redundancy into the distribution of the data that the system will operate as long as all but one of the coordinated drives are operating correctly (two out of three, or three out of four). The high level of performance is reduced if one drive has errors, turns off, or crashes entirely, but data files are not lost or corrupted. Sans Digital is a well-respected company that makes towers that hold multiple drives configured to operate as RAID 5.
Portable Drives
For immediate and intermediate storage, portable hard drives are very viable ways of carrying your footage from one computer to another. For media files that are worthwhile, you should keep more than one copy when you have time for archiving. It may be that one version is on a very large, fast hard drive or RAID, providing source files for editing, a second duplicate version on another hard drive as backup, and an archived version backed up as data files on Blu-ray discs.
For editing small projects or carrying projects from one computer to another, the 64GB and 128GB Kingston HyperX 3.0 Data Traveler flash drives connect to a USB 3.0 port. The 64GB drive sells for $124, and the 128GB for $259. Solid-state drives are expensive per gigabyte, but the HyperX is fast enough for editing on computers with USB 3.0 terminals and useful for transfers on computers with USB 2.0.
If you need a portable drive of greater capacity, you might consider a 500MB or 1TB hard drive specifically designed for portability. LaCie’s Rugged series is compact, air cooled, and shock resistant; has a protective rubber rim on its perimeter; comes with various interface configurations; and receives its power from the computer when connected. (See Figure 18.4.) The 7,200 rpm 500GB model suitable for editing has connections for USB 3.0 and FireWire 800 and sells for $169. The 5,400 rpm 500GB model suitable for archiving sells for $117. Other major brands of hard drives to consider are Seagate, Western Digital, Iomega, IoSafe, and G-Technology. Portable drives are useful for general mobility with a laptop, as well as for more extensive travel, storing separate projects, or interfacing with many computers such as at home, school, work, or a postproduction facility.
Figure 18.4 LaCie 7,200 rpm 500GB portable “rugged” hard drive.
Archiving
Archive your source files in their original MTS format along with their metadata files. Once you have set up a folder of MTS source files, you may edit and create intermediate or editing versions in whatever other formats necessary from the originals.
Class 6 and class 10 memory cards themselves are much too expensive per gigabyte to be used to archive files. Their function is to be used for video acquisition and transfer, and then reused for acquisition again. Holding data on original SDHC or SDXC cards offers some short-term protection if you wait until transferred files are verified and playable and don’t erase them until you are ready to re-use them. For long-term archiving, a 500GB portable drive, or a 1TB, 2TB, or 3TB external hard drive, is much more efficient and economical by a factor of 10:1 or 20:1. Table 18.1 compares the cost per gigabyte of various storage media.
Table 18.1 Comparative Cost per Gigabyte
Hard-drive prices vary considerably by manufacturer, the read/write speed of its protocol, and whether the drive runs at 5,400 rpm, 7,200 or 10,000 rpm. So, the figures in Table 18.1 are a very rough guide. The least expensive hard drives will run at 5,400 rpm and may handle only USB 2.0. They will take longer to load and will be suitable for archiving, but probably not fast enough for editing.
Nobody quite knows with hard drives or solid-state memory drives what their life expectancy is. So, source files worth keeping are usually archived on one additional medium like digital data tape or data discs. Up to 8.5GB from an SDHC memory card can be stored as data on a dual DVD-9 disc. If you are using 32GB SDHC cards or 64GB SDXC cards, you can still take combinations of files in roughly 8.5GB bundles and store them as data files on fairly inexpensive DVD-9 discs.
You would need four DVD-9s to hold the information from a 32BG memory card. Save your metadata files for your entire AVCHD folder on the 32GB memory card, keeping the metadata at least on the first disc, or since the files are small, repeat the entire set of metadata folders on each additional disk.
Blu-ray discs used for data storage can hold 25GB per single layer disc or 50GB per dual layer disc, provided you have a Blu-ray deck that will handle dual layer. Blu-ray is an optically recorded disc, so there is no danger of magnetic erasure. Using Blu-ray for data storage (as opposed to a playable Blu-ray disc) is a very viable long-term storage solution, both in terms of its capacity per disc and cost per gigabyte. (See Figure 18.5.)
Figure 18.5 The LaCie d2 Blu-ray drive records, rewrites, and plays 25GB and 50GB discs.
In comparison to the complexities of storing AVCHD on hard drives and discs, it may seem that archiving analog or digital videotape was an easy process. Whatever you shot on videotape existed in a physical format that could be stored on a shelf. But in actuality, having to depend on whether that tape would be playable one year, three years, five years, or 10 years later was very problematic. Over time, tape layers stick to each other, and as the binder ages, the magnetic emulsion begins a process of disintegration and dropouts occur. The magnetic signal itself erodes over time. The Mylar or acetate base of the tape expands and contracts with temperature and humidity, causing tracking and timebase errors. Playing or rewinding a videotape always entailed the risk of jamming, scratching, stretching, demagnetizing, or destroying the video signal. Physically locating and cueing a shot was a linear, time-consuming process. Capturing footage from videotape involved setup time, a cue-in time, a cue-out time, and real time for the transfer. As for storing, you would need a shelf of 16 to 40 20-minute Betacam tapes or a lot of mini-HDV tapes to store the equivalent of the video on one SDHC memory chip the size of a postage stamp.
With AVCHD, you have some protections built in and some useful parts of the process that you never had in the tape era, including directories that tell you the time and place that something was photographed and the ability to play the 98th clip in a folder without having to queue through 97 previous shots. Depending on the speed of your SDHC or SDXC cards, the speed of your hard drive, and the speed of your computer, it is possible to transfer seven hours of recorded video to a computer in a fraction of the actual running time.
Noise Reduction
Neat Video Pro ($99–$199) is a digital noise-reduction plug-in for Premiere Pro, After Effects, Final Cut Pro, Motion, Vegas Pro, and Pinnacle Studio. It removes visible noise and compression artifacts from the video image caused by excessive gain or high compression. Neat Video Pro analyzes 1,920×1,080 high-definition video, constructs an individualized noise profile, and allows standard and advanced adjustments to be made by the user. It saves individual noise profiles and noise-reduction settings for reuse with similar footage. Once the settings have been adjusted, Neat Video takes a considerable amount of rendering time for each clip, but it is an effective way to rescue marginal footage.
Standard Definition
The G10/XA10 cannot directly record standard-definition video. But the camcorder does have the ability to play its high-definition clips as standard definition and output them through the A/V analog video output and cable for viewing, for recording on an SD deck, or for quick-and-dirty editing on standard-definition software.
If you want to routinely transfer to a computer running standard DV editing software, you may need an analog-to-DV converter such as the Canopus/Grass Valley ADVC55 ($179), ADVC 110 ($218), or ADVC 300 ($429). The more expensive models offer timebase correction to stabilize the image and individual control of brightness, contrast, hue, and saturation. The ADVC 300 has digital-to-analog as well as analog-to-digital operation. The converter takes the camcorder’s AV signal and converts it to DV out through a FireWire 400 cable. Many older computers and editing software programs accept this interface.
For Web quality, MTS files can be batch transcoded to standard definition inside the camcorder itself. Clips can be selected for transcoding individually or in batches based on record date. Make certain there is a memory card in slot B. Then, attach the compact power adapter and turn on the camcorder. Toggle the Camera/Playback button to Playback; then tap the Index icon (lower left), select the Internal, A, or B memory tab, select the date, and press the WEB button (on the left bezel of the touchscreen) to convert all scenes on that date to SD on memory slot B.
The standard-definition clips can be transferred to a computer or can be uploaded wirelessly to a video-sharing website if slot B contains an Eye-Fi card. Your high-definition MTS files still exist in memory slot A or the internal memory. If you want individual clip selections and control of output quality, there is a more complicated route for converting HD to SD. To use it, toggle the Camera/Playback button to Playback; choose Edit > Convert HD to SD; select Internal-to-B or A-to-B; and select the clips individually or by date. Then choose Next (to review selections); tap the Tool icon; set the bit rate (quality); touch Return twice; and press START to convert the scenes. A confirmation screen will appear at the end of the HD-to-SD rendering process.
If your computer is capable of handling MTS files, the best way to produce a standard-definition version is to stay in high definition all the way through editing on a computer, and then use your editing software’s options to output to standard definition, including various DVD and Web formats. You could master the entire project in high definition and transcode the master with software like Apple’s Compressor or with third-party conversion software.
Deleting Files
The edit function in the G10/XA10 provides you with an opportunity to selectively erase individual files before transferring everything to a hard drive. To use it, first move the Camera/Playback button to Playback. Then choose Edit > Delete and choose to delete by date, by scene, or all scenes.
If you do much deleting and editing in the camera, the memory card will eventually have fragmented memory, which will reduce its recording performance. If recording stops or you suspect reduced performance, save your data to a hard drive, verify the transferred data, and re-initialize your memory card in the camcorder.
If you have many unwanted clips, your other option before editing is to offload all your video files to a computer’s hard drive. Then, individually select and delete unwanted MTS files from the editing source drive so they will not be taking up space. You do not have to bother with deleting the associated metafiles because they take up minimal space. In this way, you are certain to have the associated metafiles your programs may need (at the expense of retaining tiny metafiles for shots you have discarded).