Geometric Viewing Distortion: Head Position

It’s important for a 3D movie audience to view the stereoscopic pair of images correctly. When watching a 3D movie, the audience must, ideally, keep their head perfectly level.

If a viewer angles or skews their head, their eyes are not level and that causes a vertical misalignment of the stereoscopic pair. The brain tries to independently realign each eye to compensate for the problem and eventually the viewer gets eyestrain.

Geometric Viewing Distortion: Seat Position

The seating position of the audience is important.

In this overhead diagram, Viewer A has the best seat because it’s centered on the screen. Viewer B, sitting far to one side, sees a distorted view of the 3D stereoscopic pair.

Viewer A will see the stereoscopic pair without any geometric distortion. Viewer B sees a stereoscopic pair (in an exaggerated illustration) where each image in the pair is a different size and shape. This distortion increases as the Depth Bracket expands.

Geometric Viewing Distortion: Distance From Screen

Visual fatigue caused by divergence is worse for viewers sitting closer to the screen.

Sitting closer to the screen causes the viewer’s eyes to diverge more when watching a stereoscopic image pair with a measured screen parallax greater than 2.5 inches. See Appendix C for a complete explanation.

Geometric Photography and Projection Distortion

In 3D photography and display, the images in the stereoscopic pair must be perfectly aligned or rectified. Digital cinema has solved most of these alignment problems but they can still cause brain strain if left unchecked. Without getting too technical, here is a list of the most common problems that can provoke visual fatigue and brain strain.

1. Keystone distortion. Toe-In cameras, often used for live action 3D production, distort the stereoscopic pair.

This is an overhead view looking down on a pair of Toe-In cameras photographing a wall.

The two cameras are not parallel so each image will keystone in opposite directions (these illustrations are exaggerated to reveal the problem). This creates a stereoscopic image pair with geometric distortion. An audience will have difficulty watching image pairs with excessive opposing distortion. This distortion can be greatly reduced in digital postproduction. Using HIT cameras eliminates the problem.

2. Alignment errors. The stereoscopic pair can be misaligned vertically or rotationally.

Any misalignment on any axis is immediately difficult to watch.

3. Scale mismatch. The stereoscopic pair is not matched in size.

This occurs in live-action 3D because the cameras’ prime or zoom lenses are mismatched in focal length due to a manufacturing error or incorrect calibration. A similar problem can inadvertently occur during a rack focus. Some lenses, especially zooms, may “breathe” or change focal length slightly during focus shifts, which affect the image pair’s sizes. Any scale mismatch is immediately difficult to watch although the problem can be corrected in postproduction.

4. Focus inconsistencies. One image in the stereoscopic pair can be out of focus.

This problem occurs when one of the stereoscopic camera lenses is improperly focused, the lens bokeh is mismatched, or the lenses are not parallel to the image sensor.

5. Synchronization errors. 3D live action cameras must be precisely synchronized so that the stereoscopic pair is photographed simultaneously. If the cameras are even minutely out of sync the stereoscopic pair’s action won’t match. This turns a temporal displacement into a spatial disparity and on-screen movement looks rubbery.

These errors are very difficult to watch because each eye receives an image from a different moment in time.

The visual geometry of the stereoscopic pairs doesn’t match, which instantly confuses the perception of depth.

6. Left/right reversal. The left and right eye images of the stereoscopic pair are accidentally reversed in projection.

This problem destroys the 3D depth and is instantly impossible to watch. You can temporarily solve the problem by wearing 3D glasses upside down.

7. Reflection/polarization inconsistencies.

For a variety of technical and mechanical reasons, reflections or specular highlights may appear in one eye’s image and not the other eye. This happens with objects like moving water and any shiny surface. A variety of special lens filters can help alleviate these problems. Lens flares may occur in only one lens or appear differently in each lens. Any of these inconsistencies can lead to retinal rivalry, which can cause brain strain. Retinal rivalry is explained on page 86.

The retinal rivalry is easy to see when the left and right eye images are separated.

8. Lens design geometry errors. All lenses, no matter how carefully manufactured, have minute optical flaws that can distort the image’s geometry or shape. These minor errors go unnoticed until two lenses are used to simultaneously photograph the same object in 3D.

9. Color or brightness mismatches. This is not a geometry problem, but the stereoscopic pair must be matched in brightness and color. Due to exposure errors or mismatched lenses, the image pair may have color or brightness inconsistencies from eye to eye.

Even minor variations in brightness can be quite uncomfortable to view. Although color inconsistencies in the stereoscopic pair are bothersome, minor mismatches may go unnoticed.

Horizontal Violations

Horizontal window violations are visually minor because we only perceive depth information from verticals. For an explanation of this, see Appendix E. A horizontal violation occurs when an object in front of the window in the Personal Space is cut-off or cropped by the horizontal upper or lower frame of the window.

Put on your 3D glasses. The actor appears in front of the window but the actor’s head and shoulders are cropped by the horizontals of the window behind the actor. The violation: how can a foreground object be cropped by a window that’s behind it?

Essentially, the audience sees the screen image like this. The window is behind the actor and the actor’s head and torso appear to be unnaturally sliced off. Fortunately, the audience’s brain immediately compensates for this physical impossibility.

The audience assumes the window curves around the object. This is purely an optical illusion created in the audience’s head. This overhead view illustrates how the audience imagines the window crops the foreground subject. There are two ways to reduce or eliminate this horizontal window violation.

SOLUTION #1: Move the entire window.

Moving the window forward in front of the object eliminates this minor window violation.

Put on your 3D glasses and look at the violation and its corrected version with the window moved in front of the actor. It’s also possible to leave the window alone and move the actor behind the window, but this may create a new problem. Moving the actor back may increase the size of the Depth Bracket and create hyper-divergence in the background.

SOLUTION #2: Recompose the shot.

Avoid window violations that crop the subject on the frame’s top and bottom. Cropping along the bottom frame line is unavoidable, but cropping on top can be minimized or eliminated.

In a close-up, the audience notices the violation because they’re looking at the actor’s eyes, which are near the upper horizontal frame line. The lower horizontal window violation is farther from the actor’s eyes and more likely to be ignored.

Vertical Violations

Vertical window violations are more problematic because we perceive our depth information in verticals. See Appendix E for an explanation. A vertical stereoscopic window violation occurs when the vertical sides of a background window crop a foreground object.

To the audience, the foreground object appears to be sliced off vertically. The basic violation is the same: how can a foreground object be cropped by a window behind it?

There are three ways this problem can be eliminated using the stereoscopic window.

SOLUTION #1: Move the entire window.

The entire window can be moved in front of the subject removing the violation. The left photo is the violation and the right photo is the correction with the window moved in front of the subject.

SOLUTION #2: Move part of the window.

Placing part of the window in front of the subject corrects the window violation. Most of the window remains behind the subject.

Moving one side of the window usually creates the illusion that the window is angled.

The left side of the window was manipulated forward using the above configuration.

The violation has been eliminated by moving only the left vertical masking.

SOLUTION #3: Another solution is to leave the window alone and move the subject behind the window, but this may create a new problem. Like the horizontal violation solution, moving the subject back may increase the size of the Depth Bracket and create hyper-divergence in background image pairs.

Vertical Window Violations and Retinal Rivalry

Other vertical window violations can occur when the vertical frame edge blocks one eye’s view of an object’s image pair.

Look at this photo with and without your 3D glasses. Your right eye can see the screen left light pole but your left eye can’t. You feel like you want to peek around the left frame edge to find the missing pole.

Because the pole only appears in one eye, it may seem to visually vibrate. This is called retinal rivalry. Human vision prefers to see a complete image pair; one image for each eye. When only one eye sees an image, the brain gets confused.

Retinal rivalry can be caused by inappropriate camera compositions, staging mistakes, or any mismatch in the image pair. Bright, small background objects like lights at night are particularly problematic. If the retinal rivalry involves the subject, the audience will become aware of the problem immediately. If it’s only a non-subject, it may go unnoticed because the audience is looking elsewhere in the picture. There are two solutions to the retinal rivalry problem.

SOLUTION #1: Add vertical masking.

A solution is to introduce window masking over the pole’s image that only one eye sees. Neither eye sees the pole and the retinal rivalry is eliminated.

This added masking also pushes the left side of the window away from the viewer. Although the retinal rivalry problem is solved, the window placement may look odd or unmotivated.

Window manipulations can occur on the edit from shot-to-shot or during a shot. Cutting to a new shot is a perfect opportunity to invisibly change the window position. Window moves during a shot can go unnoticed if they’re motivated by camera or object movement.

SOLUTION #2: Recompose the shot.

Adjusting the composition eliminates the window violation and the retinal rivalry.

The pole’s image pair is now visible to both eyes.

3D Point-of-Attention Jumps Within the Shot

In stereoscopic 3D, the subject can be placed anywhere along the X-, Y-, and Z-axis. This can force complex point-of-attention jumps within a shot that may cause visual fatigue.

This 3D picture has two actors but there isn’t a point-of-attention jump because both subjects occupy nearly the same area of the frame (on the X- and Y-axis) and the same depth (on the Z-axis). No visual jump will occur.

In this 3D shot, the two subjects’ positions on the X- and Y-axis are fairly similar, but the Z-axis will require a large jump as the audience shifts its point-of-attention from the foreground subject to the background subject.

This diagram reveals the point-of-attention jump along the Z-axis. The audience will move its point-of-attention between the foreground subject at –25 pixels and the background subject at +25 pixels. The size of the Depth Bracket is a reasonable 50 pixels (2.5 percent), but the Z-axis jump will be uncomfortable as the audience shifts its point-of-attention between the two subjects.

There are four ways to avoid eyestrain caused by extreme point-of-attention Z-axis jumps within the shot:

1. Restage the scene.
2. Reduce the interaxial distance.
3. Shift the convergence.
4. Compositing techniques.

SOLUTION #1: Restage the scene.

Restaged, the scene has less Z-axis contrast.

Moving the foreground actor from –25 to +5 reduces the Depth Bracket to 20 pixels or 1 percent. Now, the audience has a smaller Z-axis jump from subject to subject. This lessens the eyestrain but the shot’s composition has been changed, which may compromise the shot directorially. Fortunately, there are other ways to solve the Z-axis jump problem.

SOLUTION #2: Reduce the IA. Reducing the IA alleviates the problem because it decreases the parallax in the stereoscopic pair. The actors are in their original positions but the reduced IA shrinks the Depth Bracket’s size.

Reducing the IA shrinks the Depth Bracket to 25 pixels or 1.25 percent. The disadvantage is the objects lose 50 percent of their volume.

SOLUTION #3: The third method for reducing point-of-attention eyestrain is to shift the convergence from one subject to the other as their importance changes during the shot. The convergence shift is usually linked to the lens focus as it racks from one subject to the other.

The director must decide when the audience’s point-of-attention will shift from one subject to the other and change the convergence at the right moment. This convergence technique places the more important subject on the ZPS, which is the most comfortable viewing position along the Z-axis. When done correctly, the convergence shift from subject to subject occurs simultaneously with the audience’s change in its point-of-attention. This technique minimizes or eliminates the point-of-attention subject jump.

This pair of diagrams illustrates the convergence shift as the audience’s point-of-attention changes from the background subject to the foreground subject. This keeps the more important subject at the ZPS. When the foreground actor is placed at the ZPS, the background subject’s parallax is +50, which will cause hyper-divergence. You must be confident that the audience will ignore the background subject at +50 pixels in favor of the foreground actor at the ZPS.

The advantage to shifting the convergence during a shot is that multiple subjects can be placed at various depths without causing eyestrain since the audience never has to move their point-of-attention from the ZPS. The disadvantage is the danger of shifting convergence at inappropriate times, which will make the point-of-attention jumps worse and create eyestrain.

In general, the best technique for reducing or eliminating Z-axis subject jumps is to use all three techniques and find a combination that maintains the composition, retains good looking volume, and keeps the audience comfortable.

SOLUTION #4: The fourth technique to correct Z-axis point-of-attention jumps is to use digital compositing. Often this is done with a multi-rig technique. Multi-rigging allows each object in a shot to have independent parallax settings so the ideal Depth Bracket can be created without affecting the objects’ volume or size. This avoids the side-effects of adjusting the IA, which can reduce or eliminate volume.

In CG animation, multi-rigging is a relatively simple procedure because objects are rendered separately and then composited together into one shot. This makes it possible to give each object its own IA. In live action, this same technique is used by shooting objects separately on green screen and compositing them together. This style of shooting is costly and needs to be planned well in advance.

A version of the multi-rig technique is used in the conversion of 2D movies to stereoscopic 3D. The difference is that each object in a 2D shot must be isolated by rotoscoping. Using rotoscoping, each object can be independently manipulated in depth to create the ideal 3D composition.

Put on your 3D glasses. The “before” shot was photographed using a single, conventional 3D rig that photographs the entire scene at once. The point-of-attention jump from foreground to background may be too large.

In the “after” shot, using a multi-rig technique, the adjusted foreground actor has been recombined with the scene. The point-of-attention jump has been reduced. The multi-rig preserves the original IA and volume while reducing the parallax to keep both actors within a comfortable Depth Bracket.

The multi-rig technique cannot solve every depth problem. Spatial errors can occur if the multi-rigged Z-axis rearrangement conflicts with the reality of the shot’s depth. This photo illustrates how multi-rigged 3D can look artificial. The background actor appears to be a tiny person hovering above the foreground actor’s shoulder.

Manipulation of a scene using multi-rig techniques presents all kinds of typical and unique problems that occur in any postproduction compositing situation. Experience is needed to understand all of the issues involved.

Any of the four techniques (restaging, IA reduction, convergence shifts, or multi-rig) can be used in various combinations to solve point-of-attention jumps.

3D Point-of-Attention Jumps from Shot-to-Shot

Editing presents new challenges in 3D. Some filmmakers feel that the best 3D experience is to duplicate real-life vision, which has no edits at all. But editing is a necessary filmmaking tool in both 2D and 3D so it’s important to understand the visual problems 3D editing can create and prevent them from occurring.

As the audience watches a series of shots in 3D, its point-of-attention will be drawn to the subject. If the subject jumps too much from shot-to-shot along the Z-axis, visual fatigue can set in.

Here are three shots. The editing pattern cuts from Shot 1 (distant blue actor) to Shot 2 (closeup of a red actor), and then Shot 3 (distant blue actor). Both subjects are in the center of the screen so when these shots are edited together, the audience will not move its point-of-attention on the X- or Y-axis. But the audience will move its point-of-attention along the Z-axis to and from the blue actor in the background to the red actor in the foreground. The abrupt jump created by the intercutting of these Z-axis depth changes can create visual fatigue.

These three shots will be used to demonstrate various solutions to this problem. The visual jumps can be reduced or eliminated using one of five solutions:

1. Accept the jump.
2. Restage the scenes.
3. Reduce the Depth Bracket.
4. Subject Convergence consistency.
5. Shot-to-shot blending.

SOLUTION #1: Accept the Z-axis depth jump. Don’t fix the problem. You intend to ‘jolt’ the audience so the point-of-attention jump is exactly what you need.

Shot 1 is a distant, wide shot. Shot 2 is a close-up designed to startle the audience. On the edit, the subject jumps 60 pixels or about 3 percent. The instantaneous Z-axis subject jump from the background to the foreground adds visual impact that the story requires.

SOLUTION #2: Restage the shots. Restaging the subjects closer together along the Z-axis reduces the subjects’ parallax and the Depth Bracket’s size. Of course, this also changes the shot’s composition.

The actors are closer together along the Z-axis making the point-of-attention jumps from shot-to-shot smaller and less stressful. The subject jump is 25 pixels or about 1.25 percent. But filmmakers may find the new compositions weak and the visual storytelling compromised.

Peripheral non-subjects with larger parallaxes may remain, which can preserve a larger Depth Bracket.

SOLUTION #3: Reduce the Depth Bracket. The IA controls the size of the Depth Bracket. As the IA is reduced, the parallax for each shot decreases.

Now, the three shots have a smaller Z-axis jump of 25 pixels or 1.25 percent. But reducing the IA is always at the expense of stereoscopic volume. Too much IA reduction can eliminate volume, making objects look like flat cardboard cutouts. The IA balance between Depth Bracket size and volume is discussed on pages 128–132.

SOLUTION #4: Subject Convergence consistency. The jumps can be eliminated by keeping the audience’s point-of-attention at the same Z-axis depth in every shot. This consistent depth can be anywhere along the Z-axis but is most typically set at the screen surface or ZPS.

The convergence is locked to the subject, no matter where the subject is located in depth. In this example, all subjects have zero parallax and are placed at the ZPS.

The audience’s point-of-attention will predictably remain with the subject. By placing the subject of every shot at the same depth, all of the subject point-of-attention jumps are eliminated. This plan is called Subject Convergence. Its advantages and disadvantages are discussed beginning on page 122.

SOLUTION #5: Blending. The point-of-attention jumps from shot-to-shot can be blended or softened by manipulating the convergence for each shot in postproduction using HIT. This is sometimes called depth grading. Blending techniques can be used in a variety of ways, which are somewhat subjective. The most effective technique will depend on the specific shots and their time length on screen.

There are three basic types of blending:

1. Adjustment
2. Bridge
3. Ease-Out