7

Cinematography for Games

The term “cinematography” was created in the film industry to describe the process of creating images on film; as the use of film has diminished somewhat with the widespread availability of digital video and high definition formats, the term “cinematography” has expanded. Now it is a generic term encompassing all aspects of camera work, including the creative aspects involved with making aesthetically pleasing images and the technical aspects involved with using cameras, lights, and other equipment.

Cinematography has been around for a very long time with many influential directors and camera operators, but the field has changed very little—even with the current trend of using digital/ high-definition digital mediums. Though cinematography is usually thought of as the elements that constitute the composition and style of a movie, it can also incorporate lighting techniques, camera movement, and even postproduction methods for creating the look that the director or cinematographer (usually a camera operator with lots of experience) is going for.

Though most techniques were designed for shooting movies, the visual elements of using basic cinematography techniques can also be applied to video games; both are visual mediums and both involve style, movement, and composition. When discussing the various elements of cinematography, most filmmakers refer to the work of Joseph V. Mascelli and his written work regarding the field (check out the seminal The Five Cs of Cinematography, Silman-James Press, 1998).

7.1  Five C’s of Cinematography

Though Mascelli wrote his book more than forty years ago, the basic principles he laid out still apply to visual mediums today. He came to the conclusion that the creative decisions made when shooting a film revolved around what he called the “five C’s of cinematography.” These C’s include: camera angles, continuity, cutting, close-ups, and composition.

The use of creative camera angles in the video game process can be as easy as shifting the perspective of the gamer from first person to third person or as complicated as involving multiple camera angles from various viewpoints. Either way, the choice of camera angle can affect the atmosphere of the game, the intent and meaning the designer intended for the game, and even the depth/feel of the level/scene. A basic example of this would be using a low camera angle to represent the point of view of an insect protagonist. From this angle, everything would tower over the insect and create the feeling of vulnerability. Questions that a producer should be asking when determining the camera angles to use include, “What feeling/point-of-view do we want to create for the gamer?” and “What is the context of the scene?”

One of the major advantages a game cinematographer has over a film cinematographer is the actual “camera”. Because a physical camera has size, it cannot be placed just anywhere within a scene. This limits the types of shots that can be achieved. In the gaming world, the camera has no size but is simply a point of view, so the camera can be placed literally anywhere within the shot. This can be used to achieve a level of shots and scenes that have never been seen or used in the film industry and can add great cinematic depth to the project.

Considering the context of a level or scene is a great way to make technical decisions regarding game cinematography (and maybe should be the sixth “C”). For instance, if your protagonist has just been in a helicopter crash and is trying to crawl from the wreckage, a wobbly/skewed camera angle could represent the disorientation and dizziness the character would be experiencing. Being consistent in the game with the use of creative camera angles helps keep the game on point with continuity.

The continuity aspect of cinematography involves keeping a consistent tone throughout the production. Though it can be thought of as limiting, continuity can be maintained while using a variety of camera angles and composition. Think of it as being consistent with the scene rather than consistent (or one-track) throughout the production. A lack of continuity with cameras creates gamer confusion and diminishes the overall production value of a game. It’s important that the look and feel of a game remains consistent so that gamers will become immersed and enjoy a cinematic experience when playing. This can be achieved by staying with the “line of action”, or the direction in which the game play will be heading.

Because the placement of the camera will be engineered during production, it is necessary to include notes about each level regarding the camera work. The notes should include the context of the level, any key creative decisions regarding the look and feel of the scene, and list any changes that should occur there.

In the film industry, the term “cutting” usually refers to the postproduction aspect of editing. A film editor can get involved with a movie while it is still in production—mostly to insure that continuity is taking place and that the finished footage will have the necessary shots to cut a finished version of the film. Though this may be a less pertinent aspect of game cinematography, you can think of cutting in the game industry as editing as you go. Once you have a previsualization of what the level/scene should look like, you can then make the decisions regarding the other elements of game cinematography that will essentially “cut” the scene to look the way intended, then implement these decisions in production. Again, as most of these decisions will have been made in preproduction (with the script and storyboarding processes), the engineers will already have a plan to implement in level design.

The last two C’s of cinematography involve framing. “Framing” a scene basically means the manner in which you position the camera and the proportions of the objects that are within the frame. The use of close-ups is one of the key ways to frame a scene. Another method of accomplishing this is to use the Rule of Thirds (discussed later) and the creative use of different kinds of shots to create variation within the continuity of the game.

Making sure to include true close-ups is very important in the film industry, as this is usually the method of choice for relaying intense emotion from the character (nothing beats a good close-up of the character’s expression). Close-ups are also important to use when identifying key props or items in a scene. Though this approach might work only in certain gaming situations or cut-scenes, it is still a powerful way to illustrate the emotion of a scene, the personality of a character, or a key element of the story.

The final C that Mascelli spoke of is composition, which is the placement of the individual items/people within the shot. There are several approaches for achieving good composition, but most use the principal of importance: that is, the more important the item/person, the bigger it is in the shot. Alfred Hitchcock used a similar strategy for his camera work by stating, “The size of an object in the frame should equal its importance in the story at the moment.” Cinematographers also believe that certain positions within a frame have more importance than others. This is usually called the Rule of Thirds.

7.2  Rule of Thirds

The Rule of Thirds (sometimes called the Rule of Three) is the basic technique of dividing an image into nine equal parts by imagining two equally dividing horizontal lines and two equally dividing vertical lines crossing the frame.

The four places where the vertical and horizontal lines cross are thought of as key positions for items within the frame. The use of this rule creates an image that is aesthetically better to the eye and naturally draws attention to that area of the frame. This technique has also been used as a way to misdirect a viewer; if the gamer is focused on the prop or person in the center of the screen, he or she can be led to believe that that item is the major area of importance rather than, say, a background item. This can add a sense of ambiguity or mystery to a scene.

7.3  Achieving the Look

When you visualize the way your game looks in your mind, is it in color or black-and-white? It is sharp or grainy? How about the use of color; do you think of the scenes in bright pastels or in muted monochromatic colors? What about lighting? Is everything brightly lit or shrouded in shadows? These are some of the major questions you must ask and discuss with the creative director to get the look you desire within the game.

Think of the early concept art that you developed in preproduction as your primary means to locking down the style you want to achieve. When reviewing the design of backgrounds and characters, look at the individual elements within the piece. It’s important to be able to answer the question “Why?” when an artist wants to know what does or does not work within a concept art piece. Even in the film industry, a look is often determined by the artwork (even when the art is applied as an effect in postproduction). The film Three Kings featured a simple bleach bypass process when developing the film that added the element of harsh lighting, intensity, and frenetic movement in the movie. This was completely engineered in postproduction, but was conceived from the film’s onset. This is the mindset and process that must be followed when crating a cinematic game: conceive the look, design the look, then implement the look.

Another consideration that affects the look of your game is the choice of lenses that the virtual camera will use. Your designers must discuss this issue: How much of the world around the game character do you want the gamer to see? Good cinematographers use certain lenses to achieve certain styles in a shot. For instance, because of the extreme depth of field used in most video productions, most documentaries have a very “live” look to them. As most of the frame is in focus during video production, this means that the action being shot will seem real. After all, this is how the human eye works—we look at something and we see it in focus. In the film industry, this has become one of the major factors that determine whether a digital video (or high-definition [HD] video) production has a “film look”. Most film cameras use lenses with a limited depth of field (meaning that a certain degree of the background will be out of focus), so digital video DV/HD productions have started limiting the focus within frame as a means to mimic shooting with film. This technique can also be applied to a game.

Let’s say you are developing an FPS that takes place in the Iraq conflict. By keeping the entire frame in focus and using a jerky camera style, the game have a definite live feel—much like a documentary. Conversely, a great cut-scene for the same game illustrating a grand panoramic shot of the entire aftermath of the battleground would be done with a degree of slow, smooth camera movement coupled with the use of limited focus for distant objects. The use of focus/blur is yet another key way that cinematography can be used within the game to achieve a cinematic look.

The look of the game can also be affected by any effect that has been used in conjunction with the camera style. For instance, a level that takes place in the desert can have the effect of heat waves/ripples rising off metal objects. A character when struck by a bullet or fist can suddenly cause the scene to shift out of focus or become dimly lit as the character falls into unconsciousness. You can also think of adding effects much like a film cinematographer would think about using filters on a camera.

Think about some of the looks that are achieved with some of the most popular filters—color, polarizing, and neutral density— and how that look could be applied to your game. Mimicking the application of a lens filter can add cinematic depth to the visuals of the level.

7.4  Lighting in Games

Though part of the look of the game, lighting also refers to the technical aspects of cinematography. This includes the positioning of light, the intensity of the light being used, the placement of shadows, and the effect that the light will have upon the camera. Most of these elements can also be dealt with during the concept phase of getting the art department on point early on, but some of these must be also be thought of in a technical manner. For instance, when a character looks directly at the sun, can you still see other things around him or her, or does the sun blind the character? Maybe the sun creates a sort of lens flare on the camera.

Perhaps light will be a technical feature that’s intended to affect the gamer during game play. For instance, in the game Elder Scrolls IV: Oblivion, most locations are far too dark to explore without the use of a “light spell” or flashlight. In the case of this game, the designers have made the lighting a game play element.

Lighting is also one of the ways that the game designer can affect the mood of the level. The use of a lot of shadows and limited visibility is a great way to make a game seem eerie or build tension. This is a staple of conventional horror games like Silent Hill and Resident Evil. Conversely, a lot of light can add a very sterile look to the scene. Many of the accomplished role playing games have made a fine art out of the use of dark levels. One of the oldest methods for lighting in the film industry revolves around the concept of “three-point lighting”.

Three-point lighting is the most basic way of shooting a scene in the film industry; it refers to setting up three lights on the set: key light, fill light, and back light. The key light is in front of the action and illuminates the major elements of the scene. Think of it as the spotlight that is hitting the main character or object in the scene. It is usually the brightest light used and sets the tone (usually by intensity) of the lighting scene. In a game, the key light is probably the omnipresent light set at a distance of infinity that illuminates everything evenly. The remaining lights are used primarily for mood.

The fill light hits the front and side spots of the frame that the key light does not. Typically, it is not as bright as the key light and is used to draw attention to any detail that is missed by the key light. Most cinematographers use this light to complement the key light (sometimes it is entirely omitted, but then it would only be two-point lighting).

The back light, as you would expect, is placed behind the items in the scene to create a degree of “rim light” around the subjects. The back light can be adjusted to either blend an object into a scene or separate/highlight the object. The way the back light is used is an aesthetic choice and can be manipulated to achieve the mood and look you want.

Though the use of three-point lighting is a fundamentally basic style in filmmaking, it can make a great jumping-off point for other styles of lighting. Start with this type of lighting, then adjust the light positioning, or quantity/intensity of lights to get the look you are trying to achieve. Because most games are in 3D, lighting a room or area in a game is quite different than lighting the set in a film. There will be situations therein that a movie’s gaffer or cinematographer could only dream of. Attention must be paid to the spaces/places that normally would not be in the field of vision of a film viewer and treated with equal gravity in order to maintain consistency and continuity.

7.5  Camera Movement

Another key element in cinematography is the use of camera movement. For a physical camera, this means performing dolly and crane moves, panning and tilting the camera, and arranging the set to accommodate the movement (sometimes called “blocking” or “staging”). All of these movements can be accomplished within a game as well.

“Dolly moves” refer to the use of a physical dolly to move the camera along a set of tracks horizontally. Though a practical camera is limited to a set path, the dolly move on a virtual camera can weave and bob through a crowd, go through walls, or zoom in quickly on a specific object. A “crane move” also refers to a piece of filmmaking equipment. A crane moves a camera up and down, usually without a tilt. Just like the dolly moves, a game is not limited by where or how the camera gets from Point A to Point B while executing a crane move.

The use of a moving camera can add to the excitement of a game by keeping the action in frame. Camera movement can also be used to shift the attention of the gamer from one area to another; a simple way to do this would be the use of panning and/ or tilting. “Panning” refers to the horizontal view of a stationary camera that moves from one spot in the scene to another. “Tilting” refers to doing the same thing vertically. In most games, the use of pan and tilt is usually implemented by incorporating the changing views into the point of view (POV) of the game character; in other words, the scene changes horizontally and vertically when the character looks around the level.

Once you have figured out the camera moves that you want to include within the scene or game level, you must figure out the positioning of the individual items/characters within the shot. Are they moving, too, or will they be static? Which elements in the shot are of higher importance? These are the questions that need to be asked when designing the level or cut-scene. One of the techniques for figuring out the individual elements of camera movement and placement in the film industry involves a process known as “previsualization”. This can be as simple as storyboarding the level, or using a previsualization program.

Keep in mind, too, that cameras can be used within games in ways that filmmakers could never achieve. You can attach cameras to characters, shoot through solid objects, and get POV shots that would be impossible with a practical camera. This is one of the strengths and unique aspects of a video game and should be used as much as possible.

7.6  Staging

The best way to think of staging is to go back to the rule used by Hitchcock. Staging is basically how you arrange the characters and props within the scene. Again, the relationship of the asset in the frame can be based upon importance, or simply be a product of necessity (for example, a rock is placed in the forefront of the scene, because this is a place where the protagonist can take cover). Once you have placed everything within the level, natural areas or zones will emerge.

The various zones provide natural boundaries for the placement of cameras. For instance, if you recognize that the level can be divided into four clear areas, a static camera can be placed in each of these; this way, when the gamer has finished clearing a zone, the POV can shift to the camera placed in the next zone as the gamer enters that area. Good staging should be used in conjunction with blocking, applying the rule of three, and the sound techniques of framing discussed previously.

7.7  Blocking

“Blocking” is the process of mapping out the movement of assets within the frame. For instance, a typical FPS features levels where the protagonist enters the scene and then must cross an area of space to complete the level. This involves the protagonist moving; though the movement of the protagonist can be predicted to a certain extent (paths will be limited by placed obstacles and boundaries), the interactivity of the gamer should be preserved by presenting multiple paths. To this end, the placement of props and obstacles in the scene determine the blocking of the main game character.

In addition to the protagonist, there can also be nonplayer characters (NPCs) moving within a scene. Going back to the FPS example, as the protagonist crosses the main area, there will be opposing forces moving to stop the protagonist and engage the gamer. Because these will be controlled by the game itself, the blocking of NPCs can definitely be planned. A certain degree of variation can be included through the use of artificial intelligence, but again, the movement will be limited to a degree by obstacles and boundaries.

The detailed planning of blocking within a level will help the engineering department and technical director with level design and coding. Typically, the use of blocking techniques is coupled with creative staging for dramatic effect. When used correctly, these rules of good cinematography add a huge cinematic impact to your game.

Interview: Bruce Block, Author of The Visual Story

Bruce Block has served as a producer, director or creative consultant on feature films, television shows, commercials, animated films, computer games, and IMAX movies. His feature film credits include: The Holiday, Something’s Gotta Give, Stuart Little, As Good As It Gets, The Parent Trap and America’s Sweethearts.

Mr. Block is the author of The Visual Story, which is used throughout the world as the standard text for understanding the relationship between story and visual structure. He also teaches visual structure at USC’s School of Cinematic Arts and gives seminars at animation studios, film schools, and interactive game companies.

Newman: In your book The Visual Story, you talk about “controlling pictures”. Give a quick version of what the workflow would be in creating controlled visual art.

Block: First, you have to answer these questions:

1. What’s the story conflict? Not the plot, but the conflict.

2. Who are the characters?

3. What’s the story genre?

4. What’s the story theme or premise?

5. What is the point of view? Point of view means how does the picture maker want the audience to feel about the subject? (Example: The Vietnam War was: dangerous; funny; romantic; a nightmare).

6. Where does the story take place? What natural visual components exist in that location?

Now, based on your point of view, research the story and locations. Create or find photographs, drawings, paintings, written material, or movies that suggest various visual approaches. What visual ideas in other people’s work communicate the point of view that you have in mind? This isn’t stealing; this is research. If you have to make a movie about the Civil War, you research it because you don’t have any firsthand knowledge of the Civil War. If you’re doing a movie about contemporary New York, you will visit the city, take pictures, and perhaps get photo books about the city full of pictures taken by other people. It’s likely that all of the artists who came before you had some great ideas about the visual side of New York that you can use for inspiration. If you don’t want to use research, create concept art yourself. Do this intuitively. What visuals feel right to you? Free associate and create new pictures, sculptures, photos, and so on that communicate your point of view.

Analyze the story or script to see how it’s structured. You must be able to define the conflict and see where it gains story conflict intensity. As I suggest in my book, create graphs that show the story and visual structure. Use the principle of contrast and affinity to give your pictures a visual structure that communicates your point of view and parallels the story conflict.

Newman: How does the visual structure of games differ from that of film?

Block: The visual structure of a motion picture film is predetermined. The film is the same viewing experience for every audience who has no control of any aspect of the presentation. In a video game, the player can rearrange the story structure and the visual components. The game can be a different story and a different visual experience each time it’s played, whereas the film will always be the same.

Newman: How can games take a cue from film structure?

Block: Film’s story and visual structure are critical to good storytelling. Every story (no matter how short or long) has an exposition, a conflict, and a resolution. Something as simple as a joke has all three of these story elements, as does a short story, a graphic novel, a traditional novel, or a movie (of any length). A game must also have these elements. The exposition is the information needed to begin; the game conflict is solving the puzzle, eradicating the enemy, finding something, saving something, escaping from somewhere, and so on; the resolution in a game is success or failure.

In film, when a writer, director, production designer, or cinematographer structures the visuals, certain basic questions must be answered: What is the overall mood? What emotions should the audience feel and when should they feel them? What is the conflict in the story and how can the visual structure make that conflict evident? How can the story’s intensifying conflict be paralleled by the visual structure? How can the story be made visually cinematic? A film crew is divided into departments (writing, acting, casting, directing, camera, lighting, set design, costumes, props, special effects, sound effects, and music) so that every area can be examined, controlled, and used to help tell the story, set the mood, and communicate the point of view.

Newman: What elements do you look for in a visually strong movie?

Block: I look for a controlled use of the basic visual components as they are outlined in my book. These components are space, line, shape, color, tone, movement, and rhythm. These same components are the visual building blocks of any pictorial art form, including games. Specifically, what movies do I consider to be visually strong? Everyone has a different list, but the great visual movies are the ones you can remember long after you’ve seen them. If you must have an actual list, please read my book, because it uses examples from movies with strong visuals only.

Newman: Given the interactivity of games, how do you approach the challenge of creating a strong narrative visually, because the timeline of events in a game can be in flux?

Block: You can’t create a visual narrative. You can use visuals to support a narrative that tells a story but visual components by themselves can’t tell a story. Visuals can create a mood or evoke an emotional response or create visual intensity but I don’t think pure visuals can tell a specific story. An actor walking around without talking doesn’t qualify as a narrative visual. Graphic novels without words tell a story but they are very representative drawings. A wordless graphic novel is a pantomime, so we still rely on facial expression, body language, and anthropomorphic symbols to tell stories. An interactive game, no matter how nonlinear, still has a beginning and an end. As the player plays the game, events can occur based on game play, not on a scripted plan. It is still critical to any structure that there be a build towards the end or at least structural variety so that the game play remains visually interesting. A flexible timeline or a game player’s random access to events cannot be an excuse to ignore visual structure.

Classically, the most intense part of a conflict is the climax where the conflict ends. In some video games, the final battle is the most intense, or the final puzzle is the most complicated, or the last race is the most difficult. No one wants to play a game where the most difficult stuff is first and then everything gets progressively easier. Each level of a video game is more difficult or exciting so that the structure builds in conflict intensity. Programmers must plan for the interactivity and nonlinear play that occurs in games. If the sequence of scenes (or events) is designed as A-B-C-D-E-F, the game designer can’t ignore the fact that the player might encounter the scenes in a different order like: F-A-E-C-B-D. No matter how the player encounters the scenes, the designer must create a flexible structural plan. If the scenes are encountered out of order (and F is the most intense scene), predetermining and locking the visual design of the scenes won’t work. Because the player can change the scene (or event) order, that player can easily encounter the scenes in any structural order. The scenes have to be programmed with flexibility so that an intensifying structural experience can be had no matter how the scenes are encountered. Even if a player returns to a scene that has been “completed”, it should look different based on where else the player has gone and where the player is now positioned in the overall visual structure.

In global multiplayer games, visual variety in the environments is critical to keeping the game interesting. Different “universes” will be easier or more difficult to encounter based not only on who is there but also by how it’s visually designed. Should an explosion occurring on Level 1 look like an explosion that occurs on Level 20? Probably not. How the explosions can differ depends on the visual structure that is created based on the story and point of view.

Newman: With cinematography now becoming a serious subject in the game industry, describe the impact that choice of framing and lighting can have on a scene.

Block: Cinematography and lighting have a huge impact on visual structure. Most interactive games leave framing choices to the player. The game designer does have certain controls like lens choice and limits that can be placed on the camera’s mobility, but camera angle is usually not controlled by the designer. Lighting can be controlled by the designer (time of day, existing light sources, and so on) or the player (carrying flashlights, torches, finding light switches, and so on). A good cinematographer has a huge range of styles that can be used: a film-noir style, sitcom style, horror-film style, overexposure, contrasting lighting, hard lighting, soft lighting, lighting direction, color of light, intensity of light, one light source or multiple light sources, and so on.

A game designer who is concerned with camera angle and lighting should pick ten classic films and run them with the sound off and study the lighting. The designer should also take some classes in photographic lighting styles. Camera and lighting are very sophisticated cinematic techniques that elevate visual structure. In film and television, experienced cinematographers create extraordinary moods and visual styles with lighting, but it takes a lot of experimenting and practice. A designer that thinks lighting is only there to see the action is missing a key element in visual structure.

Newman: Most games are being developed today with a well-defined story and characters, but the theme is usually murky at best. What’s the process concerning the use of space and color to help define a theme?

Block: The theme of most games seems to be: “Getting killed is a bad thing” or “Do this because it’s fun or exciting”. Generally speaking, deep space is more dramatic than flat space, because deep space creates visual contrasts. So if deep is dramatic, it can be used to define themes that are dramatic or intense. Flat space can also be dramatic, but it needs other visual component contrasts to gain the needed visual intensity. Remember that space is going to be a general backdrop behind the story to help define mood and support the intensity of the conflict. Space doesn’t have the ability to get as specific as color.

Color has three “control knobs”: hue, brightness, and saturation, which allows a wide range of color choices. Color can communicate to an audience more specifically than space. Like any visual component, meanings can be assigned to a color. Color already has a lot of stereotypic meanings that can communicate a theme: for example, bright saturated colors are happy; saturated red can communicate danger or passion; desaturated blue can show depression. Again, these are hokey stereotypes, as any color can be made to communicate almost anything.

Newman: When one reads a story or watches a movie, you can always tell when the writer has a great rhythm—the story just flows and the characters are all well-defined. What sort of “checklist” could a game designer use to determine whether good rhythm is present in the game’s visual style?

Block: The most important factor in rhythm of a game, I think, is intuitive play. A game can be difficult to win, but its rhythm must feel intuitive. I don’t think a player will get involved in a game that doesn’t have an intuitive rhythmic feel to the overall play. Visual rhythm is based on controlling a picture’s linear motif. You can find the linear motif of a picture by reducing it to lines. If you reduced your entire game to a series of pictures of lines (curved, straight, horizontal, vertical, and diagonal), how would they stack up in terms of structure?

The easiest way to see the linear motif is by removing the color and then increasing the contrast of the picture until there’s nothing left but black and white. That remaining harsh, contrasty image is the linear motif, which reveals the visual rhythm. When we look at any picture, we might describe the visual rhythm as calm, jumpy, agitated, soothing, mellow, chaotic, slow, or fast. All of those descriptive words are how we feel emotionally about the linear motif. As a game is a series of pictures, how does this series of linear motifs (the pictures) reveal contrast or affinity? In my book, there’s an entire chapter devoted to finding linear motif and understanding how to control it structurally. Visual rhythm is also created by editing or cutting from one shot to another. Most games tend to be a single, endless shot, so editing isn’t usually a factor in a computer game.