If the frames are the atoms of which documentaries, like all films or video productions, are ultimately composed, then the shots are the molecules, irreducible component parts which nonetheless retain some of the characteristic of the whole.
Just as a text is usually made up of many sentences, though in theory it could all be one giant sentence—Friedrich Dürrenmat once published a novel that was all one sentence—so is the documentary composed of many shots, though, in theory, one giant shot could make up the entire work.
In practice, as we have noted, the length of a shot generally reflects the length of the whole film. Average shot length in a thirty-second commercial will be shorter than in a one-hour film. The pacing of a production depends largely on the length of the shots which make it up. The speed of the flow of time is the glue that holds the film together.
The shot is the result of adding the element of time to the frame; what is captured between the moment of switching on the camera and switching it off again. Time registers its presence by evidence of change. Each frame of a shot is changed from the one before. That change is perceived as movement. It is the importance of time that makes change and movement crucial to the film.
Without change or movement there is only the frame—an unliving image. Repeating the same frame without change results in a freeze. Film time stands still. There is only one place for the still frame in a film: when there is a need to freeze the action and bring time to a stop. Change and movement, bring life to the film. The nature of that movement determines the nature of the shot. Understanding how movement works in the shot is essential for understanding how the film works.
The self-similar nature of film leads to the tendency for each section to have a structure that mirrors the whole, each sequence to mirror the section, each shot to mirror the sequence. The structure in question is the narrative—the story. The overall film has a story, each section is built around a story, each sequence has a story and each shot tells a story.
The story which a shot tells will usually be a simple one: a woman gets up off her chair and walks to the door; a man lifts a spoonful of soup to his mouth and drinks, a child wakes up in its cot and yawns, a soldier scans the horizon for enemies. In other words, the story of a shot will usually be that of an action. Even within this starkest simplicity, the structure of the shot’s story will demand what all stories must have: a beginning, a middle and an end—even in a single shot.
What constitutes the start and the end of a shot’s story? In general, the commencement and the completion of the action. The shot starts before the action begins and ends when the action is over. Shooting the shot demands that the moments of pause, of stillness, before and after the action are included. During the editing process, the pauses before and after the action will be adjusted to the pacing of the sequence into which the shot is being inserted.
It may seem that a shot which is above all static—a mother gazing at her sleeping child, for instance—has no beginning or end markers. And when shooting the image, this may indeed seem to be the case. But when the shots are being joined together into sequences, it is a rare editor who will not search the shot keenly for the slightest of movements—a twitch of an eyelid, a tremble of the lip—to mark the start and the finish of the shot. Reaction shots of the interviewer during an interview are often teasingly called ‘noddies’. The movements of the interviewer’s head are themselves narrative events—’the interviewer nodded’ would be the description in a text.
The story may not, in the edited film, be told by the single shot alone. Shots which tell identical stories—different views of the same action—may be joined together. The sequence of shots may take the start of the action from one shot, the middle from another, the end from yet a third. At shooting time, it is not possible to predict exactly which portions of which shots may be selected. Hence shooting an action from different viewpoints requires the action to be repeated in its entirety each time. Even so, there may be difficulties in matching the action as between one shot and another. One of the principal complications is the action’s speed.
The shot’s story unrolls over the time of the shot. The rate at which it does so is an important factor in its working. A shot will be placed among others in a sequence. The speed of the movements within it will have to relate to the speed of the movements in the shots on either side.
It may seem that this poses no particular problem, as similar movements will happen at similar speeds, but what is crucial is not so much the speed of the movement itself but its apparent speed on the screen. That apparent speed depends on the framing of the shot. The man drinking soup will move the spoon to his mouth at a speed that is natural to the action. In a mid-shot, his hand, taking care not to spill the soup, may traverse half the height of the screen in perhaps two seconds. In a close-up of his hand, the same speed will lead his hand to cross the frame in a small fraction of that time—the apparent speed of the action will be much greater. In a sequence composed of a mid-shot, followed by a close-up, followed by another mid-shot, the action will appear to speed up greatly while on the close-up and slow down again as the sequence returns to the mid-shot.
Thus in capturing movements of the shot’s subject, consideration must be given to the way in which the shot will subsequently be joined to others in a sequence, and the apparent speed of movement adjusted accordingly, so as to seem equal on the screen. In any case, a close-up of the soup-drinker’s hand traversing the screen in a fraction of a second is not a very useful or revealing shot. Movement in close-up will usually only look right when it is actually performed at a rather slower speed than is natural—one example of ‘cheating’ the action, manipulating reality to make it seem more real.
Movement on the screen in the horizontal and vertical planes can vary between movements across or up and down it (the ‘x’ and the ‘y’ axes), and movements towards and away from the subject (the ‘z’ axis). The difference can strongly affect the impression given to the viewer as well as the viewer’s understanding of the action shown.
Consistency of direction of movement is an important factor in helping the audience to comprehend the events being portrayed on the screen. Much will be determined when connecting the shots into a sequence, but since the sequence is made up of individual shots, the direction of movement in each shot must be the concern of the film-maker at the time of shooting.
It is obvious that the portrayal of a single action should maintain consistency of screen direction. In shots showing a woman getting up off her chair and walking to the door, the establishment of the geography of the room demands that if the woman is seated to screen-left of the door, her rise and walk must move her from left to right. If the action is covered by a number of shots, her screen direction must be the same in each. As mentioned earlier, this means that the camera position must be consistently on one side of her—to be exact, on one side of a line drawn between the woman and the door. This is known in director’s jargon as not ‘crossing the line’.
But what if the same location appears in a number of shots not connected together in a sequence? Standard film convention, as established in the cinema, suggests that the same screen direction is portrayed each time the scene is returned to. This is, of course, not a fixed rule, but many film-makers adhere to it as a way of helping the audience’s mental understanding of the location.
On an even larger scale than the returned-to scene, a director may choose to give to an action or movement which plays an important conceptual role in a film, a consistency of screen direction which is adhered to over the entire course of the production. In a scene of armed conflict, for example, one side may always be shown as moving from left to right, while their opponents are always shown as moving from right to left. This helps not only with the mental image the audience maintains of the location of the conflict, but also helps with instant identification of which party to the conflict is being shown.
Other extraneous and purely cultural assumptions may affect decisions about screen direction. A shot of a transatlantic flight in an aircraft may make use of the audience’s subconscious recollection of the map of the world looked at in the usual way: north at the top and south at the foot of the page. The airliner may thus be consistently shown travelling from right of screen to left if the journey is from Europe to America, and the other way round if the flight is from America to Europe.
All modern cultures share the same understanding of the map of the world. Thus screen direction to represent travel is likely to be universally appreciated. But another consideration that relates to quite different cultural assumptions affects movement on screen. The direction in which we read—left to right in European, Indian and south-east Asian languages, right to left in Semitic languages—makes us feel rather differently about actions which move from side to side on the screen. For readers of the Latin alphabet, the left to right direction is associated with the unrolling, the revealing, of information. For readers of Arabic or Hebrew, right to left is the more natural.
Movement on the screen has yet another important function which does not relate so much to its content but more to its practical effect on the screen image. Movement across or straight up and down the screen maintains the flatness of the screen’s visual world. The movement of the subject brings it neither nearer to, nor further away from the camera. Movement directly towards or away from the camera has little immediate impact because it results in not very much change on the screen—the subject just slowly grows as it approaches or shrinks as it retreats. It may be hard to detect that an object like an automobile driving towards the camera is moving at all. Movement at an angle between the two—towards or away from the lens at the same time as to the side—combines the best of both planes and adds to the illusion of depth of the shot.
In a wide shot, where the illusion of depth is a primary consideration, movement at an angle is an important way of enhancing the illusion of presence. Such actions as people walking, birds flying, motor vehicles driving, will often make their best impact when travelling at an angle towards or away from the screen, thus making explicit the depth contained within the image. In a closer shot, which has by its nature little illusion of depth, movement towards and away from the camera may also be necessary to contain the movement within the frame. Side to side movements will necessarily be limited by the need to maintain them in vision. But even here, the action of a hand, for instance, when moving forward or backward in line with the lens, will help to sink the owner of the hand back into the imaginary space behind the screen, where a simple sideways motion would only emphasize the flatness of the image plane. Note, however, that at close-up size, it may be hard for the photographer to keep the hand in focus if its forward or backward movement is too great.
The movement in a shot can be provided by the subject of the shot itself moving within a static frame, or the frame itself (or rather, the camera lens capturing the frames) can move. Often a shot will contain elements of both movement of the subject and movement of the camera.
Any movement on the screen must have a rationale, must be the effect of a cause. Where the subject of the shot moves, the purpose, the motivation of the movement is explicit in the movement itself. In other words, the action of a person or the movement of an object on the screen is part of the shot’s storyline. The woman getting up from her chair and walking to the door is understood as performing a voluntary act. So is the man drinking soup. The rationale for their movement belongs to the world behind the screen and can be accepted as such.
But since the camera represents the viewer’s eye-view of the scene, the effect of moving the camera is to represent a movement of the viewer within the scene. As the viewer is in reality merely watching a television screen, a way has to be found to give the viewer the impression that he or she is moving of his or her own volition. If it is the film-maker’s purpose to draw attention to the fact that the viewer is watching a film rather than reality, camera movement can be used to contribute to that Brechtian ‘alienation effect’. Otherwise the film-maker must find a way of persuading the viewer that the camera is moving in direct response to the viewer’s own wishes.
To motivate a camera movement is to predict the viewer’s own automatic visual response to the images on the screen. That means that the camera must do what the viewer’s eye would automatically do if the viewer were really present at the scene. Human eyes, and not just those of men on the make, are all roving eyes. Anyone looking at a scene will constantly be scanning different aspects of it, a process which, as mentioned before, builds up the eye’s effective field of view. These eye-movements are largely unconscious and automatic. But there are other eye-movements—and sometimes body movements too—which, while instinctive, are consciously performed and can be, if necessary, suppressed.
If a person to whom we are speaking suddenly, in mid-conversation, glances to the left we automatically respond by throwing a look in that direction too. If we are watching and listening to a conversation between two people standing in a doorway and one of the pair moves back so as to be hidden—masked, in film parlance—by the door frame, we again respond almost without thinking: we shift our position to bring the hidden speaker back into view again. If, while engaged in a conversation, we hear a strange noise to our side, we turn to see what is making the noise. In a television shot of these scenes, if the camera makes the moves that viewers would automatically make for themselves were they really present, in other words if the camera movement is properly motivated, the viewers may not, probably will not, notice that the camera has moved.
Motivation for camera movement is sought by the film-maker who does not wish to bring the artifices of camera work to the audience’s attention. If the motivation is well done, the audience will not notice the camera movement, but will accept it, just as they accept a well-motivated change of shot. In many cases, the motivation will indeed be for a change of shot—after all, in the real world the eye does not pan across a scene, but skips from one point of interest to another. But audiences have become used to the moving camera as a substitute for the actions of the eye and even if conscious of a well motivated camera move, at least feel their need to see another view satisfied.
One of the commonest motivated camera movements is the track-in. It satisfies the viewer’s instinctive desire to see something from closer up. A person on the screen speaking in mid-shot, if using a one-to-one conversational tone, will commonly stimulate in the viewer the desire to move in closer, the same movement that the viewer would make if engaged in conversation in the real world. The track-in satisfies the viewer’s desire to make that move.
Camera movements are of two kinds: tripod shots—movements of the lens alone, where the camera is pivoted on a fixed point, and travelling shots—bodily movement of the camera itself. Though the aim can be to change the framing of the shot in a somewhat similar way, the effect of each kind of movement is quite different.
Movement towards or away from the subject of the shot can be done by changing the magnifying properties of the lens—zooming—or by moving the camera physically closer or further away from the subject—tracking. The direction usually referred to as ‘in’ is towards the subject. Out is away from the subject.
The zoom lens is, historically speaking, a relatively recent innovation, and still suffers some optical disadvantages compared with fixed focal length lenses—more layers of glass to distort, diffuse and darken the image, the difficulty of keeping mechanical moving parts in pin-point registration. Nevertheless in television filming, zoom lenses are virtually the standard.
Effectively, a zoom lens varies the magnification of the image. Zooming in means increasing the magnification, zooming out implies decreasing it. Naturally, those portions of the image which border the edge of a less enlarged view are cropped—pushed out of the frame—when zooming in. Thus zooming in takes the central portion of the image and fills the frame with it. Conversely, zooming out—decreasing the magnification—brings new information into the frame around its sides. Nothing other than the framing is changed in the image itself. The zoom mimics the actions of the brain, when it selects a portion of our visual field to concentrate on. Though our visual field is potentially large—as was earlier suggested, extending horizontally to almost 180°—we are rarely aware of it. Most of the time, our brain concentrates on only a portion, larger or smaller, of that field. Everything outside that area of concentration is ignored—we are mostly unaware of it. To manipulate the zoom lens is to mimic the viewer’s changing attention. When, in the real world, the viewer would automatically concentrate on a smaller part of the image, zooming in will satisfy that desire.
Only very rarely are there circumstances in which a person’s area of concentration enlarges rather than shrinks. The use of the lens to zoom out is consequently very much more difficult to motivate. Only in circumstances in which the viewer yearns to see what is going on just outside the edges of the frame will a zoom-out not be noticed.
By contrast to the zoom, the track-in or -out does not merely change the magnification of the whole image at the same time, but actually changes the relationship between its elements. In shooting for the cinema, to ensure smoothness of movement, the tracking camera is usually mounted on a moveable dolly with rubber wheels which run along metal tracks fixed firmly in position on the ground. Sometimes shooting for television will use tracks and a dolly, but much more rarely.
Tracking in with the camera means moving the camera bodily towards the subject of the shot. As the camera moves closer, objects in the shot will be enlarged. But because they are all at different distances from the camera, the proportional enlargement of each will be different. In a shot containing a foreground object at a distance of ten feet (3 metres) and a range of mountains in the distance, a forward movement of the camera by five feet (1.5 metres) will halve the distance of the foreground object and enlarge it roughly to twice its previous screen size. The distant mountains, however, will hardly change. If they are two miles away (3.2 kilometres) the camera move will enlarge their image by roughly one two-thousandth.
These differences in change of apparent size are exactly what our vision system is used to when moving in the real world, and which it automatically computes when estimating distance so as to build up our normal three-dimensional view. Bodily movement of the camera therefore adds to the illusion of depth in the television image as well as potentially satisfying the viewer’s desire to come closer or move further away. This is different from zooming movements of the lens, which simulate changes only in the viewer’s area of attention.
Horizontal movement of the shot can also be accomplished either by movement of the lens only, or by bodily movement of the camera.
Side to side movement of the lens, with the camera pivoted on a tripod, gives us the pan—the panoramic shot. Physically moving the whole camera from side to side, often done on tracks like the track-in and -out, is known as crabbing. The shot can be described as a crab left or a crab right.
The difference between the pan and the crab is much like the difference between the zoom and the track. Panning changes neither the perspective of the image nor the relationship between its elements, while crabbing moves each plane of the picture by a different amount, thus seeming to shear the planes against each other and re-enforcing, as does the track, the illusion of a three-dimensional space behind the screen.
Real eyes don’t zoom. But the human head can surely pan. The panning shot claims to mimic a natural movement of the eyes and head. It should therefore be easier to motivate and to make seem instinctive than the zoom. But in truth the eyes do not pan in the real world. Rather they jump from one point of concentration to the next. Our attention does not smoothly move across a scene, at a regular speed, taking in everything on the way.
The only circumstance in which our eyes do this is when they are following a moving subject. If something we are watching moves at a steady speed across our field of view, our eyes will naturally follow it in a panning movement. This phenomenon is often used as a motivation for a pan—the camera follows a vehicle, a human figure, even an animal, as it crosses a wide vista. When successful, it leads the viewer to accept the panning shot as a natural visual response to a visual situation. Cine-photographers desperate to find motivation for a shot will sometimes follow anything that moves, even a seagull flying across the sky; though viewers may later puzzle over the significance of the bird.
If panning is not easy to make seem natural, crabbing the camera is even less like any action we perform with our eyes in the real world. There are a few circumstances in which we walk sideways: when trying to talk to someone who will not, or cannot, stop for us, for example. But walking sideways does not come easily to most of us; particularly when we cannot see where we are going and may easily therefore crash into an unexpected obstruction. A sideways-moving shot always carries with it some of that unease, even when movement of the shot is needed to follow a moving subject. For supporting the illusion of depth behind the screen the crabbing shot is unrivalled. But it is hard to make the audience unaware of its use as a device. Unless disturbing the viewer is the aim, crabbing shots are mostly found when representing the view from moving vehicles.
The third dimension of possible frame movement is up and down. When pivoted on a tripod, the action is known as tilting. Tilt up and tilt down are the two directions. Bodily movement of the camera requires a crane. The shot is known as a crane shot.
Tilting the head and eyes is a natural human movement. The nature of the kind of things we pass our gaze vertically over: masts, buildings, mountains, makes it a rather smoother action than we perform when we look around us on the horizontal plane. Consequently, tilting the camera lens is not so hard to make seem a natural movement.
Tilting the camera results in ending on a framing which is not horizontal—it is high-angle or low-angle. Or the camera may begin with the angled view and end up positioned horizontally. Either way, the camera movement is a link between a normal view of the subject and a rather different view. This can give the shot a strong sense of development and narrative. But the film-maker tilting the camera has to accept that the resulting angled perspective may seem distorted. When looking up the side of a building we are often conscious of the distorted view we receive, with the building’s top receding into the distance in a way which rational geometry tells us is correct, but subjective assumptions can make seem unnatural—even in the real world. We can usually be persuaded to accept the strange perspective caused by tilting the camera in a similar spirit. As with other camera movements, motivation is all.
The crane shot is perhaps the most unnatural of all developing shots. There are very few circumstances in normal life when a person can expect to rise vertically into the air—or descend vertically either. Yet the crane shot is attractive to documentarists because of that very unusualness. A usable lift or elevator at a location, from which to shoot a (free) crane shot, is a delightful gift to a film director. Such a shot gives viewers a sight of the subject that may alter their perception and understanding of a situation. But, as previously suggested, it is over-used as an easy cliche when employed as a final ending shot for films—the ‘eye of God’ shot—offering little more than the spurious effect of seeming to put the small details of the preceding matter into an overall global perspective.
The premise of the above descriptions are based on the illusion of presence. The suggestion has been that if performed as a result of proper motivation, the movements of the camera will satisfy the viewer’s instinctive response to a scene and remain either entirely unnoticed, or will at least be accepted as an appropriate movement in the circumstances.
But not all film-making seeks to make the viewers unaware that they are watching a film. Some directors find it dishonest to attempt to lull the viewers into a feeling that they are watching something real, and prefer to underline the artificial and one-sided nature of the medium. For such film-makers, the question of motivation for the purposes of sustaining the illusion of presence is not relevant. Of course even if trying to remind the viewers that they are watching a film, a film-maker does not necessarily want the flow of images to be constantly disturbing or surprising. A documentary in which every single shot is unexpected would be almost unwatchable.
And sometimes, even a film-maker striving to create a smooth and acceptable work will try to shock the audience out of its suspension of disbelief. Camera movements made at an unusual speed are well suited to such effects.
One of the commonest is the ‘crash zoom’: a zoom-in performed at so fast a speed that it draws attention to itself and makes the viewer immediately aware of the artificiality of the camera’s movement. A crash zoom picking out a detail from a wider scene shouts loudly at the viewer: ‘Just look at this!’
A ‘whip pan’, a panning shot performed at such a speed that it blurs the images between the pause at the beginning and the pause at the end, is less frequently used today, though in the past it has been a commonly used technique for moving from one scene to another. It is a purely artificial device, with no real analogy to the movements of a real human eye. Audiences understand it as a cinematic conceit. It has little place in documentary making today.
To keep the subject in vision when craning up or down, the camera has to be tilted at the same time. Most bodily camera moves are accompanied by simultaneous pivoting movements of the lens. To maintain an effective composition of the frame while moving the camera may need constant adjustment of the lens, both horizontally and vertically. It may also need some adjustment of the zoom setting. These lens moves are not usually noticeable within the overall camera movement.
There are times when two contrasting movements can produce bizarre effects: tracking out and zooming in on a figure at the same time, for instance, magnifies the background without changing the size of the figure itself. As the foreground figure gets smaller, the camera zooms in to compensate, thus enlarging everything else in the image. This is the equivalent of the real life impression one occasionally gets of the growing size of a mountain range when one is motoring away from it, the consequence of the greater apparent change in size of foreground than of background. Even the poet Wordsworth noticed this, while rowing across a lake, when:
‘… the huge Cliff
Rose up between me and the stars, and still,
With measur’d motion like a living thing, strode after me.’
Thus far we have considered only images produced when shooting from a tripod or other camera mount. The result is ‘classical’, formal, stable, and—at its best—persuasive and convincing. The audience’s suspension of disbelief is readily achieved. But there is another way of gathering the material in a much more informal manner, by operating the camera from the shoulder, without the constraints of a fixed mount. In a hand-held shot, all of the moves described previously may be combined, both at the same time and also by flowing seamlessly from one to another. A typical handheld shot is one long movement from beginning to end. In fact purposeful movement is necessary in the hand-held shot as, unless the camera operator has an unusually rock-steady shoulder, any attempt to shoot a static frame with a hand-held camera is likely to result in a fidgety, unstable and distracting framing.
The result of hand-held filming looks quite different and has a rather different impact from that of formal shooting. The camera can move around, can duck and weave, can push its way between members of a crowd, can look over people’s shoulders. Such constant movement cannot be motivated in the strict sense of the word. It relies for its success on sweeping the viewer up in the rush of impressions. A hand-held camera shot nearly always presents a point of view: that of the person behind the camera. A successful hand-held shot will lead the audience to identify with that person. It may be appropriate for representing the point of view of a character appearing on the screen. It may also be identified with a person never seen, but whose voice carries the narration.
For this reason, the hand-held camera is often used as standard in documentaries which offer personal reportage. In news filming, where hand-held operation may be the only possible option, and where audiences associate the informality with urgency, hand-held shots are interpreted as the reporter’s eye-view. But a number of well-known film-makers also make use of this kind of shooting by preference, even though they do not themselves appear on the screen. (Nonetheless they often allow their subjects to address them behind the camera, and often including their own voices in the final edited work.) The result is to make the entire film very much the film-maker’s personal vision. Intimacy and emotion are emphasized in contrast to objectivity. When used with skill and artistry, hand-held camera work can totally involve the viewer with the film-maker’s viewpoint.
In the opinion of many film-makers, the sound accompanying the shot is among its most important and powerful elements. It contributes to the content of the shot as well as to its flavour and atmosphere. Just as the vision is part of an imaginary world behind the television screen, the shot’s sound must conjure up a believable sound world. The sound world’s significance lies in the fact that we respond to a shot’s sound rather differently from the way we respond to the screen image.
Firstly, our consciousness of sound is not precisely located in space. The picture shown by a television screen will always, by definition, be fixed in the position of the screen—a small luminous area some distance away in the room. On the other hand sound is, as previously suggested, built up inside our heads. Though sounds may give the impression of having originated somewhere in the three-dimensional world, we become, so to speak, immersed in them, bathed in them; they become part of our environment.
Secondly, sounds seem to have a more direct access to our thoughts and associations. Just as smell can conjure up unexpectedly powerful emotions and memories, so our ears seem to be plugged in to a deeper stratum of our brain than our eyes. To generate true illusion of presence at a scene, the sound is among the shot’s most crucial components.
Capturing the full richness and totality of sound at the same time as the visual shot is usually impossible. Thus film sound is almost always an artefact. Although, when well done, it is a convincing representation of what a person would have heard were he or she really present at the scene, in nearly all documentaries the sound of a shot is constructed after the event. To be sure, in most cases, the foundation for the sound track of the shot will be the sound recorded at the scene. But much will be added to it at later stages of production to make it perform its full function.
Generally speaking, the viewers will expect the perspective of the sound to match the camera position. This is a demand of the illusion of presence. If the camera, and therefore the viewer, is close to something which is making sound, a viewer will expect to hear that sound from close to. If the camera, and the viewer, is some way off, the viewer will expect to hear the sound more distantly. A long shot will normally be accompanied by a very wide sound perspective, gathering in all the sounds from the entire scene—though some directors do delight in coupling wide vision with very close sound, as in the closely-miked hiss of skis played over a long shot of a solitary skier on a mountainside. A close shot will usually suggest much more selective sound, diminishing or even excluding all noises other than those coming from the subject of the close up. For in the real world we automatically exclude irrelevant sounds from our consciousness.
Thus at the time of shooting, film-makers will concern themselves with questions about what sounds are to be included and what excluded, as well as from what perspective to record the sound. The audience will expect to hear the sounds produced by objects which appear on the screen. They will expect to hear those sounds in a proportional relationship with each other. The further away the screen object, the further away the audience will expect it to sound. In a scene which includes a person speaking in foreground as well as a car driving in the background, the audience will expect the voice to be louder than the car.
The human ear, or rather the part of the brain that deals with hearing, applies extremely elaborate filtering processes to the raw sound before it arrives in our consciousness. Psychology textbooks often mention the ‘cocktail party effect’. This is the ability of the human hearing system to select from among a chaotic babble of voices, just the one single voice to which the person is listening. Background noises are suppressed, extraneous sounds are ignored. The redundancy of information available to a person really present at a scene allows that person to fill in gaps in comprehension of speech from other senses; watching a person’s lips move helps us actually to hear what that person is saying. The phenomenon is similar to that which happens when watching subtitles translating a language with which we are familiar but perhaps not entirely fluent. The subtitles actually help us to hear the foreign speech, not just to understand it. We are not necessarily aware of that transfer from sight to sound, it happens subconsciously.
The cocktail party effect, like other brain processes performed on sounds that we hear, depends on the fact that the vibrations which constitute the sounds come at us from all directions. Their different frequencies, phases, echoes, distortions all help the brain to distinguish between their different sources. But sound recorded as part of a shot at the shooting location, is compressed into a single—double if in stereo—channel on the recording device. All the sounds are inextricably mixed together. When it comes out of the loudspeaker at the other end of the chain, the brain can no longer separate its different sources. Thus while a conversation taking place in real life in front of a very high background of ambient noise can still be heard and understood by a person present at the scene, once recorded and replayed, the same conversation may be completely inaudible, or at least, unintelligible.
In consequence, what the film-maker will try to capture at the location will be a strongly selective version of the sound, concentrating on only those elements which need to be tightly synchronized to the image.
Modern microphone technology allows for great selectivity. Where there is a high level of surrounding noise, microphones with a very narrow angle of acceptance are used. But even if a directional microphone is so positioned as to reject as much as possible of the surrounding noises, the reflections of the ambient sound will inevitably find their way into the microphone’s path. If care is taken, they will usually be sufficiently attenuated so as not to interfere with the foreground sound.
In later stages of the production, the sound scene will be built up by adding to the foreground sound both other sounds recorded at the location and further atmospheric sounds culled from elsewhere.
From the viewer’s perspective, the sound of a shot can originate from an on-screen object or subject, from an object or subject which is off-screen but understood to be present at the scene, or, like accompanying music, it can be related to something in the film which is not at all part of the actual on-screen world of the shot. It is not always possible to distinguish between these three categories, which may subtly merge into each other. In addition, the sound of a shot can influence the viewer to reinterpret the visuals in a way which justifies the sound.
Thus, for example, a shot taken in a patch of English woodland might have bird song added to it at a later stage. The audience will accept the bird song as what they would expect to hear at that location. But were the sound of tropical insects substituted for the British birds, the audience would be likely to interpret the visuals in a different way—as a tropical forest perhaps, so as to justify to themselves the presence of insect noises. Unless the sound of the shot is quite clearly established as not coming from the visual scene, viewers will reinterpret what they see to rationalize the sounds they hear.
Few documentary makers would need or wish to change the audience’s perception of the location of a shot from a British wood to a tropical forest. But one striking use of non-synchronous film sound is a television commonplace. Natural history documentaries are commonly shot without any sound at all. The sounds that the audience hears are all added later. Where the visual world of the film is at human scale, the added sounds are naturalistic, and viewers are mostly unaware that they were not recorded at the time of filming. But when such documentaries deal with the world of the small scale and the microscopic, the filmmakers have to invent what they imagine a person would hear if scaled down to the same size as the subject of the documentary. Such productions also often make use of speeded up shots—plants growing visibly, for instance—and slowed down shots—humming birds hovering or cheetahs running are often seen examples. In such cases the documentarist must make an even more imaginative leap to suggest what the appropriate sounds might be. The resulting choices may well seem bizarre—dividing bacteria do not in fact rustle. But some sound is always necessary to give an impression of reality to what is actually an entirely artificial series of images.