For over 100 years we’ve used the word “movies ” to refer to your rapid slideshows of still-frame pictures in which nothing actually moves, so it is no secret that the entire entertainment industrial complex is founded on an ­illusion—the illusion of motion.

But for 100 years, meme-makers involved in the film industry have referred to the wrong psychological tendency to explain why it works. Don’t feel bad, so have many psychologists.

Sorting out all of this confusion will help those of you who make movies or video games understand the barriers you’ll encounter in representing exciting action scenes with bleeding-edge technology. And not unlike the takeaway from the last chapter, it will also help you prioritize whether to get into a technological arms race with this bottleneck, or to focus instead on storytelling.

Where did the confusion about motion begin? Before film went digital, the mechanics inside projectors pulled down each of the pictures on a film strip into a beam of light, held it there for a bit, then pulled down the next picture. However, pulling down frames this way was blurry, the same way waving your hand in front of your face is blurry. So this blur needed to be obscured with what was called a “shutter ,” a piece of metal held in front of the light while the frame was being positioned in a momentary blur (Figure 6-1). The mystery at the time was: why don’t people perceive the flickering caused by the shutters? As long as the moments of darkness were shorter than 16 milliseconds, they were invisible. Why?

Figure 6-1. 1970s era projector with a three-bladed shutter. ⁱ

To answer, we need to introduce memory. But we’ll forewarn you this still doesn’t explain how we perceive motion.

Broadly defined, memory is our neural ability to preserve the sensation of something past the point that it is gone. It is quite a talent for an organic creature. After any stimulus, be it your meme or anything else, has stopped enervating our sensory systems, we retain a version of it, a representation we can pull up and re-experience at will, sometimes for another second, sometimes for a century. Neurological media is the original media; it predates pixels, cellulose, and paper.

Being remembered is also the next bottleneck for your meme to survive; if it does, it means we valued it highly enough to copy it for later. This is simply not done for the vast majority of things that stimulate our senses. Unremembered memes that fail to survive this third bottleneck cease to exist when they move out of sensory range.

Iconic memory(its stage name; it also goes by “sensory memory” and “persistence of vision”) is the first and briefest form of preservation. It lasts at most a quarter of a second, and that is under ideal conditions where it is not displaced by something else. Our fovea are once again at play. Most sensory cells reset very quickly after they are no longer stimulated (including the rod cells in our peripheral retina). But the cone cells in our fovea and the ganglion cells behind them continue to fire for up to 250 milliseconds longer after ­photons stop hitting them, letting us experience something that has past. ⁱⁱ

When someone waves a sparkler around in the dark, the trail we see is our rapidly-fading iconic memory of it (Figure 6-2). In reality, there is no trail. Iconic memory is also the reason we barely perceive our own blinks.

Figure 6-2. Iconic memory is responsible for sparkler trails. ⁱⁱⁱ

And yes, it is iconic memory that preserves each frame of a movie, which fills in and replaces the darkness of the “shutter ” as they block the light to pull down the next frame.

This explanation is as neat today as it was 100 years ago. (And it is still relevant, since computer monitors also flicker imperceptibly on a certain “refresh rate.”) But the explanation was so neat it was over-extended. Iconic memory has been cited as the reason we see motion by many writers, but it only explains why we don’t see the flickering. Think of it this way: iconic memory is performed by the nerves in the eye and they’re smart enough to fill in a bit of darkness. But to perceive something as complex as motion, we need a brain and more advanced cognition. This error wasn’t just made by meme-makers, psychologists made it too. That may be why our collective understanding of motion perception is still somewhat infantile.

Here’s what we know: There are pathways, and possibly individual neurons, in the occipital lobe at the back of our brain that are built to fire only when they detect motion in a certain direction or vector. Some neurons report motion moving north to south, others report motion moving northwest to southeast, others more complicated patterns. So basic motion perception appears to be hard-wired.

But a Gestalt principle from Chapter 4 also plays a critical role in perceiving the motion of complex things like living beings. Gestalt principle (d) that we introduced in Figure 4-4 states that we perceive things to be related that are moving the same direction. When the brain perceives two dots moving the same direction, it reports to us that they belong to a single rigid object (a principle Wertheimer called common fate) rather than separate objects. This leap allows us to perceive moving structures (like dinosaurs, cars, and spaceships), not just moving pixels.

Gunnar Johansson demonstrated this in 1973. ^iv He essentially made a very stripped-down movie consisting of only 10 dots. To explain why it worked, we put the dots into Figure 6-3. We’ve labeled the dots with letters that pair them up in order to make our point, but Johansson did not; they were just dots. The dots that we’ve labeled with the same letters were shown as moving the same direction in an animation that Johansson showed to students. Despite the graphical minimalism, every one of the students saw a person walking. Because of the perception of common fate, the pair of A dots were seen to belong to a unified head and torso; the pairs of B and C dots to arms; the pairs of D and E dots to legs. In the students’ brains, the combination of neural motion detectors and Gestalt perceptions were stitched together into a human.

Figure 6-3. Dots that move in the same direction (shown here with letters in common) appear to be part of the same rigid structure (i.e., arms, legs, torso).

Johansson added sinusoidal wave patterns and a few other complexities to the motion of the dots, and the students now saw a person walking toward them, or running, or bicycling, or even dancing with a partner. (We’d love to see a whole movie of dots, or even a whole movie genre.)

So why then do we perceive motion? Iconic memory is not the explanation. It’s the joint operation of visual vector analysisat the neural level and the Gestalt principle of common fate. Here is another example in a video of moving dots: https://youtu.be/pNe6fsaCVtI. Between :45 and :55, we perceive the two white dots as entirely independent of one another. But something magical happens at :55 as the third dot is added, and at 1:05 as the fourth dot is added - our Gestalt perception of common fate bolts the dots together into a rotating shape.

This means that if you ever take a job as a movie or video-game maker, you’ll find that motion perception is about as easy to evoke as depth perception. You don’t need much more than 10 dots and two converging lines to show a person walking off into the sunset.

That is, unless you want that motion to be perceived as totally fluid. That’s where it gets hard.

When making this kind of meme, you will need to choose which level of motion perception to strive for: successive motion perceptionor fluid motion perception. Successivity is easier to achieve and more crude; it’s simply our perception that something has shifted position upon seeing successive poses, despite our full awareness of the frozen before and after still-frames. Fluidity is achieved when all the artificial jerkiness and flickering in such a shift is rendered imperceptible, thereby resembling motion perception of real-life objects.

To illustrate, one of the earliest maze-themed video games was 3D Monster Mazereleased in 1982 (Figure 6-4). ^v The game was programmed at only six still-frames per second (fps). ^vi This frame-rate was at the slow end of any continuum you can name. The Hobbitof 2013 was filmed at 48 fps, video games like the Halo series are produced at 30 fps, and most 35mm movies run at 24 fps. But 3D Monster Maze was well below the minimum 12 fps needed to perceive fluid motion . The movement of Rex the dinosaur appeared jerky and strobe-like because we were able to make out individual frames in the animation. But make no mistake, we still thought Rex was moving, just in successive motion not fluid motion.

Figure 6-4. Early video games had successive motion rather than fluid motion.

So if you achieve fluid motion at 12 fps, why do you and other producers want to shoot up to four times faster than that? Well apparently there are certain things you cannot depict at slower (cheaper) frame-rates, and as professionals who are committed to a certain level of quality, compromising here drives you crazy.

What things? First is panning. You’d like to show a beautiful vista that cannot fit in one frame. So you need to pan across it. But if you pan too fast, fluid motion breaks down to successive motion in our perceptions, and we notice the “strobing” or “judder.” How fast is too fast to pan? Experts say anything faster than three degrees a second. That means you should take a full seven seconds to pan the width of one frame, 30 seconds to pan 90 degrees, and a full minute to pan 180 degrees! ^vii In movie time, that’s forever. And with your executive producers pressuring you not to waste our precious attention on such non-plot-advancing scenery shots, you often have to hurry up and squeeze out the beauty, which is the whole point of the shot. Tough dilemma.

Next, and perhaps more aggravating for you, is depicting very fast-moving objectslike jets, missiles, spaceships…asteroids…Thor’s Hammer…erhm, pretty much anything cool. Same thing: the faster these objects move, the less fluid they look without higher frame-rates, regardless of how high-def you shoot it. This is what galled Peter Jackson into filming 2013’s Hobbitat 48 fps. To land the meme with audiences, he wanted the vistas to take our breath away and the Orcs’ arrows to really move. So Jackson upped the frame-rate. (In many ways he was chasing the higher standards for action set by video-games, which had the advantage of 30 fps on PC monitors, not unlike many filmmakers of the time.)

With all respect to your craft, as your audience, we have to suggest that all this may be wasted effort. If you think your movie-making technology is poorly equipped to fluidly pan and pursue fast-moving objects, it turns out so are our nervous systems. If we were standing on a snowy ridge looking at the Lonely Mountain where Smaug the dragon lives, we wouldn’t fluidly pan from left to right. Instead our eyes would jump from spot to spot in what are called saccades.

The same thing goes for fast-moving objects. Our eyes cannot fluidly pursue anything moving more than 30 degrees per second: major-league baseball players can’t follow a pitch for the last 10 feet, and neither would we be likely to follow Thor’s hammer. In these cases, we need to perform catch-up saccadeswhere we rapidly shift our gaze (as fast as 500 degrees per second) to the point where we guess the object will be next. ^viii (Fortunately, thanks to iconic memory , we don’t perceive the blur as our eyes catch up, only our iconic memory of our last fixation point.) Producers should experiment more with what we might call “saccadic shots ” that show the first and last positions of objects (and maybe a little blur in the middle) and experiment whether that evokes in us the real-life wow of a really fast moving thing.

Put another way, you should not rule out the drama that can still be contained in successive motion. Malcolm Evans, the programmer of 3D Monster Maze, himself noted that, “Quite a few times the monster would come up on me with absolutely no warning whatsoever and I’d jump out of my skin.” ^ix We did too when we played it. He tried to program in a warning for us, “HE HAS SEEN YOU.” But this only amplified our dread, spawning a genre of “survivor terror” games played by millions to come.

The greater lesson here is sometimes you’ve already made it through a bottleneck and you don’t even appreciate it. If the investment needed to find a new way to achieve dimensionality, or to remove the last bit of judder in moving objects, grows exponentially as your marginal returns shrinks, you have to ask yourself if it’s really necessary. You have to ask yourself who is asking for it, because sometimes your users, customers, and audiences are not. Rex made it through the bottleneck of motion perception, regardless of his campy strobe-like gait, and it was the Hitchcockian plot-mechanics of fear and dread that catapulted the game into a commercial and critical success.