2

A Stone Bridge Covered in Microchips

The Purpose and Goals of this Book

New fields and disciplines take time to develop. The only way that we eventually stop wandering around in the dark is when someone begins to create a map around the area they have found and someone else shines a light on it. Early developers and storytellers in VR have begun to create those maps. The purpose of this book is to shine a light on their discoveries and then draw a few conclusions as to what those discoveries might mean.

There has been some distrust between classical thinkers and technology proponents. Heidegger wrote in 1977, “Everywhere we remain unfree and chained to technology.”¹ Those who find deep value in the philosophies, methods, and truths of the ancients can be dismissive of ideas that seem to go viral overnight while ironically ranting about it on their smartphones. Those who spend their time pushing forward our technological capacities found an interest in doing so often because the ways and thinking of the past either didn’t make sense or simply weren’t working for them. What we will explore here is the desperate need that each camp has for the other in order to progress their work. There has long been a stone bridge covered in microchips that led the way to the future.

Interestingly, in technological fields that rely on principles of good storytelling, such as video game development and design and 3D animation, the quantity of training or discussion around the timeless truths that have long been taught about constructing narratives is minor compared to the technical training. A quick scan of the syllabi of the top three academic programs in these fields confirms this to be the case. It is sometimes assumed that since the platform is new, all narrative methods should be new as well. This occurs at the expense of effectiveness in storytelling. While exceptions certainly exist, and many developers have solid instincts for crafting stories, many more examples fall flat and do not resonate with their intended audience simply because the stories are ineffective. Creators scratch their heads and wonder why, failing to realize that while the world they have built and the characters in it resonate with them, they lack the universal elements necessary for a narrative to maintain longevity and engagement on a mass scale. Understanding what has endeared stories to audiences for millennia psychologically, philosophically, neuroscientifically, and even unconsciously is essential, especially when developing experiences on the scale that VR, AR, and Mixed Reality (MR) offer. These technologies may be often compared with the Wild West. However, it is important to remember that this is simply the season they presently exist in. Even the Wild West was eventually tamed with laws, structure, and form. Doing so is exactly what allowed our society to develop into a culture that would eventually produce the sorts of technology we all enjoy today.

In order to tame the Wild West of alternate realities, it is important that we look at the principles that have governed narratives since the days of Aristotle and perhaps before. Examining the architecture behind good storytelling gives the understanding we need in order to discuss how we move forward in crafting audience experiences that are successful. Understanding how stories work, what makes them effective, and the ways that they can be created will give us the materials to take into new realities in order to build new narrative structures and methods for new mediums and platforms.

What Makes a Good Story?

Revered author Flannery O’Conner once said that everyone seems to know what a story is until they sit down to write one. Understanding what makes a story good first requires that we define a few terms, namely “story” and “good.” As with most words, there are multiple definitions that can be applied to the terms, all of which help us get the words’ use and the applications we may have for such an expression. Story is derived from the Middle English word storie, which likely came from the Latin historia or the Anglo-French estorie. We see these terms as early as the 13th century. However, our word story doesn’t come into popular use until the 15th century.²

The meanings of the word today can range from the unit of measure used to describe the space in adjacent floor levels in a building to a widely told rumor. The definitions that Merriam-Webster offers, such as a narrative, a report about incidents or events, and a short, often amusing tale get us closer to the use we will apply but still lack precision. While we will eventually land on a definition that can be used for applications in VR, it may be helpful to further deconstruct the term “story” in terms of the elements used to construct stories. Stories often consist of the following features, which will be unpacked in greater detail later in the text.

Characters

It would be difficult, if not impossible, to convey a story without at least one character. Later, we will examine all the various characters found in stories, including essentials such as the protagonist, the antagonist, and a plethora of other archetypes.

External Goals

While various types of stories work with internal journeys and abstract expressive forms, visual storytelling usually requires a photographable goal that at least one character or protagonistic force is trying to achieve. While the protagonist in a story is often a single individual, a protagonistic force can include a duo or group of individuals that share the same external mission. It is important that the goal they share is something that can be seen. Finding love is an abstract concept that occurs inside a character. We cannot photograph the act of finding love; therefore it lacks the visuals required to stand on its own in visual storytelling. Finding a date to a dance, however, is a photographable external goal that would work well as evidence of a protagonist trying to find love.

Internal Goals

Seeing a character accomplish something is fundamental in a visual story. However, we also desire to know that they have accomplished something internally. The internal goal of a protagonist is also unique from the external goal in that the character is rarely aware of their own internal goal. It is clear to the audience but greatly resides in the character’s subconscious. An example of an internal goal might be to prove a character’s worth to his or her father.

Conflict

This is perhaps the most significant element found in stories. Without conflict, we merely have a scenario. Conflict can be built in a number of arenas within the story world. The conflict can come from the great odds the protagonist faces in accomplishing their goals. The conflict can come from another character, such as the antagonist, who wishes to accomplish the exact same goal. The conflict can also come from within the character in the form of self-doubt and insecurity.

Resolution

Stories resolve in a wide variety of ways. However, in order to feel some sense of satisfaction with the story, the audience needs to experience resolution of the conflict. The only exception would be if the goal of the story is to leave the audience unresolved. However, this can be risky and should only be executed in a precise fashion by experienced storytellers.

Stories in Alternate Realities

While there are certainly nuances to the stories told in alternate realities, for the purpose of this book, we will hold the following definition for a story told in VR, AR, or MR.

• A story is a sequence of events or scenarios that demonstrate characters trying to resolve conflict by accomplishing goals.

This definition will leave us space to expand the uses of the term “story” to fit most if not all applications of VR, AR, and MR technologies. The events or scenarios might be fiction, nonfiction, or taking place in real time as the viewer is navigating space. They might be for the purpose of entertainment or education or purely informational. The sequence of events used in the alternate reality story might be linear or nonlinear. There may be a single protagonist or multiple protagonistic forces. That protagonist might be the viewer or another character the viewer is observing. The resolution might be through the viewer accomplishing a clear external goal or through the feeling experienced when viewing another accomplish that goal. The resolution, as well as the goals, may be internal or external. All of these various combinations of applications we can experience in alternate reality space can qualify as stories.

Forms versus Formulas

This book is not a collection of formulas for how to tell stories in VR but instead a way of seeing VR so that you might tell stories in it. In essence, we will be attempting to highlight the establishment of a form or a language that seems to be emerging for telling stories in this new medium. Ironically, the narrative arts seem to be the lone resistant art form when it comes to form.

Courses in painting and in drawing all begin with rigorous exercises walking students through forms. Learning how shapes and colors affect audiences is a time-honored part of the discipline. When learning music, fundamental elements such as notes and scales are offered long before complex arrangements. In architecture, understanding how load-bearing walls, pitched roofs, and foundations work is essential before choosing how many windows will be in a given structure. All disciplines accept that structure is a key part of how their art form works. One cannot imagine a musician trying to invent a new note or chord. Architects never attempt to build a new home without walls or a floor. Yet storytellers try to ignore the forms and structural principles of the medium on a regular basis, often to their own detriment. This is usually because form is confused with formulae. In his book Story Proof: The Science Behind the Startling Power of Story, Kendall Haven states, “Story is not the information, the content. Story is a way of structuring information, a system of informational elements that most effectively create the essential context and relevance that engage receivers and enhance memory and the creation of meaning.”³

Even in narrative endeavors, formulas do work—if they are hidden well. There’s an old adage in Hollywood storytelling that says, “Give me the same thing—only different.” Though they might deny it, a large segment of audiences enjoys the experience of familiarity in a story. There’s a reward triggered in the brain when we see a pattern we recognize. The key is to then surprise the brain with a bit of irony. Even without this element, however, audiences often return to films they have seen dozens of times simply because they have a similar emotional experience even when they are aware of the structure and ending of a narrative. The power of narrative is in the journey.

Story Opportunities with VR and How VR Lends Itself to Certain Types of Stories

Every medium has its advantages and disadvantages when it comes to storytelling. Graphic novels offer visuals that tell stories in a way that novels are incapable of. However, novels can provide depth and internal insights that graphic novels are unable to replicate. Determining the advantages and disadvantages of VR can assist us in crafting stories that lean in to the medium instead of searching for workarounds. Current VR experiences require the use of an HMD. While there will likely come a time when virtual experiences can be had without a headset, this element is a necessary component of the VR experience presently. This provides advantages and disadvantages for the viewer as well as the storyteller. The advantage is that the viewer has their eyes sealed in darkness, save for the light emitted through the lens of the HMD. This quickly transports the viewer into an immersive experience. When headphones are added to the HMD, the sound further immerses the viewer by covering their ear canals. This disadvantage is, of course, that headsets can be heavy and, with their very presence, remind the viewer that what they are experiencing is manufactured and not real. While ineffective in most story worlds, stories that journey the viewer through an experience in which they might be hearing a similar headset in real life, such as with a spacecraft simulator, become much more effective.

Stories that take place in worlds outside our own immediately lend themselves to VR storytelling, allowing the viewer to be immersed in a land they have never or could never experience in real life is unique to the medium of VR. This includes created worlds as well as existing worlds and landscapes. Even stories that take place in familiar environments such as a domestic kitchen can be turned into new worlds with creative camera placement. While we could easily observe a couple having an argument in a kitchen in reality, we would never likely be allowed to stand between the couple and experience the tension held in that space. We could also never experience the same scenario from the perspective of a mouse on the floor. VR, of course, also affords us the ability to experience this argument in first-person form with someone we are not familiar with in reality in an unfamiliar space we have never stood in before. Our immediate response to VR storytelling might be to only craft narratives that take full advantage of the 360-degree landscape that we now have access to, which can be an effective strategy. However, we must also remember that many other experiences will be successful in VR space simply because of access we now have to locations and scenarios. Every narrative has the potential to be made interesting in virtual space.

The Role of Gaming and Game Theory in VR

While being a fairly recent addition to the potential storytelling canon, video games have become a major cultural force in a short amount of time. There is solid evidence that human beings have enjoyed games since times before language was a part of the human experience. Anyone who has played peek-a-boo with a baby understands this reality. There is additional evidence that games may even stretch beyond human activity into other forms of consciousness, as there is research that suggests certain species of animals participate in games as well. Throwing a ball for a dog to retrieve has become a common experience across cultures that would indicate this to be true. Games are part of the human experience. It should be no surprise that they will be part of our virtual experience as well.

Game theory is the study of how people behave in certain interactions, usually social and economic, and how they make decisions when immersed in these settings. It is an area of study originally developed by economists that has widened to other disciplines. A game in game theory is not just what traditionally has been described as games (soccer, chess, roulette, Monopoly, or tennis) but is any interaction between individuals or social groups. Essentially, it is the study of how these parties make decisions during interactions. When dealing with games, the participants are usually called players. Players must make decisions that display behaviors, usually called actions. The behavior leads to payoffs for the players. This, of course, closely resembles aspects of narrative structure, where we have characters, who make decisions and take action that cause setups, leading to eventual payoffs.

Game theory is usually approached with one of three lenses or sometimes a combination of the three. The social science approach explores how games affect viewers, both individually and on group and society levels. The humanities approach explores what meanings and philosophies are expressed through games. The industrial and engineering approach explores the technical aspects of the discipline including elements such as computer graphics, artificial intelligence, and networking. Often, theories evolve as the discoveries in one of these areas affect our understandings of the others. Selmer Bringsjord of the Rensselaer Polytechnic Institute (RPI) has looked at game theory through a narrative lens. He suggests that while many games are compelling, they may not be dramatically compelling in the same way that Dante’s Inferno, Hamlet, Gibson’s Neuromancer, and the plays of Ibsen are compelling. Bringsjord goes on to ask if it is even possible to build dramatically compelling interactive digital entertainment in the form of computer games.⁴ He suggests that doing so will require seminal advances in the intersection of artificial intelligence (AI) and narrative. “Since interactive digital narrative will need to be crafted and massaged as the story is unfolding, computers, not slow-by-comparison humans, will need to be enlisted as at least decent dramatists—but getting a computer to be a dramatist requires remarkable AI,” he states.⁵ This level of AI and narrative capability, however, seems to be quickly approaching.

Ludology versus Narratology in Game Studies

The most heated debate in game studies since the 1990s has been around this topic of the unfolding story and what philosophies and approaches apply to it. While the intensity of the debate has subsided, neither philosophy has become dominant. One group of theorists, called ludologists, support that stories are only a subset of games and that games should not be analyzed primarily in terms of narrative. Another group, called narratologists, insists that games are a subset of stories and thus subject to primarily narrative analysis. Still others offer that this binary approach is altogether the wrong path.⁶ There is value to both approaches for VR, and ludologists do concede that there are, without question, narrative elements found in video games.⁷ Some VR experiences would tend to favor the case for ludology, building the world of the user as he or she progresses in the experience, free from any narrative traditions or constraints. Other experiences clearly employ an underlying philosophy of narratology, assuring the user that every decision and element of agency they experience fits into a larger planned narrative. The title of this book likely gives the reader an indication of where the author falls on this spectrum. However, since VR is an emerging medium, not a video game and not cinema, considerations from every philosophical approach should be welcome. Some immersive creators will gravitate toward an approach that favors the entire experience that he or she creates to feel as an overarching narrative. Others will prefer only to see minor narrative elements in the work they create. As VR continues to progress, there should remain room for both types of creators.

Theoretical Lessons in 3D Space

The origins of how objects work in three-dimensional space and the basic study of geometry go back to ancient times. As technology has advanced, the representations of objects and shapes in 3D space has become of greater interest, especially to those in the computing industry. Most recently, the orientation of these objects and the “camera view” through which the viewer sees them have become important aspects of storytelling in virtual three-dimensional space. While the approaches and methods behind these ideas come from earlier disciplines of painting, photography, and filmmaking, nuances have risen specific to computerized versions of these representations. While painters took advantage of and crafted with natural light, photographers and filmmakers used technology to achieve effects with lighting. Now VR creators have the ability not only to tell stories with shadow and light but also to create those elements themselves from scratch. In addition, they have the ability to craft the textures on the objects that will respond to the light. While a greater degree of control has been handed to the creators, the way that human beings react to shadow and light has not changed since the early painters were working with these elements. The most dedicated artists in virtual world building still go back and refer to the lessons and discoveries of early artists, as the basic principles have not changed. What has changed is that viewers experience a greater sense of entering the space of the virtual third dimension rather than simply letting their brains only imagine that process. The irony is that it is still the brain imagining the process, but the realization of the mediation is less opaque. Perhaps the most repeated anecdotal lesson from those designing in three-dimensional space is that every angle must be considered in terms of the viewer’s experience. If you are going to allow a viewer to pick an object up or enter through a doorway, the entirety of the experience must be considered as each option is presented. In other words, the larger narrative must be held in one hand while holding every angle of every detail in the other.

Concepts to Consider from Brian Allgeier

• Practical aspects of VR lend themselves to certain genres such as horror.
• Environmental storytelling is a key part of VR gaming narratives.
• Older art forms such as sleight of hand offer principles that still work in directing an audience’s attention.
• Three-act structure persists in many immersive VR experiences.
• Transitions between narrative scenes are important and can break the immersion for the viewer.
• Perception continues to be an important part of the VR user experience.

Putting Ideas Together

Brian Allgeier reinforces the concept of story structure in VR experiences, allowing for modifications that suit emerging immersive technologies, such as the redefining of scene transitions. He also reminds us that other, older methods, such as those used in the world of stage magic, may be worth our consideration as we continue to seek ways to tell stories, in which the audience becomes unaware of the medium with which they are being communicated to. His suggestions prompt us to consider the necessity of the creator’s invisibility as we strive to create immersive experiences. It can be tempting to call attention to novelty or cleverness when designing an experience. The most effective hard work, however, seeks to disguise itself, allowing the audience to become caught up in the narrative—their minds convinced that the experience is their own and not only that of a character they encounter in the virtual world.

The Land beyond the Bridge

We began this chapter by looking at the metaphor of a stone bridge covered in microchips. But what lies beyond the bridge once we have crossed it? In 1902, a French magician named Georges Méliès created a film called Le Voyage dans la Lune (A Trip to the Moon). It is a story about a group of scientists, who bear a strong resemblance to the wizards of Harry Potter lore, that successfully launch a manned rocket to the moon. While there, the explorers encounter aliens, battle them, and even find a way to destroy them. They safely return to Earth by pushing their spacecraft back off a cliff, where it falls back to the earth. The film is widely credited as one of the first pieces of cinema that told a coherent story. While the practical effects used by Méliès get the majority of the attention, one factor that rarely receives discussion is what the filmmaker got right and completely missed the mark on when it came to actually embarking on a trip to the moon. It is hard for modern audiences to appreciate the fact that when the story was told, humankind had never been to the moon before. Our technology was far from being capable of such endeavors. From a purely scientific perspective, we can appreciate that Méliès correctly identified one of the key factors that would keep us from making such a voyage in real life for decades—power to blast a craft that distance against the forces of gravity. The first act of his film explores this problem. Méliès also correctly assumed the atmosphere of space would not be welcoming to humans. He costumed his characters in space suits with helmets that would assist with their breathing. His ideas about our encountering alien life forms, necessitating a battle, is still a common trope used in storytelling today. However, there are a vast number of assumptions Méliès got wrong—things he just could have never known or anticipated. No historians fault him for such errors; we instead celebrate his forward thinking and the creative vision. History will likely look back at modern creators with the same grace. We must not be afraid to push the boundaries and explore this new medium of storytelling. We should embrace the principles from the past that still serve us well. We should seek innovation and modification of those that do not. There are unfathomable stories that lie beyond that stone bridge covered in microchips. Let us move forward into the new world that awaits.