4
Storytelling Principles for Immersive Space
A Look at Immersive Theater
Among the various fields that can teach us potential storytelling principles in VR, immersive theater ranks among the most helpful. The origins of immersive theater go back to the beginnings of modern theater in the 19th century. A great deal of experimental and interactive theater and their aesthetics in the modern setting is constructed around ideas that first came to prominence in the 1930s with John Dewey’s book Art as Experience. Dewey teased out the significance of a piece of art from its “expressive object.” He argued that it was the experience of that significance that stayed with the viewer and not the image of the object itself. Dewey built his work around the biological and psychological effects that art had on viewers around his own theories in functional psychology. His work moved many creators from thinking in terms of object-based meaning making to experiential forms, opening doors for immersive theater and eventually experiences such as VR. Even traditional theater has used some immersive or interactive elements from time to time. In 1985, the Tony Award-winning Best Musical, The Mystery of Edwin Drood, required that the audience vote on who killed one of the key characters, making seven different endings possible. Practices such as call and response and other interactive elements have fallen in and out of fashion throughout the decades, but there has long been a desire to more deeply immerse audiences in produced theatrical experiences.
Some date the explosion of the current immersive theater movement to the production of Sleep No More, a 1930s film noir adaptation of Shakespeare’s Macbeth, based in New York. The defining characteristic of immersive theater is that it breaks the fourth wall, which has been discussed previously. As we recall, this wall is what traditionally separates the viewer from the audience both physically and verbally. Some of those working heavily in the practice have suggested that there are two major reasons the word “immersive” has risen in popularity. Mikhael Tara Garver has articulated these as such: “The first is that our audiences are in a cultural moment of two-dimensional overload and are craving experiences. The second is that from shopping to Twitter they are already participating in non-hierarchical interaction. When people say immersive the two things I believe they always mean are: multi-experiential and freedom to respond.”1
One approach we can take to examine any immersive medium, including immersive theater, is to consider that it is a mediated attempt at creating a “real-life” experience. These experiences strive to replicate the sounds, images, and feelings we have become familiar with in given situations. For example, when sitting in a city park, we hear the sounds of the environment—the wind blowing around us, children playing nearby but slightly behind us, a fly that keeps buzzing by our ear, planes that fly by above us, and a variety of other even more subtle sounds that surround us in three-dimensional space. In mediated immersive experiences, this is accomplished through sound effects, performers’ voices, and even recorded music that rouse our emotions to places that reality seems to offer so naturally.
Theater of the Oppressed
In examining what can be gained by looking at immersive theater, we could likely find no better template than the Theater of the Oppressed. Created by Brazilian theater producer Augusto Boal in the 1950s, the Theatre of the Oppressed describes theatrical forms influenced by the work of Paulo Freire. Boal’s techniques involved the audience in innovative ways, promoting social and political change. The audience becomes “spect-actors” in the Theater of the Oppressed. They explore, show, analyze, and transform their reality. The key characteristic that made this movement different was that it was based on the idea of dialogue and interaction between audience and performer.
The structure of productions in these experiences revolves around a neutral party to be at the center of the action. This performer is usually called the “facilitator.” The facilitator takes responsibility for the logistics of the process and guides the experience but never comments on or intervenes in the performance itself. This is the domain of the “spect-actors,” who are both spectators as well as actors in the experience. The philosophy behind this approach is that it eliminates the concept that the ruling class is forcing their ideals on the audience and thus making the audience victims of those ideals. In other words, the ideas and themes of the work are not given to the characters in the story, who end up becoming a substitute for the audience themselves, but instead directly to the audience, who are now free to think and act for themselves. The obvious applications and connections to VR, AR, and MR are endless. Most major cities now offer a number of immersive theatrical experiences. Creators of emerging immersive storytelling mediums would do well to investigate them.
Experimental Art Installations
The installation work of Janet Cardiff and George Bures Miller has offered concepts and ideas of interest to immersive creators. “The Dark Pool,” shown in Vancouver in 1995, consisted of a dimly lit room as a highly designed set piece, complete with artifacts and props. As viewers moved through the installation, various sounds were triggered ranging from musical selections to snippets of conversations and narratives around objects. One of their most celebrated pieces, “The Paradise Institute,” shown at the Venice Biennale in 2001, focused on the experience of cinema itself and consisted of a reproduced mini-movie theater. Viewers climbed a short stair case and entered a plush two-row theater. They could then peer over the balcony of a movie theater from the golden age of cinema. Headphones provided audio for the film once it began but also audience intrusions such as ringing from cell phones and whispers of conversation. The immersion was created for the viewer using a concept the creators termed hyper-perspective, in which a realistic experience is crafted through manipulating and mediating the viewer’s point of view. Audio was also used as a major component in directing the user’s attention in these installations. A viewer would tend to look in the direction of the source of an audio segment if they felt it was significant in some way and not just diegetic to the room, even if actors were engaged and visible in their direct line of sight. These techniques are now often used by VR creators to direct the audience’s attention in 360-degree open worlds. As has been discussed in the work of Bolter and Grusin earlier, the installations gave users the immediacy they craved with the potential of transforming that experience into hypermediacy. The work of Cardiff and Miller provided early ideas about how audiences would respond in mediated immersive environments such as VR.
Theme Park Attractions
Another industry that has driven audience expectations for the immersive is that of the theme parks. Experiential attractions have offered visitors the opportunity to move from being simple spectators to active participants in adventure and relatively safe from risk—motion sickness notwithstanding. Of course, motion sickness has also been a risk and reality in VR experiences as well. Different than traditional theater or cinematic experiences that are presented to crowds, theme park attractions are experienced individually or in small groups. One other notable similarity is that the two experiences are relatively short compared with many other entertainment experiences. Theme parks have focused on giving audiences a number of short experiences, rather than, say, a two-hour roller coaster. Presently, VR experiences tend to be most effective when presented in experiences of similar lengths.
While we will likely one day see VR experiences that mirror that of feature-length films, it could be important to recognize that audiences have established paces, rhythms, and even tolerances for experiences that involve their physical immersion. Theme parks have built in passive attractions, such as shows and exhibits, that allow the visitor to escape the immersion the more active experiences offer. While there are significant differences between VR and theme park attractions, the similarities are important enough to mine for methods and principles we should at least experiment with as we move forward with our new technological possibilities.
Nonlinear Storytelling
Nonlinear storytelling occurs when narrative events are portrayed out of chronological order or in ways such that the events don’t follow a structure where effects are the direct result of causality. The structure and recall of human memory is often nonlinear, and this category of story is often meant to mimic that process. Some research suggests that even when stories are told in a nonlinear fashion, they still must adhere to linear logic.2 Since audiences have gotten more familiar and comfortable with nonlinear narratives through the use of hyperlinked media on the internet, nonlinear storytelling in other mediums has become even more common. Many immersive theatrical experiences rely on a nonlinear storytelling approach while still adhering to other basic forms in storytelling, such as introducing a protagonist or empathetic character(s) for the audience to follow. A great number of the existing VR experiences and certainly many of those that will be to come will rely on the risks and rewards that are inherent to telling an audience a story in which chronology and causality take lesser roles. The key factor that will allow for their success will be continual reliance on linear logic—those inherent elements in our psychology that are needed to even engage an experience. As has been stated previously, the audience must be given something to orient some part of their experience.
SPOTLIGHT ON IMMERSIVE THEATER: Storytelling through Immersion
Noah Nelson is the creator and host of the No Proscenium Podcast, a program that focuses on the world of interactive and immersive theatrical experiences.
| John Bucher: | Let’s begin by talking about the idea of “experience” in this technological age. Whether it be experiential theater or experiential storytelling—why are people craving a deeper experience? Why is the word “experience” on everybody’s tongue, from the people at Disney to the people in the technology world? What is it about “experience” that we need it right now? |
| Noah Nelson: | I think we talk about it because we finally have the means by which to talk about it. We couldn’t talk about it until we could see it, and we saw it once we got the language. Now that we have the language, we are obsessed with it. It doesn’t mean that there aren’t other things that need to be considered, and it doesn’t mean that it’s the be-all and end-all, but it is a very potent tool. And these are the things that people have always done, but only now do we have the language to talk about them in these sorts of engineering ways. We talk about stories. We talk about memes. I happen to have a horrible allergic reaction when using the word “content.” We started to view the world through the lens of engineers and their lens in terms of the experience of designers. The late Brian Clark gave some great speeches about phenomenology. That level of knowing that the final assemblage point for anything is in the brain of the audience member, of the participant, then you start to see that everything is conditional in the experience. |
| If you’re keeping that in mind, you can either provide an optimal experience for someone to encounter the idea you’re trying to engender on the set or you can manipulate it in such a way that it defies conventions. Or, in the world of immersive theater, we start bringing in more senses such as sound—which is also incredibly important in the world of VR—because sound is the main driver of attention. To get somebody to turn around, you’re going to use that. Binaural audio has been around for a while now but is only going to grow in popularity with the emergence of these immersive experiences. | |
| John Bucher: | I want to expand and ask you even more about audio because I think there’s something about audio where we are going back to an even earlier form of communication that’s very telling, when memories and stories were passed down through word of mouth. Before stories were ever acted out in a theatrical expansion, stories were told. Anybody that knows anything about modern story theory knows that we’ve pushed away the idea of telling people stories. Do you think we are returning back to some of those ancient telling of stories, or are we going to create stories in a different way altogether? |
| Noah Nelson: | I don’t think we are using storytelling. TV didn’t kill radio. Radio didn’t kill books. VR is not killing film. If anything, there’s a chance for a renaissance. There is a chance to rediscover some of the old tools. Podcast will start rediscovering radio tools again. Radio storytelling. The idea of an immersive audio landscape. Or even when you think about a piece of immersive theater like Then She Fell where storytelling is part of the game, there are parts where they are just telling you a bedtime story. |
| I think what’s more interesting there is how interactive is a return to primordial storytelling. Whenever I would run into people in the transmedia world, people were like, “interactive storytelling this” or “interactive storytelling that.” It was a brand new field for them, and I’m like, “when my mom told me a story, when I was a kid, it was interactive.” Because I would be like, “what about this,” or like, “what happened to the dog?” Then the parents are sitting there saying, “The dog did this.” It calls upon that. So, there’s that plasticity between the storyteller and the audience. You find that in role-playing games. You find it in improv. That strata is everywhere. But it’s largely invisible, because you can’t make money directly off that experience. You have to somehow invent the rule set that exists and sell that to people. That’s how role-playing games do it. They invent a world and sell you all the pieces of the world. Anyone who has Dungeons & Dragons explained to them could run off and make up their own Dungeons & Dragons rules in five seconds straight. You have to convince people to buy the paraphernalia in a different way. But that strata is there. | |
| In VR what’s interesting is every moment you’re in the landscape, got the headset on, you’re in that relationship with the story world that is unfolding. Whether or not the storyteller has accounted for that or has the tools to deal with that is what’s an issue right now. I do know that there are people who are prepping a demo in about a month, some people will work on the problem of triggering events based on head turns, a bunch of people are working on that. | |
| I know folks like Jessica Brillhart are working on editing and the idea of drawing attention and using that attention point to create a focus for an edit, and watching edits actually work in VR for the first time. Which, when the Ozo launched, I finally saw editing on the action and editing on points of focus. And some of what that is, is saying the point of an edit, or the thing that gives you the opportunity for an edit, is if you successfully capture the attention of the audience and now you’ve got them so you can move them into a new phase. | |
| I think a couple people were at first obsessed with the idea that “they can look anywhere so I’m not going to restrict where they look,” and I would respond, “yeah, but you can’t tell a story if you don’t have some of the tension captured.” But even then, it’ll have different end results from a very well-managed experience, in much the same way that there’s a big difference between the immersive theatrical experiences Sleep No More, which is a sandbox experience where you can go wherever the hell you want to go, and Then She Fell, which is guiding you from point to point. They know they have your attention, at least in terms of what room you’re in, and so then they’re playing with your attention other ways. | |
| John Bucher: | What can VR storytellers learn from immersive theater, which has been around much longer? |
| Noah Nelson: | Let’s just start by saying everything. It’s funny because I’ve been saying this for a few years now. I know I’m not the only one saying it, but it’s funny to watch everyone else catch up. I think the biggest thing, the absolute biggest thing, the fundamental, is space. The relationship of actors to physical space, of the audience to the physical space, and indeed the virtual physical space. When I see VR filmmakers fail, oftentimes the first thing they don’t get right is camera placement or the distance between the subject and the camera. The next thing they screw up on often is constraints for the field of view. |
| Everything comes down to the bodies in space and reclaiming the z-axis. I like to talk about the reason why this generation of 3D films failed. Pretty much only James Cameron seemed to have an innate sense of what the z-axis does in a 3D film. That’s not to say that filmmakers don’t understand the z-axis, but they have a different relationship between the depth of field on a two-dimensional camera from the depth of field on a three-dimensional camera. It’s a very, very different tool. One that I think you see is traditional filmmakers who haven’t embraced 3D get arrogant about it because they think, “I know the way a z-axis works. I know what’s going to happen if I do this.” Not once you’ve kicked in the third dimension. Then you’ve got more stage direction. | |
| This idea of design, of everything that’s in that space is there for a reason. That’s what creates that sense of heightened reality. It’s the way that in Star Wars, that story world, everything’s got a backstory. In VR, everything has to be there for a purpose. Figure it out, and if you can’t figure it out, either take it out or create something. Because we will attach meaning to it. We want meaning. That’s why we tell stories. Period. We want to make sense of the world. | |
| John Bucher: | Right now the majority of people that experience VR experience it through a headset. We’ll obviously get away from that technology at some point—most likely. At what point do we stop calling it Virtual Reality and just call it reality? |
| Noah Nelson: | For those that are looking far enough ahead, which I’m trying to do, there are great concerns about the ethics of this stuff. There are technologies that promise to eventually be indistinguishable from normal reality. At that point you don’t, on some level, keep calling it Virtual Reality; it becomes a part of our world in much the same way that the internet increasingly has. |
| The AP Style Guide has now dropped the capital “I” from internet. I hate that because I like to think of it as a thing. But I’m from the nineties, and we like capitals on everything. Eventually, it will be just like water out of the tap. Of course, there’s an automated data layer in this café. Of course, if I tap on my contact lenses and I look over there, I could wave my hand to the cash register and pay my tab. A lot of the game is going to be figuring out what the gestural and the nongestural language is for accessing the data layer in ways that don’t feel stupid. | |
| That’s one of the things that’s interesting about immersive theater and this move towards experience away from the digital tools. You can see it too in the slow food movement. You can see it in the resurgence of third-wave coffee shops. You can see this desire to get away from the screens. We used to go online to get away from reality, and now we go to reality to get away from online. The most valuable and dangerous thing in the world is going to be an off switch. | |
| John Bucher: | At an immersive theatrical experience once, I had someone say to me, “This is salvation for the touch-starved and the lonely.” Can you talk about the relationship between Virtual Reality and loneliness? |
| Noah Nelson: | The question is, how far does the tech go? How good do the haptics get? How long until I put a data glove on and when someone touches my hand in VR, it feels like someone’s touching my hand? Because if I feel someone else’s touch, then some of that existentialist loneliness might go away. But we are not necessarily functions of all the stimuli that are coming in to us in our minds. So, can it fake the smell? Can it fake the temperature of another person’s hand? Can it do everything else? Until we’re doing that, deep down inside, you know it’s not real. I actually think we are aware now how disconnected we are from each other. We drive ourselves back to things that are. They talk about things not being authentic and then try to market “authenticity” to us and it’s grotesque, but we are looking for that. We’re looking for something that’s real. Which is a luxury now. It’s a very first-world problem. I just want an experience that’s real. |
| John Bucher: | Is there a danger that we become Descartes’s brains in vats just having a mad scientist send electrodes to us and thinking we’re experiencing a reality that we’re not? |
| Noah Nelson: | How do we know we’re not already? It becomes harder and harder to avoid how good the tools are. At the same time, each successive generation becomes savvier and savvier. I read one of Ray Kurzweil’s books in the late nineties—pre–9/11. I remember his metric was perfect transcription. We’re not there yet. For that reason, I don’t believe in perfect Virtual Reality because it always takes longer than we think it’s going to. |
Concepts to Consider from Noah Nelson
- Sound is the main driver of our attention and should be taken advantage of in VR stories.
- Immersive storytelling is a return to the form rather than an invention of it.
- An edit in VR should successfully capture the attention of the audience so that they might be brought into a new phase.
- The relationship of actors and the audience to space is key to creating immersion.
- Every element in a VR story experience should have a purpose.
Putting Ideas Together
Noah Nelson rightly suggests that every narrative decision, from the color of a wall to the point of view of the audience, should be motivated. If not, the creator misses an opportunity to add a layer of depth to the story. Because of the level of detail that VR offers in narrative creation, it will be easy and often tempting to leave what may seem to be minor details to chance when creating a VR experience. Storytellers must avoid doing so. The ability to keep an audience immersed in a story is one held with a sense of great fragility. Immersion can be broken at any moment. Having motivated reasons that serve the story for every creative decision within an experience provides one less opportunity for that immersion to be fractured. The narrative precision required for storytelling in VR may be higher than any other medium that has been available to creatives thus far. It should never be overlooked or taken lightly.
Notes
1Garver, Mikhael Tara. “Immersive Theater: The Senses to Take the Wall Down.” http://howlround.com/on-immersive-theater-the-senses-to-take-the-wall-down
2Martens, Chris, João F. Ferreira, Anne-Gwenn Bosser, Marc Cavazza. “ ‘Linear Logic for Non-Linear Storytelling’, ECAI 2010–19th European Conference on Artificial Intelligence,” in Coelho, H., Studer, R. and Wooldridge, M. (eds.) Frontiers in Artificial Intelligence and Applications. Pittsburgh: IOS Press. pp. 1–7.