Idea

The first decision is an obvious one – what idea are you going to depict? What are you trying to visualize to share with others? Don’t try to figure out which of your dozens of ideas is the best. Pick the one you feel could be explained best given your skillset and team.

For example, maybe you want to depict telepresence to your audience. Telepresence is broad, and could be in a business setting, a medical setting, an educational setting, and the list goes on and on. Don’t waste time examining every scenario in a futile attempt to find “the best” one.

Just choose! Flip a coin if you must, or just choose one that’s “good enough.” Got one in mind? Great. Let’s move on to figuring out how to tell its story.

Scenario

Every breakthrough idea has a compelling scenario that really brings it to life for people. Scenarios are a sequence of actions (or possible outcomes) that highlight the human side of the idea you are excited about. These are effectively short stories or individual scenes in a play.

Well-constructed scenarios allow for paths through the narrative, provide some sort of tension (just like good sequences within films or books), and ultimately leave us with a satisfying ending. Scenarios don’t need to be complicated. In fact, you can typically explain a scenario in just a few well-crafted sentences.

Scenarios follow all the same rules as stories – they have a beginning, middle, and climactic ending. The setup of a scenario involves a situation or problem the main character is grappling with. Next, something happens to introduce our heroes (the holograms) into the scene, and finally through interaction we see how they have made the situation so much better or more interesting.

Remember that it’s never about the technology when we’re doing envisioning, so this scenario so focus on the human side of the equation. It needs to show off how your idea and associated tech will help people do something they can’t quite accomplish today.

Picking the right scenario to go deep on can seem daunting. It shouldn’t be. There’s no data required to figure this out. It’s more about listening to your gut instinct on what story will play out best for people in this narrative form.

Technique

With so many compelling ways to capture and express your ideas for mixed reality experiences like sketching or acting things out, it shouldn’t be that hard to just pick one and start running. But, like many other things in life it’s never really that simple. As the sage designer Bill Buxton points out, all design techniques are great at some things and lacking in others. Some envisioning approaches are quick to execute since they’re a bit less refined, while others are slick and persuasive in order to provide an amazing emotional connection. There’s no shortage of ways to tell your story memorably. The trick is picking the one that lets you work through it quickly to create something that can be shared and discussed.

We’ll cover why certain techniques work better than others in the next section, but for now here are the decision points when choosing – those easiest for non-technical people to leverage, the most persuasive, higher production quality, and those best for delving into interaction questions.

Deciding on which envisioning technique to use should be goal based, not skills based

Is the scenario a technical one or not? Does it require special software or hardware to pull off? Will this be a quick exploration or is it more involved? All of those questions figure into the final decision on technique.

In the end, your final decision may come down to which technique you’re most familiar and comfortable with. And frankly, that’s just human nature. We’re not always going to jump into a completely new learning situation just to broaden our skillset or satisfy our curiosity. There are deadlines, clients, and schedules to appease. All that said, time permitting, I’d encourage you to try various approaches to see for yourself the pros and cons of each, so the next round of envisioning is that much easier. For now, let’s just pick and move.

Timeframe

Nothing sucks the life out of a new project faster than putting a schedule in place before doing any real work. Really. Once you set the end date of something it’s almost as if you just shutdown the creative possibilities. When you start working backwards to figure out what fits and what doesn’t in your schedule, you have essentially boxed yourself into a forgone conclusion. I suppose that’s the reality of creating things within agile development sprints these days, but doesn’t mean you have to run your envisioning project that way.

We can choose to look at the envisioning phase of projects another way, where the overall fixed timeframes provide a great opportunity to reconsider our approach. Why not do many small explorations that far less in-depth rather than one super deep dive to cover more ground and a wider scope?

The value of doing many quick explorations versus one long sprint is magnified in MR

This situation highlights the classic dilemma in project management – go faster to get further in a shorter time with less quality, or spend the time to more deeply explore the design space? In envisioning, we need to lean toward the first approach, meaning faster is better in almost all cases. Overthinking and overanalyzing things is not helping anyone.

Deciding to aggressively timebox an envisioning exploration forces you to consider the key aspects upfront (or run the risk of floundering about toward the middle of the project).

Impact

The ability to leave a lasting impression directly affects the success of your envisioning efforts. Creating real impact means influencing someone’s thinking in significant ways. That often requires you to distill things down to just one or two key impressions. Identifying those can be a difficult thing to pull off, but when done well, these signature moments will stay with people long after they first encounter them. The great ones leave permanent imprints on our memories because of the connections they made.

It’s a tall order, impact. Given we are trying to evoke particular emotions and feelings in the viewer, it’s best to consider impact as a goal rather than an expected result. We can say that our chosen approach will cause people to laugh or be entertained, but it’s another thing entirely to believe we can alter their thinking.

The delivery mechanism (or medium) used also plays a critical role in getting to the desired impact. Pick something too low fidelity and you can’t create enough of an impression to be memorable. Choose a higher fidelity and you’ll find yourself too busy to try out different ideas. Best to shoot for something in the middle that delivers the emotions you need without dragging you into a drawn out production process.

One subtle thing to realize as we’re working through these key aspects is that creating impact is not the same as driving an outcome. Impact is emotional. Outcome is action oriented.

Outcome

Behaviors and actions that happen as a result of an envisioned experience are the outcomes of our effort. Outcomes are the easiest aspect to picture and define, yet hard to predictably achieve. People’s behavior will vary. Environmental factors play a role. Even the quality of the execution figures into how scenarios turn out in practice. Outcomes happen. They do not follow logically.

That said, there’s a good reason to try and propel the scenario toward the desired outcome through any means you feel is appropriate. Well-designed outcomes yield higher satisfaction from the participant. Giving people direction, advice, or hints as to the next logical step along their scenario’s path helps to provide a sense of accomplishment and achievement, much like leveling-up in a video game. We want people to have a sense of satisfaction from their efforts.

The skill of knowing which envisioning approach and technique is most efficient in getting to a desired result comes with practice. That’s why we talk about exploring quickly and often, to gain that comfort and expertise. If you are trying to convince someone to fund your idea, choosing a higher fidelity technique like video or working code is probably going to get you further than a sketchy hand-drawn storyboard to show your coworkers. Fidelity and style definitely affect impression and ultimately outcome.

Another interesting part of determining outcomes is figuring out how to deal with unintended side effects and unexpected consequences. Just because we didn’t realize our scenarios would result in unforeseen outcomes doesn’t mean they aren’t valuable. Digging into the thought process and actions that led to the side effect or consequence will help you become a better interaction designer. Look for these opportunities as you work through the build-out and testing of your ideas.

Skillset

The talents that enable people to communicate using particular techniques are the skillset we’re most interested in understanding and harnessing. Individual skills in this area can be as specific as the ability to draw and create 3D animations, or be so general as to not usually show up on a resume (such as being convincing or persuasive).

Knowing what skills you and your team have before starting down the build sequence is helpful in planning or establishing a confidence level. That said, lack of skills shouldn’t be an argument to not try something out. In this new medium of mixed reality, new skills should be picked up as needed. Don’t be afraid of what you don’t know how to do. Being uncomfortable is the new normal in this field. It’s all happening so fast, it’s best to set off in the direction you think will result in convincing storytelling and then find ways to get there as you go along.

Another aspect of growing your skillset as you go is purposely seeking out diverse perspectives and new techniques. It’s incredibly clear from working with fast moving, innovative teams that having a more diverse background and expertise results in a better collaboration and outcomes. Whether you are working by yourself or can recruit a team to collaborate with, try to find ways to leverage the skills you may not feel as comfortable leading with – dramatic storytelling, technical detail, visionary aesthetic, whatever it takes to get the idea across persuasively.

Realism

It’s important to note that by its nature envisioning does not have to be completely realistic to the current capabilities of the platforms or tools that currently exist. Envisioning is generally “of the future” and as such can feel more inspirational than factual.

As designers, part of our job is to push the boundaries of what could be to illustrate how things should be. Conceptual design or envisioning is a great way to do that. When done well, it poses hard questions, raises concerns, pushes engineering to consider, and creates dialog to further explore the issues. At its best, this type of envisioning is inspirational and provides a North Star for aspirations and dreams.

For example, in my HoloScenes series there’s an underlying premise that anyone can see the holograms in open spaces because they have been publicly shared and the technology used to see them is prevalent and frequently used. None of that is true today, but that’s not the point. It illustrates a possible future that we can work toward.

Envisioning can also be used to show your ideas quite realistically, remaining true to existing capabilities and norms. This type of design is done to help people visualize something that has yet to be built. It’s meant as a blueprint of sorts rather than a directional suggestion. The real difference between this type of envisioning and a more fanciful treatment of the subject matter is that people can recognize the realistic nature.

Using highly detailed models for your holograms is stunning, but can affect performance

Combining both realistic and futuristic aspects of ideas is my favorite way to work in this medium as you saw throughout my examples. We can combine established practices and capabilities with hopeful improvements to take your audience where you need the story to go — whether to push on the technology side of the project and platform, or to set new ideas forth for interaction and participatory possibilities.

Envisioning is exactly what you need it to be, realistic or fanciful. It’s an aspect that determines how your idea will ultimately be perceived.

The Creators

It’s all too easy to forget about those behind the scenes – the creators of the holographic experiences people participate in. Theirs are some of the most important views to consider. Understanding and taking stock of the perspectives of those who’ll be creating these visions will help determine the right path to head down. Chances are several people will be involved in bringing this to life, so let’s understand how each of their roles is best suited to help move the vision to reality. This is no means exhaustive, just a representative sampling of the roles you undoubtedly have on your team or possess yourself.

Designers tend to take a three hundred sixty-degree approach to the scenario, wanting to explore all of the different elements in play before committing to any particular execution. We like to understand the people side of the scenario, the motivations, setup, and emotional aspects as well as the interactions and tech. That well-rounded viewpoint makes working through the scenario that much easier. Add to that the attention-to-detail that most designers have and you’ve covered a huge amount of ground by engaging with people in this role.

Developers love to dig right into the technical aspects of the scenario, making sure we get how this will all unfold from an executional perspective. That’s incredibly valuable in the early stages of figuring out the scenario (as long as this reality check doesn’t squash ideas). There’s always going to be technical issues when turning ideas into working code. The trick is to not let the weight of the challenge drive the enthusiasm out of the effort to overcome it.

Storytellers (otherwise known as marketers and executives) are worth their weight in gold during the early phases of envisioning. We’re not all as skilled in weaving narratives that stick with people long after they’ve heard them, nor are we articulate and concise enough about our own thoughts to convey them clearly and succinctly. Some of us are natural storytellers, yet the majority of us can certainly use another pair of ears to listen and repeat back to us what we wish we had said. People who spend most of their working lives constructing narratives and quickly getting to the point often have the innate ability to see into the heart of complex scenarios.

The Participants

Empathizing with the perspective of the people within the experience is really important when envisioning. We talk at length about the fictitious “user” (which doesn’t actually exist, don’t get me started) but little about the viewpoint of real people with feelings and emotions. Putting yourself into situations you can’t normally relate to is eye opening and enlightening if you let it be. It’s something you may need to work at a bit to get good at.

There’s also a very old adage in the high-tech industry that goes something like “Don’t design for yourself, because you’re not the user.” That couldn’t be further from the truth when it comes to designing holographic experiences. We are all still enthralled with this new medium, so even the most experienced among us have similar reactions. The shine doesn’t wear off that fast. Taking the viewpoint of someone in the mix isn’t that hard – it’s natural. Be the ball. Put yourself and your own views inside the scenario and explore how you’d feel in that position. Lean into it. Own it. Then test it yourself. Be ready to prove your findings to others through immersing them, too.

After understanding how you’d feel within the scenario, ask others how they’d feel. Listen carefully. Ask questions. Other people’s views are often the bolt of lightning you need to disrupt a locked in view. Don’t let the sting of a contradictory opinion shut you down. Make sure you really get what they’re saying, then ask them to act it out if possible. Now here’s the kicker - take the feedback you get from others more seriously than your own perspective. I’m not saying temper or bend your vision to fit other’s views, but the fact is there are so many things we can’t see from our vantage point we need to err on the side of other’s viewpoints. There’s a whole spectrum of things we miss – from cultural norms to intellectual sophistication. Political views. Age appropriateness. Technical savviness. They all play into how your vision is accepted or not by the audience.

This is another great example where putting people before technology helps get you into a much better positon to start from. Consider how people will react to your vision before doing much else. It’ll help guide every decision from then on.

The Onlookers

Let’s not forget about the people who aren’t directly involved in action. They aren’t distanced like an audience in a play. They are standing right next to you. Knowing how these onlookers think and feel about what they’re witnessing can only help to positively influence the work. They may or may not be directly interacting with the digital actors themselves, but they are certainly a key part of the overall experience. The participant is aware of them and vice versa. They play off each other, perhaps not as intensely as a live band and its crowd feed off each other, but there’s a constant awareness.

It’s also easy to forget all this holographic stuff is a mighty strange thing to behold for the first time as an observer. Let’s not kid ourselves. People with headsets, visors, or glasses waving their arms and talking with invisible objects or people is pretty strange. It’s hard to not react to that.

Don’t kid yourself. People wearing special optics are not blending in.

Remember how you reacted the first time you saw someone wearing one of those ear-mounted Bluetooth headsets? Exactly. It’s weird. Hard to have some reaction or at least an internal conversation as an onlooker. Even someone talking to their personal assistant on a mobile gives you pause every now and then. Just be aware that we are not quite at that point of people regarding this type of interaction as an every day, run of the mill occurrence. Something to keep in mind.

You look completely ridiculous wearing any kind of headset or mixed reality gear. Just accept it and move on.

Since the participant can see all of the onlookers in mixed reality, they probably should come into play in some fashion when as the scenario unfolds. Well-designed scenarios fully integrate these spectators into the action rather than ignoring their existence. Why not utilize them are key elements in the narrative if they exist? And if not, be prepared for interruptions by onlookers and people who genuinely belong in that same space as the participant.

The Holograms

You better believe holograms have a unique perspective. Not merely digital actors or props, holograms are the stars of the show. They need to emote, provoke, and entertain as much as any actor on the stage. That requires a perspective and a point-of-view to properly calibrate. This is another great design exercise to jump into – put yourself in the position of the hologram, whatever it is, and play out the possibilities within the scenario from that perspective. We talked earlier about how physically acting out a scene is a valuable technique for envisioning. This situation of needing to understand a hologram’s perspective and viewpoint is perfect for that technique.

Finally, let’s agree that holograms have feelings, too. Not kidding here. They are not just digital constructs that carry out our programming within the scenario and do our bidding. They show as much emotion as we do (if we as creators think that far into the scenario). We need to treat holograms as actors in the play, co-stars in the production, meaningful partners in our endeavor. It doesn’t matter whether a hologram is a rock or a rock star – they both have the ability to understand and react to anything in the scenario if we make it so.

Imagine how it would feel to have a holographic rock react to the emotional state of the scenario participant, much like a 1970s mood ring would purportedly change colors depending upon its wearer’s emotions. Does a mood ring chemically responding to changes in body temperature and pulse constitute an emotional perspective? Of course not. But, that’s the kind of effect that creates a connection with people and makes them believe in the illusion. The mood ring provided a means of tracking and responding to physical state, not so different than we can provide via our elaborate sensors and cameras. The hologram doesn’t actually need to understand the participant, but it can reflect what it observes about their current frame of mind to work on that connection.

Even the hint of emotion from a hologram endears them to you

Emotions are the most powerful motivators we have in any kind of storytelling, so it follows that holograms employing at least a semblance of emotion when interacting to any kind of stimuli will add to the overall immersive illusion. Using this premise that holograms do really have feelings adds a powerful driver of action to any scenario. It can easily be overdone or misused, but imagine the possibilities if we learn how to add just enough to cross the line into belief.