Identify data type

Let’s start by choosing an activity or point in time where you collect data for some purpose, or rely on data to guide decisions or communicate key metrics. That situation requires the use of a particular type of data – numeric, textual, graphic, and so on.

Think about the data itself in that instance, and the form that data takes when it’s being looked at by people. I don’t mean what kind of chart or graph. That’s an artificial mapping. Think about the data’s pure form - is it numerical, graphical, auditory, tactile? Simple or complex? Is it static or dynamic? Historical, evolving, or real time? Knowing how to answer all those questions will help with the next part – transferring the essence of the data into a more usable form.

For example, you may need to analyze census data to figure out how to distribute funds to communities for better social programs. That data is typically in a spreadsheet with many columns of text and numbers that represent individuals who live in specific areas, who reported different levels of education and income.

That data’s form is columnar text and numbers. Pretty straightforward. Once you have isolated the form of the data you’re most interested in, it’s time to unleash it as a hologram.

Free the data

Now comes the interesting part (and the most difficult it turns out). We’re going to convert the existing data from its current form into a new holographic container that provides something you don’t get currently. The hope is that transformation reveals a leap forward in the ability to understand or work with the data. But, to do that, we have to free the data from its current form and container.

I’m not saying these answers will come easy, or that converting a traditional media form to a three-dimensional holographic representation is a straightforward thing to do. In fact, if we’re being completely honest here, it’s actually quite advanced on the design thinking and conceptualization scale. But, I know you’re up to it.

Sketch ideas – To answer some of things questions you’ll want to sketch out how the data transformation might play out.

Try sketching out a few quick transitions from current form to released

In my example, I took census data in columnar format and converted it into a map of the region that the data represented. Then I looked at what additional value we could add now that we’re working in multiple dimensions. That turned out to be using the z-axis or height of data points applied to the flat map. The new form of the information is not just a table of text and numbers in columns, but an easy to recognize map of the region with data points rising above the map plane representing income from the census data. Using more dimensions to overlay information is a prime method for realizing better recognition and understanding.

Figure out the scale - Another thing to consider is the scale or size of the new form. You can easily fall into the trap of thinking that the new holographic form of your data needs to be exactly like the original form you have isolated. Not true. In fact, that’s most often the exact wrong thing to do. Familiar is always a good foundation to build upon when transitioning someone to a new experience based on existing behaviors. That said, the whole point of envisioning our data in a new form is to literally release it from its current container and let it flow into an entirely new shape, of any size, to most optimally represent it in a more malleable and understandable form.

It would make sense to view US Census data in a map-like shape

You’ll find that often in holographic design it pays off to make the experience revolve around a room-scale set of holograms, using life-sized characters and accurate dimensions of objects. That’s not to say every experience has to be a 1:1 real-world match, but you’ll be surprised how the recognition and comfort with your experience goes up with things appear in their natural sizes. Wait, you say. Data has no real-world size – it’s just a bunch of numbers, right? No, actually those numbers represent things in the real world, or theoretical world, or alternate universe. No matter. Things have size. Think about using it to your advantage.

Match shape to its creator - Now that we have a rough idea of the shape of the new holographic container for our data, what type of tool creates that basic shape? Exactly. Nothing. No really, that’s not quite true. There’s probably something that can create the dimensional shape you are looking at in your sketch. Pretty certain that tool won’t export the shape directly as a hologram (few do), but you can probably figure out what type of app or tool can create that shape with particular parameters (size, shape, color, orientation, etc.). For example, I could use any search engine’s map tool to create a flat map of the areas my census data covers.

Identifying this data shape creation tool now will save us a ton of time later.

Avoid doing this - Now that you have the gist of how that works, let’s talk through a few things that might trip you up along the way to finding a new representation for your original form.

Now that we have a few different sketches of holographic forms that our data could be released into, it’s time to imagine how we might interact with them if they were right in front of us.

Choose data set

For each new experiment with a data-inspired hologram, we need to pick a specific data set to work with so our interactions are grounded and focused in something we can test and measure. If we were to use made up data, it would be hard to know if what we’re doing is any better or worse than expected.

Having some degree of familiarity with the data set we pick ensures we will realize when it becomes unrecognizable or distorted. By using a data set that matches the original data type from the ideation phase, we’ve aligned the container and its contents.

For example, I picked a spreadsheet containing the U.S. census data for a particular year (2012) covering the continental United States.

You aren’t limited to choosing a big data set or a data feed from service. Use your own sales data charts. Get a report of your credit card spending in spreadsheet form. Download the number of steps you’ve taken this month from your fitness app. Whatever works for you. Again, the key is being familiar with the data.

Choose a generator app

To create all of the variations of the new holographic data form, we need an app that generate the shape or something close to it without too much trouble.

The best kinds of apps to choose when doing this type of envisioning are those that allow fluid exploration and rapid creation of content. In our case, being able to generate the form that we think unlocks the insights or important aspects of the data.

Programs that let you do 3D modeling and animation are great for this task if you know how to use them efficiently. We need throughput, so don’t discount more consumer-grade software. Programs like Paint 3D, SketchUp, Blocks are all good choices. For the more advanced forms you want to bring to life try out things like Maya for complex combinations or Tilt Brush for the freedom of creating flowing shapes.

Since my example deals with census data and the regions it applies to, I choose an app that can do more than generate shapes. This kind of app specializes in dealing with data sets. SandDance, a Microsoft Garage project, is a world-class data exploration web service that allows you to load just about any data set and visualize it in many different ways. It also happens to be built into Microsoft’s flagship data product Power BI.

For my purposes it’s a perfect fit since it can plot data points on a grid that would resemble a map due to inclusion of latitude and longitude in each census data point.

SandDance, a Microsoft Garage project, proved to be a great form generation app

You don’t have to use anything like SandDance for your data form generator. For example, if the shape you were thinking about using is more like a person or character, use a 3D modeler that can pose figures in any position you need. Or conversely, if your form resembles architectural spaces or furniture, use something like SketchUp to quickly generate those forms.

Generate multiple examples

Now that we have a generator app picked out and are fairly comfortable flying it, time to start cranking out lots of experimental forms.

Generate at least one good example of what you’re trying to depict first. Examine it a bit critically to see if it embodies the key aspects we look for – clarity, impact, depth, ability to generate questions, and a newness that wasn’t there before.

To warm up, try adding slight variations on the basic shape, but then go radical and completely change one important element in each successive iteration to see if helps. Make copies of the base form so you can really try out big changes without worrying about losing a good path. Save lots of tries. You never know which one will be the best “take” when you look back over them.

Building on the basic form, generate several very different approaches

Elements to experiment with are the form’s shape, size, orientation, color, camera angle, lighting, overlays, or different data to fill it out. We’re looking for those moments where you know something has either locked in or gone too far in the wrong direction. Both help dial in the appearance and presentation we’re looking for.

This part of the process can be a time sink because of all this tinkering, so watch how much you invest. Speed is our friend, always.

Play out interactions

One of the best things about envisioning is the ability to figure out how things will behave without writing a single line of code. We can simply use our imaginations, or better yet, our teammates and coworkers, to help act out how these data-inspired holograms respond to our inquiries, actions, or even inaction.

Having just generated a bunch of different variations of our new data form, it’s time to see if this works for people. That requires figuring out the interaction basics and then engaging with people to try out our assumptions.

Using the Acting Out technique, let’s have the new holographic form we just created by played by a person instead of a pillow or inanimate object. By using a real person, we enable a much richer set of interactions and possibilities. We may even be inspired by the way they play the part, respond to stimulus, or ad lib a funny retort. So many subtle interactions come to light when the object we’re modeling is played by a person. And in this case, how funny that a new representation of data is brought to life by a living, breathing entity.

It doesn’t take much convincing to get people acting as holograms

Of all the things we get to do during envisioning, this step is the most fun. You always learn a great deal from being inside the interactions with real people, no question. Yet, the lasting impression from the phase is the laughter that inevitably comes with running this type of design. Give it a try. I guarantee it’ll become a standard part of your envisioning, no matter what the subject happens to be.

Package it up

Now that we have basic data form figured out and know roughly how it behaves, time to put it into a vehicle that can deliver the intended impact to our audience. The whole point of this is to get in in front of people that will (hopefully) see the value in using our new representation. We’ll have to think about the best to reach them in the natural flow of their work.

Consider the intent of this envisioning exercise when you choose the packaging. Think back on why we started doing this experiment in the first place. Was it to quickly convey a concept? Impress upon someone the need to unlock the data in this way? Show a technical proof it could be done in this way?

Quickest – Use the HoloScenes technique of combining the holographic form of the data with a photo of a physical space, like someone’s office or home. This is an easy way to depict exactly how you’d like to see this play out. You control the setup, camera angle, overlays on the data, and position of people in the scene for maximum impact. Downside is not being able to show the interactions in motion. They are implied, but left to interpretation.

Effective – Wireframing the scene and animating the key interactions can easily be done using PowerPoint or other presentation software. They have all the capabilities you need to drop in photos, drawing, models, and add your own markup over the top. By creating a clickable model that responds to someone’s touching things or issuing voice commands, you can simulate pretty well what you are going for. The downside is this technique is clearly more lofi than you’d optimally want for high impact. It will be seen for what it is – a quick and dirty interactive sketch.

Highest Impact – A slightly polished video production of the scene playing out with the interactions you figured out through acting is going to win people over. They may know right away you used slick video compositing to drop the holograms into the scene, but being able to show the richness of the presentation and interaction aspects provide a value that’s hard to get in other forms of envisioning. The downside is you need at least some rudimentary video editing and compositing skills on the team.

Most Impressive – Working prototypes using real data sets are without question going to make the biggest impression on people entirely because they are real. The old saying “Code wins” is apt here. Being able to demonstrate the concept with working models, data, and code will excite people’s imaginations and have them telling you exactly what needs to be better. Mission accomplished. Downside is this is expensive time-wise and requires resources to get working.

Show it around

This is where your narrative and storytelling skills come in. You could construct a good test for the data form where you do a before and after type of comparison.

Start with telling the story to someone about how you normally work with the type of data you choose originally. Show them how you’d normally consume or work with it. Quickly run through the difficulties. Conjecture about the opportunities to improve it. Then reveal the new data form. Stop talking.

What people should immediately jump to is how it can be improved or how it’s more clear, understandable, or actionable. What you hope to not hear is silence, denoting you missed the mark. Of course, hearing this doesn’t make sense is a good indicator you may want to try another one of your explorations out on them. Either way, the idea is moving forward.

Apply special effects

Now that we have the basics down, there are several things you can apply to the presentation vehicle you choose to show off the holographic data form in its best light.

Lighting – The key to any great rendering of 3D objects is how they are lit. Using convincing light takes away some of the resistance and lets the viewer focus on the substance. You can bring still frames or individual elements into Photoshop to touch them up quickly. Don’t have to use a full-blown 3D modeler to get this done.

Sound – More impactful than visuals in many ways, spatial sound (or any sound for that matter) will punch up the important moments in the consumption or interaction. Try using any sounds as placeholders just to get the timing right for its intended purpose. Then find the right one. Nothing is worse than using the wrong sound.

Atmosphere – One of the most visually interesting things to experiment with is adding cinematic effects to your holographic data forms. Placing the elements into a real-world scene is a way to convey the scale of the data it represents. Atmosphere like fog, rain, clouds, or bright sun illicit a particular feel that can work to your advantage. Experiment here with focus.

Affordances – There are typically widgets that we use to manipulate digital objects that sometimes are left as part of the surrounding chrome or interface of the programs we use. There are times when we can embed those within our holograms to assist with actions. The key is inserting them in natural and straightforward ways so they don’t district from the data and insights themselves. An example of that is a “datablade” that cuts through the data set to focus on one particular slice.

Find new data

The best way to prove whether you really hit on something valuable or even a breakthrough experience is to “reload” your new hologram with new data. Maybe it’s from a different year or completely different region. Perhaps you try to fit a whole new type of data into your container to see if it still holds up. Whatever you decide to try will help you learn more about the original idea you had for this particular form.

A great way to tell if you should dig deeper with this prototype is getting a new data set to almost work. That last bit that didn’t is your new area of exploration. Why was that? The form or data set? What if there was a ton of data that needed to be visualized and worked with in this way – would it work?

We can’t possibly anticipate all the different kinds of actual data people will want to use your holographic container for, so pushing ourselves to try data that’s non-obvious is always an eye-opening experience.