Video Envisionment
Play is the highest form of research.
By now we have encountered a number of examples where video has played a role in mocking up, sketching, and capturing various design ideas. There is a subspecies of this genre that I want to spend some time discussing, for a number of reasons. It is what can best be called Envisionment Videos. As their name suggests, these are clips that are created to communicate some holistic view showing an envisioned system in context. Unlike many of the examples that we have seen, they go beyond just presenting a demo. Rather, they usually are built around a narrative that tries to capture a way of working with technology, as opposed to the design specifics of the device itself. Actually, Sketch-a-Move could well be considered an example. Overall, they are very much part of the story-telling tradition of design.
The Xerox 895X Matter Duplicator
Let me start by giving you a really simple example that was made in a workshop that I did in Cambridge England in 1988. It is a simple and playful tape called The Matter Duplicator by Randy Smith. The video gives a “live” demonstration of the Xerox 895X Matter Duplicator, frames of which are shown in Figure 137.
Randy demonstrates how one makes a copy of a screwdriver simply by placing an original in the input hopper of the 895X copier (which bears an uncanny resemblance to a regular photocopy machine), and pushing the green copy button on the console. Sure enough, after he has done this, he is able to retrieve an exact copy of the original from the output tray.
Of course, the tape’s style and content ensure that nobody will ever take it seriously as a demonstration. I include it here for three reasons. First, I just think that it is funny, and thought that you might enjoy it, despite the poor technical quality. Second, it is a nice and short introduction to the genre. Third, it is a simple but effective example of the type of thing that one can do in a short workshop to learn how to make such things—from both sides of the camera. This last point is not to be taken lightly. It is doing such simple “finger exercises” that develops the technique that lets one pull off what we saw Bob Spence do in The Bifocal Display video.
Figure 137: The Matter Duplicator
This short video gives a demonstration of the Xerox Matter Duplicator, a product that lives only in the warped mind of my friend and one-time colleague Randy Smith. Here he demonstrates that a screwdriver placed in the input hopper can be copied as simply as you copy paper documents today.
Office of the Professional
Having just mentioned Bob Spence, let me use another video of his as my next example. Office of the Professional is the first example of envisionment videos that I am aware of. It was made at Imperial College, London, in 1980 by Mark Apperley and Bob. Figure 138 is a frame from one of the scenes in the video.
The tape consists of a number of short scenarios built around the types of things that a professor does in the normal course of events. These include interacting with colleagues, examining experimental data, looking at student records, making travel arrangements, and filing information. What the video brings to the equation is a cinematic manifestation of how future technology might be integrated into our environment and lives so as to facilitate the performance of such tasks.
It is interesting to view such a video more than 15 years after it was made. For example, there are curious anachronisms, such as the circa 1980 telephones, references to sending a TELEX, and an Apple ][ computer, all in the midst of scenarios involving now current interaction techniques such as speech recognition, pen-based input, touch screens and tablets, remote pointing onto the wall, and advanced graphical visualization.
But such technological inconsistencies are, and were, not the point. The video made no pretence of predicting the future (although it was remarkable for its insights, it is worth remembering that it was done more than 10 years before Mark Weiser published his classic 1991 paper on ubiquitous computing and two years before the launch of the Apple Macintosh).
Significantly, the real purposes for making the tape are clearly articulated in the commentary that serves as a kind of coda at the end. Key among these were:
To show the bifocal display in context: This in itself is an example of why I have distinguished envisionment videos from, shall we say, demonstration videos. These may not be the best terms, but contrast the form and content of The Bifocal Display video and Office of the Professional. The former went deep into a particular problem and worked through a possible solution. The latter paints a much broader context, physical and social, into which the bifocal display can be placed. We learn far less about the display’s characteristics in it. But on the other hand, we gain a deeper perspective of why we might want to, or how it might fit in with the larger social and physical ecology in which it might find itself. Bob and Mark understood this, even then, when they were pioneering the use of video. A lot of us could still benefit by learning this lesson.
Demonstrate the use of intuitive skills rather than taught skills: This is also really telling in that it is centred on the human user, not the technology. In many ways this presages an attitude not popularized until the publication of books such as Norman … Draper’s User Centered System Design (1986) and Norman’s Psychology of Everyday Things (1988). The notion here was to communicate, by example, how future systems could be designed so as to exploit existing everyday skills, rather than demand the acquisition of fundamentally new skills, and incurring the attendant costs.
To capture a particular view of the future: Yet keeping in mind, as Bob Spence says on the video, that what might seem futuristic today “has the habit of coming next Wednesday.” This harkens back to something that we discussed much earlier in the book. Even when designing for the immediate future, we must do our best to anticipate the future into which our design has to evolve. I characterize this as walking with one’s head in the clouds and feet deeply entrenched in the mud.
Provide food for thought for those doing research and designing tomorrow’s systems: The video is clearly envisionment, and has the production values of a concept rendering, or sketch. Nobody would confuse this with product, or a real system. This speaks to one of the prime attributes that we have associated with sketching, namely being a critical component in a conversation, by both what is shown, and what is not. This video is not telling us about the future. Rather, it is inviting us to have a conversation about it, and giving us a good place to start.
If it is not clear already, I really like this video, and I have a lot of respect for those who made it. It is a good example of the importance of knowing our history and traditions. Even today there is about as much to learn from this video as there was when it was made.
Figure 138 (Following Spreads): Office of the Professional
The first envisionment video that I ever saw.
Source: Apperly … Spence (1980)
The Knowledge Navigator
The next example that I want to talk about is a marked contrast to Office of the Professional. It is a video called The Knowledge Navigator. It was made in 1987 by Hugh Dubberley and Doris Mitch for a presentation by the then president of Apple Computer, John Sculley.
It is the contrast between it and Office of the Professional that led me to include it. But first, by way of background, take a look at the accompanying sidebar, where Hugh Dubberley gives a first person account of the making of the video (See Figure 139.)
The Knowledge Navigator is certainly an envisionment video, but it is equally certainly not a sketch. The first hint, even without seeing it, is its budget and production schedule, as outlined by Hugh. Yes, they were pressed for time for what they were doing. But they were also not doing something that was quick, timely, plentiful, or disposable. This was a video that was created in the visual vocabulary and with the production values of the TV commercial or, as Hugh describes it, a science fiction film. And, as such, they did a good job.
On the other hand, this, and many of the similarly produced films that it spawned, also gave rise to some real issues, some of which are hinted at in Hugh’s account. Remember what I said earlier about good stories being like a kind of viral marketing—they get retold over and over? Well, so it was with The Knowledge Navigator. None of us had ever seen anything like it before, and everyone wanted a copy to show. But when this happened Apple lost control. Out of the context of Sculley’s presentation the visual vocabulary and production values were too persuasive: people being people inevitably started to believe that this was a system that Apple was actively working on. Not way down the road as a vision, but right now. “Why else would they spend all that money on this and tell us this story if that were not the case?” they might ask. And if you actually persuaded someone that it was not something in the works, that almost made things worse. Raising expectations that high and then not delivering meant that there was all the further for them to fall.
I included this example as a reminder of what we said much earlier about the properties of a sketch:
Clear Vocabulary: It must be distinguishable from other types of renderings.
Minimal Detail: It must include only what is required to render the intended purpose or concept. Superfluous detail must not distract attention to the sketch’s purpose.
Appropriate Degree of Refinement: A sketch should not suggest a level of refinement beyond that of the project being depicted.
Even if The Knowledge Navigator had been quick, timely, inexpensive and disposable, it still would not work as a sketch. It was about telling, not asking, and what it said, it said with conviction. And it was too vulnerable to misinterpretation—the wrong kind of ambiguity. The Knowledge Navigator is an extreme case, but hopefully that makes the point all the easier to see. The thing to take away from this example, besides an important part of our history, is a vigilance in deciding upon the visual vocabulary and types of production values that you use in your own work, as well as a more critical eye in terms of how you read the work of others.
Seamless Media Design—Clearboard III
I want to conclude this thread of our discussion with one final example. I include it because it caused me problems. In many ways, it seems to break the very principles that I just spoke about.
The example is an envisionment of a future incarnation of a shared electronic drawing board. It presents a concept called Clearboard, first developed at NTT Labs in Japan by Ishii … Kobayashi (1992). What Ishii and Kobayashi did was build a drawing surface that two people could draw upon at the same time, one on each side of the surface, such that they could see each other through their common drawing. A frame from the envisionment video illustrates this in Figure 140.
So what is the problem? Well, in watching the video, it seems to me that it may well have cost significantly more to make than The Knowledge Navigator. The production values say anything but, “This is cheap, fast, plentiful, and disposable.” Just the opposite. So why is it when you watch it, it does not conjure up memories of TV commercials past? Why is it that I never once heard anyone suggest in person or in writing that this was a representation of something that NTT was making as a product?
These are not idle questions. If we are going to use the language of video and film in our work, then it is really important that we think about such things and develop our literacy to the point where we are alert and vigilant enough to both ask the questions and be able to sustain a reasonable discussion that helps us gain some insights about them.
In this, I am not suggesting that this video is a sketch. Far from it. It is an envisionment, it is not part of a TV show, and not a TV commercial or product announcement. Beyond that, it is may not matter too much about what it is. It is just important that we know what it is not.
For me, the reason that this video does its job so well is rooted in the role of stylization that is so much a part of ancient Greek and Noh theatre. I leave it to you to watch Office of the Professional, The Knowledge Navigator, and Seamless Media Design, to come to your own insights about the relationship between content, intent, visual style, and production values. I just want to prompt the ensuing conversation.
Figure 139 (following spread): Sketches of Frames from the Knowledge Navigator
Both the video and numerous still frames from it can readily be found on the Internet.
The Making of the Knowledge Navigator
by Hugh Dubberley
We made the Knowledge Navigator video for a keynote speech that John Sculley gave at Educom (the premier college computer tradeshow and an important event in a large market for Apple). Bud Colligan who was then running higher-education marketingg at Apple asked us to meet with John about the speech. John explained he would show a couple examples of student projects using commercially available software simulation packages and a couple university research projects Apple was funding. He wanted three steps:
He asked us to suggest some ideas. We suggested a couple of approaches including a short “science-fiction video.” John chose the video. Working with Mike Liebhold (a researcher in Apple’s Advanced Technologies Group) and Bud, we came up with a list of key technologies to illustrate in the video, e. g., networked collaboration and shared simulations, intelligent agents, integrated multi-media and hypertext. John then highlighted these technologies in his speech.
We had about 6 weeks to write, shoot, and edit the video—and a budget of about $60,000 for production. We began with as much research as we could do in a few days. We talked with Aaron Marcus and Paul Saffo. Stewart Brand’s book on the “Media Lab” was also a source—as well as earlier visits to the Architecture Machine Group. We also read William Gibson’s “Neuromancer” and Vernor Vinge’s “True Names.” At Apple, Alan Kay, who was then an Apple Fellow, provided advice. Most of the technical and conceptual input came from Mike Liebhold. We collaborated with Gavin Ivester in Apple’s Product Design group who designed the “device” and had a wooden model built in little more than a week. Doris Mitch who worked in my group wrote the script. Randy Field directed the video, and the Kenwood Group handled production.
The project had three management approval steps:
It moved quickly from script to shooting without a full storyboard—largely because we didn’t have time to make one. The only roughs were a few Polaroid snapshots of the location, two sketches showing camera position and movement, and a few sketches of the screen. We showed up on location very early and shot for more than 12 hours. (Completing the shoot within one day was necessary to stay within budget.) The computer screens were developed over a few days on a video paint box. (This was before Photoshop.)
The video form suggested the talking agent as a way to advance the “story” and explain what the professor was doing. Without the talking agent, the professor would be silent and pointing mysteriously at a screen. We thought people would immediately understand that the piece was science fiction because the computer agent converses with the professor—something that only happened in Star Trek or Star Wars.
What is surprising is that this piece took on a life of its own. It spawned half dozen or more sequels within Apple, and several other companies made similar pieces. …These pieces were marketing materials. They supported the sale of computers by suggesting that a company making them has a plan for the future. They were not inventing new interface ideas. (The production cycles didn’t allow time for that.) Instead, they were about visualizing existing ideas—and pulling many of them together into a reasonably coherent environment and scenario of use. A short while into the process of making these videos, Alan Kay said, “The main question here is not is this technology probable but is this the way we want to use technology?” One effect of the video was engendering a discussion (both inside Apple and outside) about what computers should be like.
On another level, the videos became a sort of management tool. They suggested that Apple had a vision of the future, and they promoted a popular internal myth that the company was “inventing the future.”
Rendering Style and Process: A Brief Case Study
In 2005 I was in Rome giving a talk. While there I met a young Italian designer, Lucia Terrenghi. She was a PhD student studying at the University of Munich with Andreas Butz. She showed me a video, several frames of which are shown in the left-most column. It struck me as interesting, but I also felt that it suffered because of the style in which it was rendered. I suggested that it would be interesting if she redid it using a sketch style of rendering. We talked a bit, said goodbye, and I thought little more of it. Then, about 2 weeks later I got an email with the original video as well as the remake, matching frames of which are shown in the column to the right of the previous one. I was as impressed as I was interested. And as it provides a good opportunity for you to compare the impact of rending the same material different ways, I have included this example, along with its story, as told by Lucia and Andreas.
Lucia: My PhD thesis deals with using the hands to directly manipulate information on large interactive surfaces. On a train 1 came up with the idea of using physical objects and their affordances as a metaphor. For example, a mug could be a ubiquitous container of information from which information, like a fluid, could be stored, and poured in or out of. While on the train I sketched the concept on a piece of paper and then on the whiteboard during a team meeting the next day. We agreed that the idea had potential and opened up several other possibilities. Consequently I sketched a more detailed storyboard and asked Sebastian (a student working with us) to make an interactive animation using Flash. Sebastian interpreted my sketches and gave a realistic visual appearance to the items. When Andreas saw the flash animation though, he was not completely satisfied with the result.
Andreas: When I saw the Flash demo, I liked it for the ideas that were made more explicit, and for the way in which it was able to communicate the basic concepts and functionality. Nevertheless, something didn’t feel right. The student who had implemented the demo had interpreted Lucia’s concepts in his own way and I was somewhat disturbed by the varying level of detail among different visual elements. While the mug was rendered almost photographic with reflections and shading, the menu backgrounds were flat and in a solid color. The kitchen sink in the left hand menu was an actual photo, but the laptop in the same menu was drawn in rectangular shapes. We discussed this and I told Lucia that somehow I had liked her original sketches on the whiteboard better. This was because they were somehow more inspiring and open.
I suggested redrawing all visual elements in a coherent style to get rid of the disturbing inconsistencies. However, we wanted to show the work at a workshop, and there was little time. So instead of changing the rendering, I suggested that the time would be better spent making a movie where Lucia simulated the interaction as if it was on a large wall-mounted display. It struck me that it would be more effective than simply showing it with a mouse using a conventional PC. So the time was spent doing this using the existing version of the Flash file.
Lucia: I presented the same movie during the Doctoral Colloquium at INTERACT’05 Conference in Rome. I also showed it to Bill, whom I met there. He immediately suggested remaking exactly the same movie, but using a hand-sketched representation instead of the existing realistic one. I recalled Andreas’ comment about the sketches on the whiteboard and told Bill this was an interesting “coincidence” - that after seeing the original Flash animation, my supervisor had had a kind of “nostalgia” for the earlier whiteboard sketch. This probably reinforced Bill’s (and my) intuition that there was something about the style of representation that was worth exploring. Hence, we wanted to make the second movie so that we could compare them. Back in Munich I sketched the elements of the interface on paper, scanned them, and then replaced the symbols in the existing Flash file. We then re-shot the movie with exactly the same storyboard as the previous one.
Significantly, many people to whom we showed both versions assumed that the sketch version was the older, and that the more realistic version a remake. While this was not true for the actual movies, it was true when thinking of the original sketches on the whiteboard.
Figure 141: Paper Sketches of Oscilloscope Front Panel
Two frames from a videotape made during usability testing of oscilloscope front panel concepts. The usability of the numerical keypad is being tested. In each version, I have circled the location of the keypad.
Image: Tektronix Corp.
Figure 142: Interacting with Paper Front Panel
This is a frame from the same video. In this image, and the second one in Figure 141, notice how the hand gesture used by the test user is exactly what would be used with the physical control represented. This is a subtle but important point that illustrates the power of the affordances of the controls, even on paper.
Image: Tektronix Corp.