Visual images or visual understanding of what one is trying to do are definitely helpful … I would say understanding is achieved and results come more readily if one has a picture rather than by looking at a lot of formulas.

Morris Kline,

The Creative Experience (1970)

THE IMPORTANCE OF FLOW

Another important theme that has pervaded our work is something that has come to be called flow (Mihaly Csikszentmihalyi, 1990). Over the years we have developed an intuition about what makes information visualization (and other) interfaces work well, and we have discovered congruence between these ideas and the concept of flow, an idea from the psychology literature. Though we don’t have a strict formula for a successful interface, we know that a few basic approaches do help. In general, we believe it is important that the users stay in control and that the computer offers choices with appropriate feedback for user actions. Conversely, computer-controlled interactions often lead to unpredictable, and therefore unacceptable, interfaces.

We also have learned that people are primarily interested in focusing on their tasks and not on operating the interface —and yet so much of a user’s experience with a computer is manipulating widgets, resizing windows, and selecting from menus. It is crucial that computers give users prompt and informative feedback at every step along the way. Finally, users must stay engaged in the task for their experience to be effective in the long run. This means that the interface must not be too complex or confusing as to alienate users, nor so simplistic or condescending as to make them bored.

A computer interface that strikes the right balance can enable users to concentrate on the task at hand. The computer becomes a “tool” in the best sense of the word—an extension of the user’s body. Time passes quickly, and the users develop a sense of control and confidence while making progress toward their goals.

When people experience this kind of focus, they sometimes refer to “being in the flow.” Some psychologists refer to this as optimal experience, a shorthand that describes the best experience that one can hope for.^* And though it may first seem far afield from computer work in information visualization, as researchers let us consider it our ideal: to create computer interfaces that enable users to forget they are using a computer and think only of the important work they are accomplishing.

This book is about that process in innovation during the last ten of the lab’s twenty years as we concentrated on the field of information visualization, a subfield of HCI.

EVALUATING OUR WORK

How do we assess our progress? Are we any nearer to our goal of creating interfaces that support flow? Tough internal assessments—critiquing each other, challenging assumptions, and demanding evidence help prevent us from falling into traps of wishful thinking.

External reviews from colleagues add to our continuing assessments. We send drafts of papers to colleagues, invite visitors to see our work in progress, and engage potential users to try our software. We appreciate good feedback, taking to heart constructive comments that push us to refine our work.

The next level of assessment comes from anonymous reviewers of conference papers, journal articles, and grant proposals. We discuss rejections and try to learn from them. Even when we disagree with reviewers, we try to examine how we might have told the story more effectively. Some of our strongest papers have been shaped by tough reviewing processes.

Published papers are the clearest signs of our progress—they have been validated by peer review, and they are publicly available. Several members of our group appear high on the list of authors ordered by frequency of publication in HCI papers, conference presentations, and books (www.hcibib.org/authors.html).

Another imperfect but useful metric of success is the number of references to a paper. The NEC Citeseer (citeseer.nj.nec.com/directory.html) index has a special section on human–computer interaction, and we are proud that the most cited paper for many years has been one of our works on information visualization. Similarly, when PARC researchers studied reference patterns in information visualization, they found that their group’s papers were cited most frequently, but our lab came in second.

Citations in academic papers are one manifestation of our influence, but downloading our software is also quite validating. More than 30,000 individuals have downloaded PhotoMesa, one of our image browsers (Chapter 2).

Our software also influences commercial and government applications. Many of our ideas have become part of larger success stories, such as SmartMoney’s MarketMap (www.smartmoney.com/marketmap), the Hive Group’s treemaps (www.hivegroup.com) (Chapter 6), and Spotfire’s (www.spotfire.com) visualization tools (Chapter 1). Contributions to important national and international projects include the Visible Human Explorer for the U.S. National Library of Medicine, NASA’s Earth Science Information Partnerships, and the Library of Congress’s American Memory Web site.

Finally, an important internal measure of the HCIL’s success is the frequency with which our students graduate and join companies, universities, or government agencies where they make valuable contributions. It’s especially satisfying to see young, often shy or quiet students become self-confident professionals who are valued by employers and respected by colleagues. As a community, we are gratified when former students return to tell us how much their time at the the HCIL influenced them both professionally and personally.

WORKING WITHIN A BROADER COMMUNITY OF SCIENTISTS

It is difficult to rank or even list all the people in our professional networks, so we must begin with an apology to anyone we have left out in this discussion. Those who want completeness can examine the hundreds of references in our papers. However, we cannot honestly review our work without reflecting on the influences of our colleagues. In several sections, we include more details related to that topic, but this opening mentions a few of the major groups whose influence cuts across many of the sections.

Our strongest and most enduring bonds have been with the community of researchers at the Xerox Palo Alto Research Center, now simply PARC (www.parc.com). Our contacts have been mostly with the user interface group and its related teams, especially the manager and long-term researcher, Stu Card. Stu is a leader in theory-driven thinking and research and is a remarkable innovator, as testified by his numerous patents and papers. The PARC group has also included key people such as Jock Mackinlay, George Robertson (now at Microsoft Research), Ramana Rao (now at Inxight), Peter Pirolli, Mark Stefik, and many others.

As Microsoft Research grew, we enlarged our contacts with George Robertson, Mary Czerwinski, and others. Other industry groups include those at AT&T-Bell Labs and spinoff groups, including Stephen Eick, Andreas Buja, and Stephen North. We have also enjoyed long-running interactions with Clare-Marie Karat and John Karat at IBM. Special mention goes to Nahum Gershon of Mitre, who has been an effective champion and organizer for information visualization conferences and journals.

University colleagues include Steven Roth at Carnegie-Mellon University, Steve Feiner at Columbia University, George Furnas at University of Michigan, John Stasko and Jim Foley at Georgia Tech, Andries Van Dam at Brown University, Jim Hollan now at UCSD, Saul Greenberg at University of Calgary, Robert Spence at Imperial College, Keith Andrew at the University of Graz, and Alfred Inselberg at Tel Aviv University. Another special mention goes to Edward Tufte at Yale University, who is well known for his independently published books (1983, 1990, 1997) and for his annual public lecture tour—we regularly pay for students to attend when he swings through the Washington, D.C. area.

There are many others, but these people form the core of our community. We jointly write books and articles, organize conferences, and participate in workshops—all to promote information visualization to broader circles. Seeing each other for a beer or dinner once a year is important, and the continuity of contact is maintained by email. We tell our latest stories, probe for their new ideas, and seek each other’s respect. These colleagues are who we turn to validate our innovations, to ask for reviews of our draft papers, and to be our partners in proposals.

THE MARYLAND WAY FOR INFORMATION VISUALIZATION

Occasionally, visitors and colleagues who appreciate our accomplishments will ask how we go about our work. In Sparks of Innovation (1993), Ben Shneiderman described what he called the Maryland Way. It has remained a useful guide. Of course, we’ve learned some new lessons, the field of HCI has matured, and the lab has grown. So we’d like to revisit those ideas in the narrower context of information visualization.

We begin by choosing motivated, strong researchers who will interact well with others. Then, the Maryland Way is to foster innovation through these seven steps.

1 Choose a Good Driving Problem

Fred Brooks’s advice to choose a good driving problem is especially relevant for information visualization, given its strong practical component. It helps enormously to have a clear goal; for example, design a video library that consumers can browse, or build a photo library program that three-year-olds can navigate. Finding good problems is like antique hunting: you are not quite sure what you want, but when you see it, you know it. In the early stages of choosing a problem, we brainstorm to come up with alternatives. Then, over a period of a few weeks, we discard the extreme ideas, refine the remaining possibilities, and focus on one.

Our favorite problems entail improving designs for a wide range of users in real-world contexts—building interfaces for museum users exploring historical topics; library patrons searching for a book or document; scientific researchers trying to understand gene expression levels; or business analysts seeking patterns in customer behavior.

2 Become Immersed in Related Work

We expect each of our students to become the “world’s leading expert” on the problem he or she is investigating. Our students must acquaint themselves with related studies, sample similar commercial products, and personally contact active researchers. We expect our students to educate us about related work.

Our students must go to the library or the Internet and chase down every reference to their topic. This process has the dual benefits of compelling them to work on something narrow enough that they can become the leading experts and forcing them to clarify exactly what they are working on.

Trying out commercial products brings a sense of practical reality. The students come to understand the parts in the context of the whole and to see the tradeoffs that designers must make.

Getting in touch with current researchers or developers is a novel and threatening task for many students. Email helps facilitate the process, but phone calls, letters, and visits are also important. Shy students overcome their awkwardness and are often rewarded by a helping hand from a respected researcher or an invitation to present their work at a major company.

3 Clarify Short-Term and Long-Term Goals

After the brainstorming process (see Step 1), which sharpens our understanding of the project, we establish long- and short-term goals. Long-term goals provide a destination and a shared set of expectations that focuses effort. Short-term goals provide immediate feedback about progress and a chance to make inexpensive midcourse corrections.

4 Balance Individual and Group Interests

We give each student or staff person a clear role that serves his or her individual goals (e.g., getting a master’s degree within 18 months, doing an independent study summer project, or building a resume to get a desired job). Individual goals need to be in harmony with the overall goals and directions of the lab. A student who wants to do a master’s thesis on a topic that is poorly related to our existing work will be encouraged to consider alternative topics.

When visitors tour the lab, students and staff show their work, get feedback, and promote their ideas. Our visitors often prefer chatting with the students who are doing the work to attending a private presentation by senior staff. When visitors are potential funders, direct student and staff involvement in the future of research projects increases motivation, participation, and work quality.

When individual and group goals are in harmony, fortuitous collaborations are likely. One of the ways we have been able to accomplish so much with limited resources is that individuals help each other. When one Ph.D. student needed a special routine on an unfamiliar hardware and software environment, another student stepped in and provided a few days of programming help. The favor was repaid by help in reviewing paper drafts and in preparing subjects for an experiment. Since our lab operates with a diverse hardware and software environment, hardly an hour goes by without someone calling out for help on some system.

5 Work Hard

Thomas Edison remarked that innovation requires 1 percent inspiration and 99 percent perspiration. An exciting and novel idea is just the starting point. Most ideas have a cascade of smaller ideas behind them and details to be worked out. Special cases, exceptions, and extreme conditions must be be investigated carefully to reveal the limitations of a new idea. Then converting an idea to a piece of functioning software, a set of screen designs of a prototype, or the materials, tasks, and statistics in an experiment takes devoted effort. Polishing, refining, and cleaning up can take ten or a hundred times more effort than the original innovation. Simply expecting things to take a great deal of effort removes some of the anxiety or expectation of perfection.

There is a definite improvement in quality when you can revise a project after reflection or comments from colleagues. The second time through almost any process or path is often smoother and faster. The “third-time charm” for experiments or designs suggests that persistence and repeated tries leads to excellence.

6 Communicate with Internal and External Stakeholders

Our group operates with a high degree of internal communication and external reporting. Internally, research teams working on related topics meet frequently. We also hold weekly seminars to discuss journal or conference papers or to hear formal presentations of results. Even more compelling than these traditional meetings, however, are the spontaneous demos, informal pre-experiment reviews, participation in pilot studies, pleas for help with statistics, and personal requests for reading draft papers.

While internal communication helps form and guide our work, external communications increase our intensity as we prepare for demonstrations to visitors; presentations at our annual Symposium & Open House; writing reports, theses, or journal articles; production of videotape reports; lectures at companies or universities; and papers and sessions at conferences.

Preparing a presentation for a friend, staff person, or professor may encourage some diligent effort, but it seems that preparation for a conference talk, a lecture to supporting companies, or important visitors raises the stakes considerably. Telling the story and listening for feedback are often unfamiliar skills to technically oriented people, so we try to practice often.

7 Get Past Failures. Celebrate Successes!

Many days seem filled with hundreds of responsibilities such as reviewing journal papers, showing visitors around, responding to requests for technical reports, writing proposals, or reading a draft of a thesis chapter. We are sometimes burdened with filling out travel vouchers, repairing computers, or preparing budgets; however, when it comes time to write annual reports or prepare for our Symposium & Open House, we are struck by how much we have accomplished during the previous year.

The good days are when students invite us to see a demonstration of their latest design, improvement, or experiment. As lab members gather around a computer display, we cheer, comment, and criticize. Other memorable days include working intensely to finish a paper, resolving a problem with statistics, brainstorming on designs, rehearsing for a videotape, fantasizing future user interfaces, and especially celebrating a student’s successful dissertaiton defense or journal submission.

Even a successful research group must deal with disappointments. After many years of writing, it is still disappointing to be turned down by a conference program committee or journal editorial board. Requests from journal editors for major revisions are also unpleasant, but some of our most successful papers have had the longest gestation periods and endured the most revisions. We have had our share of rejected grant applications, students who choose to go elsewhere, and funders who decide not to renew.

However, careful acknowledgement of contributors, reviewers, and supporters of all kinds helps to keep such disappointing events to a minimum. In addition, we avoid much internal strife by discussing author credits early and often and seeking creative ways to resolve conflicts.

The HCIL’s annual Symposium & Open House (Figure 0.1) is a major celebration in which students, staff, and faculty present their work to several hundred attending. In the morning, we make formal presentations and respond to questions. The afternoon is given over to tours, demonstrations, and personal discussions. At the end of the day, senior staff and faculty dine with the advisory board to reflect on the day and seek suggestions for future work.

In addition to the symposium, the HCIL holds an annual all-day retreat at some bucolic location within an hour’s drive of the lab (Figure 0.2). We discuss our research directions, envision the big picture, and make resolutions in a safe and supportive environment. There is never enough time to discuss every project, but long lists are made for later contemplation.

CONCLUSION AND FUTURE DIRECTIONS

Through these processes the HCIL has continued for over 20 years to focus on topics that put humans at the center of technology. This book tells the story of the HCIL’s work in information visualization over the past decade. The selected papers which exhibit our lab’s most important outcomes, show our work process and evolution of ideas from one project to the next. Each chapter starts with our reflections on the people and problems that inspired the work, followed by the papers in chronological order (except Chapter 7). We’ve also included short lists of favorite papers from outside our lab that were the most relevant and influential. The listing of all technical reports published by the HCIL in the last 10 years is presented (in reverse chronological order) in Appendix D

As a research topic, the field of information visualization is still forming with a growing number of university courses, professional conferences, and scientific journals. The central research problems include perceptual psychology issues such as understanding change blindness, choosing color palettes, showing relationships between nonproximal items, and using retinal properties (color, size, shape, etc.) properly. Interface design research topics build on perceptual issues for presenting information, providing user control widgets, and using animation effectively. There is also a need for traditional computer science topics such as algorithms for rapid search, data structures for compact storage, software architectures for efficient implementation, and modular programs to facilitate collaborative development. New research methodologies are needed to improve user-needs assessments, controlled experimentation, and ethnographic observations.

Central problems for commercial developers of information visualization tools include data integration to smoothly import data, data cleansing to remove or repair bad inputs, and data export to send result sets to other users in formats that will be acceptable. Once these basic problems are solved, commercial developers will succeed in crossing the chasm (to use Geoffrey Moore’s term) if they can provide a whole product solution for a genuine need.

When developers solve problems for their customers, information visualization products will move from “nice-to-have” to “must-have.” Industries likely to be the early adopters are those driven by continuous innovation and repeated discovery, including pharmaceutical drug research, oil-gas exploration, financial analysis, and manufacturing quality control. Other candidate adopters are transportation safety analysts, business fraud detectors, crime or terror investigators, and medical diagnosticians.

Researchers and product developers will have to cooperate in a massive educational process to teach potential users about suitable applications and appropriate visualizations. This process may take decades, just as it did for the move to graphical user interfaces. Collaboration with data mining enthusiasts, statisticians, information technology specialists, software engineers, business analysts, and other professionals will accelerate this adoption process.

BOOK REFERENCES