Chapter 3

Contextual Inquiry
Eliciting Work Activity Data

 I don’t build a house without predicting the end of the present social order. Every building is a missionary1… It’s their duty to understand, to appreciate, and conform insofar as possible to the idea of the house. (Lubow, 2009)

– Frank Lloyd Wright, 1938

Objectives

After reading this chapter, you will:

1. Understand the concepts of work, work practice, and work domain

2. Understand the need to study users’ work activities in the context of their work practice

3. Be prepared to write a clear and concise product or system concept statement for your envisioned system

4. Know how to prepare for undertaking user research activities

5. Be ready to conduct user research by meeting with customers and potential users to gather contextual data

6. Understand the history and roots of contextual inquiry

7. Appreciate the difference between data-driven and model-driven inquiry

3.1 Introduction

3.1.1 You Are Here

We begin each process chapter with a “you are here” picture of the chapter topic in the context of our overall Wheel lifecycle template; see Figure 3-1. The process begins with understanding user work and needs by “getting your nose in the customer’s tent.” To understand the users’ activities in the context of their current work practice (or play practice), using any currently existing system or product, we do contextual inquiry (this chapter) and contextual analysis (Chapter 4). Sometimes contextual inquiry and contextual analysis are collectively called contextual studies or “user research.”

Work

Work is the set of activities that people undertake to accomplish goals. Some of these activities involve system or product usage. This concept includes play, if play, rather than work per se, is the goal of the user.

Work Domain

The entire context of work and work practice in the target enterprise or other target usage environment.

Work Practice

Work practice is the pattern of established actions, approaches, routines, conventions, and procedures followed and observed in the customary performance of a particular job to carry out the operations of an enterprise. Work practice often involves learned skills, decision making, and physical actions and can be based on tradition, ritualized and habituated.

Work Activity

A work activity is comprised of sensory, cognitive, and physical actions made by users in the course of carrying out the work practice.

Contextual Inquiry

Contextual inquiry is an early system or product UX lifecycle activity to gather detailed descriptions of customer or user work practice for the purpose of understanding work activities and underlying rationale. The goal of contextual inquiry is to improve work practice and construct and/or improve system designs to support it. Contextual inquiry includes both interviews of customers and users and observations of work practice occurring in its real-world context.

image

Figure 3-1 You are here; in the contextual inquiry chapter, within understanding user work and needs in the context of the overall Wheel lifecycle template.

3.1.2 A True Story

In southwest Virginia, somewhat remote from urban centers, when the first-time computer-based touchscreen voting machines were used, we heard that quite a few voters had difficulty in using them. Although an official gave instructions as people entered one particular voting area, a school gymnasium, he did it in a confusing way.

One of the voters in line was an elderly woman with poor eyesight, obvious from her thick eyeglasses. As she entered the voting booth, one could just imagine her leaning her head down very close to the screen, struggling to read the words, even though the font was reasonably large.

Her voice was heard floating above her voting booth, as she gave some unsolicited user feedback. She was saying that she had trouble distinguishing the colors (the screen was in primary colors: red, green, and blue). A member of another major gender nearby said aloud to himself, as if to answer the woman, that he thought there was an option to set the screen to black and white. But oddly, no one actually told this, if it was true, to the woman.

In time, the woman emerged with a huge smile, proclaiming victory over the evil machine. She then immediately wanted to tell everyone how the design should be improved. Remember, this is an elderly woman who probably knew nothing about technology or user experience but who is quite naturally willing to offer valuable user feedback.

It was easy to imagine a scenario in which the supervisors of the voting process quickly flocked about the voter and duly took notes, pledging to pass this important information on to the higher-ups who could influence the next design. But as you might guess, she was roundly humored and ignored. Call us superstitious, but that is just bad UX ju ju.

There are a few things to note in this story. First, the feedback was rich, detailed, and informative about design. This level of feedback was possible only because it occurred in real usage and real context.

Second, this woman represented a particular type of user belonging to a specific age group and had some associated visual limitations. She was also naturally articulate in describing her usage experience, which is somewhat uncommon in public situations.

So what does this have to do with contextual inquiry? If you do contextual inquiry in a real environment like this, you might get lucky and find rich user data. It is certain however that, if you do not do contextual inquiry, you will never get this kind of information about situated usage.

3.1.3 Understanding Other People’s Work Practice

This chapter is where you collect data about the work domain and user’s work activities. This is not about “requirements” in the traditional sense but is about the difficult task of understanding user’s work in context and understanding what it would take in a system design to support and improve the user’s work practice and work effectiveness.

Why should a team whose goal is to design a new system for a customer be all that interested in work practice? The answer is that you want to be able to create a design that is a fit for the work process, which may not be the same as what the designers think will fit.

So, if you must understand something about what the users do, why not just ask them? Who knows their work better than the users themselves? Many customers, including those in large and complex organizations, may wonder why you want to look at their work. “Just ask us anything about it; we have been doing it for years and know it like the back of our hands.”

The answer is that what they “know” about their work practice is often biased with their own assumptions about existing tools and systems and is mostly shaped by the limitations and idiosyncrasies of these tools and practices. It is not easy for users consciously to describe what they do, especially in work that has been internalized. Humans are notoriously unreliable about this.

Also, each user has a different perspective of how the broader work domain functions. Knowledge of the full picture is distributed over numerous people. What they know about their work is like what the seven blind men “know” about an elephant.

Why not just gather requirements from multiple users and build a design solution to fit them all? You want an integrated design that fits into the “fabric” of your customer’s operations, not just “point solutions” to specific problems of individual users. This can only be achieved by a design driven by contextual data, not just opinions or negotiation of a list of features.

That is why contextual inquiry has taken on importance in the UX process. It takes real effort to learn about other people’s work, which is usually unfamiliar, especially the details. It can be difficult to untangle the web of clues revealed by observation of work.

Even surface observables can be complex, and the most important details that drive the work are usually hidden beneath the surface: the intentions, strategies, motivations, and policies. People creatively solve and work around their problems, making their barriers and problems less visible to them and to outsiders studying the work.

Because it is so difficult to understand user needs, much upfront time is wasted in typical projects in arguments, discussions, and conjectures about what the user wants or needs based on anecdotes, opinions, experience, etc. The processes of contextual inquiry and analysis remove the necessity for these discussions because the team ends up knowing exactly what users do, need, and think.

3.1.4 Not the Same as Task Analysis or a Marketing Survey

Oftentimes people might say, “We already do that. We do task analysis and marketing surveys.” While task analysis does identify tasks, it does not give enough insight into situations where tasks were interwoven or where users needed to move seamlessly from one task to another within the work context.

Task Analysis

Task analysis is the investigation and deconstruction of units of work. It is the process of representing the structure of these units plus describing how they are performed, including goals, steps, and actions.

Task analyses also do not work well in discovering or describing opportunistic or situated task performance. Paying attention to context in task analysis is what led us to contextual inquiry and analysis.

Similarly, you cannot substitute market research for contextual inquiry. They are just two different kinds of analysis and you may need both. Marketing data are about sales and can identify the kinds of products and even features customers want, but do not lead to any understanding about how people work or how to design for them. Customer/user data about work in context are what lead to design.

3.1.5 The Concepts of Work, Work Practice, and Work Domain

We use the term “work” to refer to the usage activities (including play) to achieve goals within a given domain. It is what people do to accomplish these goals. In most cases, use of the term “work” will be obvious, for example, using a CAD/CAM application to design an automobile.

“Work practice” is how people do their work. Work practice includes all activities, procedures, traditions, customs, and protocols associated with doing the work, usually as a result of the organizational goals, user skills, knowledge, and social interaction on the job. The context of this kind of work often includes some manual activities in association with some interactive activities.

If we are talking about the context of using a product, such as a consumer software product, then the “work” and “work activities” include all activities users are involved in while using that product. If the product is, say, a word processor, it is easy to see its usage to compose a document as work.

If the product is something like a game or a portable music player, we still refer to all activities a user undertakes while playing games or being entertained with music as “work” and “work activities.” Even though the usage activities are play rather than work, we have to design for them in essentially the same way, albeit with different objectives.

Similarly we call the complete context of the work practice, including the usage context of an associated system or product, the work activity domain or simply the work domain. The work domain contains the all-important context, without which you cannot understand the work.

3.1.6 Observing and Interviewing in Situ: What They Say vs. What They Do

Okay, so we agree that we have to learn about customer/user work, but why not stay in our own offices, where we have a good working environment and lots of logistical support, such as secretaries for note-taking and transcription, and spacious comfortable conference rooms? The answer is that you cannot get all the information you need by talking with users outside their work context, which only accesses domain knowledge “in the head.” Observing users and asking users to talk about their work activities as they are doing them in their own work context get them to speak from what they are doing, accessing domain knowledge situated “in the world” (see Figure 3-2).

image

Figure 3-2 Observation and interviewing for contextual data collection.

Even when occurring in situ, in the user’s own work environment, asking or interviewing alone is not enough. When gathering data in contextual inquiry, be sure to look beyond the descriptions of how things work, what is commonly believed, and what is told about the same. Observe the “ground truth”—the actual work practice, problems, issues, and work context. It is especially important to notice workarounds created by users when the intended system support or work practice does not support real needs.

Contextual inquiry in human–computer interaction (HCI) derives from ethnography, a branch of anthropology that focuses on the study and systematic description of various human cultures. In an article describing the transition from ethnography to contextual design, Simonsen and Kensing (1997) explain why interviews as an exclusive data-gathering technique are insufficient: “A major point in ethnographically-inspired approaches is that work is a socially organized activity where the actual behavior differs from how it is described by those who do it.” You need to observe and determine for yourself how the work in question is actually done.

Just as interviewing users is not enough to uncover their unmet needs, observation without interviewing also has its potential downsides. First, if you use observation as an exclusive data-gathering technique, you could miss some important points. For example, an important problem or issue simply might not come up during any given period of observation (Dearden & Wright, 1997).

Second, observation itself can affect user behavior. This is the famous “measurement effect”2 adapted to observation of people. The very act of observation can cause people to change the behavior being observed.

For example, when a person is subjected to new or increased attention, for example, being observed during task performance, the “Hawthorne effect” (Dickson & Roethlisberger, 1966) can produce a temporary increase of performance due to their awareness of being observed and perceived expectations of high performance. Diaper (1989) points this out as a natural human reaction. Simply put, when users are being observed, they tend to act like they think you want them to. When we are observed at work, we all want to do our best and be appreciated.

3.1.7 The SnakeLight, an Example of How Understanding Work Practice Paid Off

Here is an anecdotal example about why it helps to understand how your users do their activities and how they use products and systems. This example of the effectiveness of in situ contextual inquiry comes to us from the seemingly mundane arena of consumer flashlights. In the mid-1990s, Black & Decker was considering getting into handheld lighting devices, but did not want to join the crowded field of ordinary consumer flashlights.

So, to get new ideas, some designers followed real flashlight users around. They observed people using flashlights in real usage situations and discovered the need for a feature that was never mentioned during the usual brainstorming among engineers and designers or in focus groups of consumers. Over half of the people they observed during actual usage under car hoods, under kitchen sinks, and in closets and attics said that some kind of hands-free usage would be desirable.

They made a flashlight that could be bent and formed and that can stand up by itself. Overnight the “SnakeLight” was the product with the largest production volume in Black & Decker history, despite being larger, heavier, and more expensive than other flashlights on the market (Giesecke et al., 2011).

3.1.8 Are We Gathering Data on an Existing System or a New System?

When gathering data and thinking about designs for a new system, analysts and designers can be strongly biased toward thinking about only the new system. Students sometimes ask, “Should we be modeling the existing way they do it or as it would be done with the new system?” This is asking whether to do modeling in the problem domain or the solution domain, the work domain or the design domain.

At the end of the day, the answer might well be “both,” but the point of this particular discussion is that it must start with the existing way. Everything we do in contextual inquiry and contextual analysis in this chapter and the next is about the existing way, the existing system, and the existing work practice. Often team members get to thinking about design too early and the whole thing becomes about the new system before they have learned what they need to about work practice using the existing system.

In order for all this to work, then, there must be an existing system (automated, manual, or in-between), and the proposed new system would then somehow be an improvement. But what about brand new ideas, you ask, innovations so new that no such system yet exists? Our answer may be surprising: that situation happens so rarely that we are going to go out on a limb and say that there is always some existing system in place. Maybe it is just a manual system, but there must be an existing system or there cannot be existing work practice.

For example, many people consider the iPod to be a really innovative invention, but (thinking about its usage context) it is (mainly) a system for playing music (and/or videos). Looking at work activities and not devices, we see that people have been playing music for a long time. The iPod is another in a series of progressively sophisticated devices for doing that “work” activity, starting with the phonograph invented by Thomas Edison, or even possibly earlier ways to reproduce “recorded” sound.

If no one had ever recorded sound in any way prior to the first phonograph, then there could not have been an “existing system” on which to conduct contextual inquiry. But this kind of invention is extremely rare, a pure innovative moment. In any case, anything that happens in sound reproduction after that can be considered follow-on development and its use can be studied in contextual inquiry.

3.1.9 Introducing an Application for Examples

As a running example to illustrate the ideas in the text, we use a public ticket sales system for selling tickets for entertainment and other events. Occasionally, when necessary, we will provide other specific examples.

The existing system: The Middleburg University Ticket Transaction Service

Middleburg, a small town in middle America, is home to Middleburg University, a state university that operates a service called the Middleburg University Ticket Transaction Service (MUTTS). MUTTS has operated successfully for several years as a central campus ticket office where people buy tickets from a ticket seller for entertainment events, including concerts, plays, and special presentations by public speakers. Through this office MUTTS makes arrangements with event sponsors and sells tickets to various customers.

The current business process suffers from numerous drawbacks:

ent All customers have to go to one location to buy tickets in person.

ent MUTTS has partnered with Tickets4ever.com as a national online tickets distribution platform. However, Tickets4ever.com suffers from low reliability and has a reputation for poor user experience.

ent Current operation of MUTTS involves multiple systems that do not work together very well.

ent The rapid hiring of ticket sellers to meet periodic high demand is hampered by university and state hiring policies.

Organizational context of the existing system

The desire to expand the business coincides with a number of other dynamics currently affecting MUTTS and Middleburg University.

ent The supervisor of MUTTS wishes to expand revenue-generating activities.

ent To leverage their increasing national academic and athletic prominence, the university is seeking a comprehensive customized solution that includes integration of tickets for athletic events (currently tickets to athletic events are managed by an entirely different department).

ent By including tickets for athletic events that generate significant revenue, MUTTS will have access to resources to support their expansion.

ent The university is undergoing a strategic initiative for unified branding across all its departments and activities. The university administration is receptive to creative design solutions for MUTTS to support this branding effort.

The proposed new system: The Ticket Kiosk System

The Middleburg University Ticket Transaction Service (MUTTS) wants to expand its scope and expand to more locations, but it is expensive to rent space in business buildings around town and the kind of very small space it needs is rarely available. Therefore, the administrators of MUTTS and the Middleburg University administration have decided to switch the business from the ticket window to kiosks, which can be placed in many more locations across campus and around town.

Middleburg is home to a large public university and has reliable and well-used public transportation provided by its bus system operated by Middleburg Bus, Inc. There are several bus stops, including the library and the shopping mall, where there is space to add a kiosk for a reasonable leasing fee to the bus company.

A number of these bus stops seem good locations for kiosks; buses come and go every few minutes. Some of the major stops are almost like small bus stations with good-sized crowds getting on and off buses.

In addition to an expected increase in sales, there will be cost savings in that a kiosk requires no personnel at the sales outlets. The working title for the new system is Ticket Kiosk System, pending recommendations from our design team. The Ticket Kiosk System will have a completely new business model for the retail ticket operation.

3.2 The system concept statement

A system concept statement is a concise descriptive summary of the envisioned system or product stating an initial system vision or mandate; in short, it is a mission statement for the project. A system (or product) concept statement is where it all starts, even before contextual inquiry. We include it in this chapter because it describes an initial system vision or mandate that will drive and guide contextual inquiry. Before a UX team can conduct contextual inquiry, which will lead to requirements and design for the envisioned system, there has to be a system concept.

Rarely does a project team conceptualize a new system, except possibly in a “skunk-works” kind of project or within a small invention-oriented organization. The system concept is usually well established before it gets to the user experience people or the software engineering people, usually by upper management and/or marketing people. A clear statement of this concept is important because it acts as a baseline for reality checks and product scope and as something to point to in the middle of later heated design discussions.

ent A system concept statement is typically 100 to 150 words in length.

ent It is a mission statement for a system to explain the system to outsiders and to help set focus and scope for system development internally.

ent Writing a good system concept statement is not easy.

ent The amount of attention given per word is high. A system concept statement is not just written; it is iterated and refined to make it as clear and specific as possible.

An effective system concept statement answers at least the following questions:

ent What is the system name?

ent Who are the system users?

ent What will the system do?

ent What problem(s) will the system solve? (You need to be broad here to include business objectives.)

ent What is the design vision and what are the emotional impact goals? In other words, what experience will the system provide to the user? This factor is especially important if the system is a commercial product.

The audience for the system concept statement is broader than that of most other deliverables in our process and includes high-level management, marketing, the board of directors, stockholders, and even the general public.

Example: System Concept Statement for the Ticket Kiosk System

Here is an example of a system concept statement that we wrote for the Ticket Kiosk System.

 The Ticket Kiosk System will replace the old ticket retail system, the Middleburg University Ticket Transaction Service, by providing 24-hour-a-day distributed kiosk service to the general public. This service includes access to comprehensive event information and the capability to rapidly purchase tickets for local events such as concerts, movies, and the performing arts.

 The new system includes a significant expansion of scope to include ticket distribution for the entire MU athletic program. Transportation tickets will also be available, along with directions and parking information for specific venues. Compared to conventional ticket outlets, the Ticket Kiosk System will reduce waiting time and offer far more extensive information about events. A focus on innovative design will enhance the MU public profile while Fostering the spirit of being part of the MU community and offering the customer a Beaming interaction experience. (139 words)

This statement can surely be tightened up and will evolve as we proceed with the project. For example, “far more extensive information about events” can be made more specific by saying “extensive information including images, movie clips, and reviews about events.” Also, at this time we did not mention security and privacy, important concerns that are later pointed out by potential users. Similarly, the point about “focus on innovative design” can be made more specific by saying “the goal of innovative design is to reinvent the experience of interacting with a kiosk by providing an engaging and enjoyable transaction experience.”

Usually a system concept statement will be accompanied by a broader system vision statement from marketing to help get a project started in the right direction. None of this yet has the benefit of information from customers or potential users. However, we do envision the customer being able to find event information, select events to buy tickets for, select seats, purchase tickets, print tickets, and get information and tickets for transportation while enjoying the overall experience interacting with the kiosk.

Upon interacting with the customers and users, some of our objectives in this system concept statement will be adjusted and assumptions corrected.

Exercise

See Exercise 3-1, System Concept Statement for a System of Your Choice

NB: All exercises are in Appendix E, near the end of the book.

3.3 User work activity data gathering

Much of the material in this chapter comes from the contextual design material existing in the literature. We do not try to reproduce these entire processes in this book, as those topics already appear in books of their own, with credit to their respective authors. What we do here is draw on these processes, adapting them to establish our own frame of reference and integrating them into the context of other requirements-related activities.

We gratefully acknowledge the sources from which we have adapted this material, mainly Contextual Design (Beyer & Holtzblatt, 1998) and Rapid Contextual Design (Holtzblatt, Wendell, & Wood, 2005). Other work we have drawn upon and which we acknowledge include Constantine and Lockwood (1999). A CHI Conference paper by Hewlett-Packard people (Curtis et al., 1999) contributed to our understanding by spelling out an excellent large-scale example of the application of contextual design.

To do your user work activity data gathering you will:

ent prepare and conduct field visits to the customer/user work environment, where the system being designed will be used

ent observe and interview users while they work

ent inquire into the structure of the users’ own work practice

ent learn about how people do the work your system is to be designed to support

ent take copious, detailed notes, raw user work activity data, on the observations and interviews

In these early chapters we are generally taking the perspective of domain-complex systems because it is the more “general” case. We will describe several methods and techniques that have proven successful, but you should be creative and open to including whatever techniques suit the needs of the moment. This means that you might want to use focus groups, for example, if you think they will be useful in eliciting a revealing conversation about more complex issues.

Domain-Complex Systems

Domain-complex systems are systems with high degree of intricacy and technical content in the corresponding field of work. Often, characterized by convoluted and elaborate mechanisms for how parts of the system work and communicate, they usually have complicated workflow containing multiple dependencies and communication channels. Examples include an air traffic control system and a system for analyzing seismic data for oil exploration.

The goals of contextual inquiry are the same in both perspectives (domain-complex systems vs. interaction-complex consumer products), and most of the steps we describe apply to, or can easily be adapted for, the product perspective. Where appropriate, we will offer descriptions of how the process might differ for the product user perspective.

3.3.1 Before the Visit: Preparation for the Domain-Complex System Perspective

Learn about your customer organization before the visit

Preparation for the visit means doing upfront planning, including addressing issues such as these about the customer:

ent For work activities situated in the context of a system with a complex work domain, get a feel for the customer’s organizational policies and ethos by looking at their online presence—for example, Website, participation in social networks.

ent Know and understand the vocabulary and technical terms of the work domain and the users.

ent Learn about the competition.

ent Learn about the culture of the work domain in general—for example, conservative financial domain vs. laid-back art domain.

ent Be prepared to realize that there will be differences in perspectives between managers and users.

ent Investigate the current system (or practices) and its history by looking at the company’s existing and previous products. If they are software products, it is often possible to download trial versions of the software from the company’s Website to get familiar with design history and themes.

Learn about the domain

While designing for complex and esoteric domains, working first with subject matter experts helps shorten the actual contextual inquiry process by giving you a deeper understanding of the domain, albeit from a non-user perspective. Your contextual inquiry process can now include validating this understanding. In cases where time and resources are at a premium (not an insignificant portion of projects in the real world), you may just have to make do with just interviewing a few subject matter experts instead of observing real users in context.

Issues about your team

In addition, there are issues to address about your team:

ent Decide how many people to send on the visits.

ent Decide who should go on each visit, for example, user experience people, other team members, documentation folks.

ent Set your own limits on the number of visits and number of team members involved, depending on your budget and schedule.

ent Plan the interview and observation strategy (who in the team does what).

Your visit-group size can depend on how many are on your initial project team, the number of different user roles you can identify, the size of the project overall, the budget, and even your project management style. Practitioners report taking as many as two to eight or more people on visits, but three to four seems to be typical.

A multidisciplinary team is more likely to capture all necessary data and more likely to make the best sense of data during subsequent analysis. We have found using two people per interview appealing; one to talk and one to take notes.

Lining up the right customer and user people

Among the things to do to prepare for a site visit for contextual inquiry, you should:

ent Select and contact appropriate users or customer management and administrative people to:

ent explain the need for a visit

ent explain the purpose of the visit (to learn about their work activities)

ent explain your approach (for them actually to do the work while you are there to observe)

ent obtain permission to observe and/or interview users at work

ent build rapport and trust, for example, promise personal and corporate confidentiality

ent discuss timing—which kinds of users are doing what and when?

ent set scope: explain that you want to see the broadest representation of users and work activities, focusing on the most important and most representative tasks they do

ent establish or negotiate various parameters, such as how long you will/can be there (it can be up to several intense weeks for data gathering), how often to visit (it can be up to every other day), how long for the average interview (a couple of hours maximum), and the maximum number of interviews per visit (as an example, four to six)

ent Select and contact appropriate support people (determined by the management people you talk with) within the customer organization to arrange logistics for the visits.

ent Select and contact appropriate people to meet, observe, and interview: customers, users (who do the work in question), especially frequent users, managers; aim for the broadest variety, cover as many usage roles as possible, plan visits to multiple sites if they exist.

This latter item, selecting the people to meet, observe, and interview, is especially important. Your fieldwork should include all work roles, selected other stakeholders who impact work directly or indirectly, and (depending on the project) possibly grand-customers (customers of the customer) outside the user’s organization. You want the broadest possible sources to build a holistic and multi-perspective picture of the contextual data.

Get access to “key” people

For projects in a domain-complex system context, you might also be told by your customer that users of the system in question are scarce and generally unavailable. For example, management might resist giving access to key people because they are busy and “bothering” them would cost the organization time and money.

If you sense reluctance to give access to users, you need to step up and make the case; establish the necessity for gathering requirements that will work and the necessity for firmly basing requirements on an understanding of existing work activities. Then explain how this extra work upfront will reduce long-term costs of reworking everything if analysts do not get the right requirements. Ask for just a couple of hours with key users. Persevere.

At the other end of the spectrum, for consumer software, such as shrink-wrap word processors, users abound and you can recruit users to interview via a “help wanted” ad posted in the local grocery store.

Do not interview only direct users. Find out about the needs and frustrations of indirect users served by agents or intermediaries. And do not forget managers. Here is a quote from a team that we worked with on a project, “It was eye-opening to talk with the managers. Managers are really demanding and they have different kinds of requirements from those of the users, and they see things from a totally different viewpoint than the other users.”

Sometimes you may have access to the users for only a small period of time and therefore cannot observe them doing work. In such cases, you can ask them to walk you through their typical day. You must work extra hard to ask about exceptions, special cases, and so on. This approach suffers from many of the problems we described earlier regarding not observing users in context but at least provides some insights into user’s work.

What if you cannot find real users?

In the worst case, that is, when you have no access to real users (this has happened in our consulting and work experience), the last resort is to talk to user proxies. User proxies can be business experts or consultants who are familiar with the user’s work.

This approach suffers from many disadvantages and often results in hearing about high-level functional needs and crude approximations of what a broad class of users need in the system. The accounts of such proxies are often tainted by their own opinions and views of the work domain. They also suffer from serious omissions and simplifications of often nuanced and complex user work activities.

Setting up the right conditions

The environment, the people, and the context of the interview should be as close a match to the usual working location and working conditions as possible. We once found ourselves being ushered into a conference room for an interview because, as the employer put it, “it is much quieter and less distracting here.”

The employer had even arranged for time off from work for the worker so that he could focus his complete attention on the interview. But, of course, the conference room was not anything like the real work context and could not possibly have served as a useful source of information about the work context. We had to convince them to move the whole thing back into the active workplace.

Make sure that the observations and interviews are conducted without undue political and managerial influences. You want to create the right conditions for observation and interviews, conditions in which users feel comfortable in telling the “real” story of the everyday work practice. We once had to deal with the supervisor of a person we wanted to interview because the supervisor insisted on being present during the interview. His reason was that it was a rare opportunity to learn more about what his workers did and how.

However, we also suspected that the supervisor did not want the employee to be complaining to strangers about working conditions or the organization. However, from the worker’s view, having a supervisor present looked a lot like an opportunity for the supervisor to evaluate the user’s job performance. It meant not being able to be open and candid about much of anything. Instead, the employee would have to pay very close attention to doing the job and not saying anything that could be interpreted in a way that could be used against him. It would be anything but a sample of everyday work practice.

How many interviewees at a time?

It might work out that, via a group interview, multiple users can work together and produce data not accessible through a single user. However, group interviews can also mask individual thoughts. Each user may have a very different view of how things work and what the problems are, but these differences can be sublimated in an unconscious effort to reach “consensus.” Additionally, group dynamics may be influenced by hidden agendas and turf battles.

Preparing your initial questions

Script your initial interview questions to get you off to a good start. There is no real secret to the initial questions; you ask them to tell you and to show you how they do their work. What actions do they take, with whom do they interact, and with what do they interact? Ask them to demonstrate what they do and to narrate it with stories of what works, what does not work, how things can go wrong, and so on.

We found that instead of asking them generally “What do you do here?” it is sometimes more helpful to ask them to walk us though what their work specifically entailed the day before and if that was typical. This kind of a specific probing gives them an easy point of reference to make their descriptions concrete.

Before the visit: Preparation for the product perspective

While the aforementioned guidelines for preparing a visit in a domain-complex system context generally also apply to a product perspective, there are a few differences. For one, the context of work in a product design perspective is usually much narrower and simpler than that in an entire organization. This is primarily because organizations contain numerous and often widely different roles, each contributing to a part of the overall work that gets accomplished.

In contrast, the work activities within a product design context are usually centered on a single user in a single role. To observe the full range of usage patterns by a single user of a product, you usually have to observe their usage over a long time. In other words, to do this kind of contextual inquiry, instead of observing several users in a work role for a relatively short time, you have to “shadow” single users over a longer time.

For example, the work, or play, activities associated with a portable music player system sometimes include searching for and listening to music. At other times the same user is trying to manage music collections. Even in cases where the design needs to support multiple users, say the user’s family, the complexity of the interaction among different roles is usually much lower in the product perspective, and often more homogeneous than in a domain-complex system perspective.

Where do we start with our contextual inquiry process for such products? The best place to start is by understanding the complete usage context of this kind of product, including desirable features and limitations.

We also have to ask about things such as branding, reputation, and competition in this product segment. To find unbiased information about these issues, instead of looking online for the customer’s organizational policies and culture, we need to look for user groups or blogs about using this kind of product and check out reviews for similar products.

Do some initial brainstorming to see what kinds of user communities are primary targets for this product segment. College students? Soccer moms? Amateur photographers? Then think of good places to meet people in these user classes. If necessary, use marketing firms that specialize in recruiting specific target populations.

Cross-Cultural User-Experience Design

Mr. Aaron Marcus, President, and Principal Designer/Analyst, Aaron Marcus and Associates, Inc. (AM+A)

Modern technology and commerce permit global distribution of products and services to increasingly diverse users who exist within different cultures. Culture affects every aspect of tool and sign making. Culture-centered design of user experiences seems “inevitable.” Designers/analysts are aware of culture, but may not be informed of specific dimensions by which cultures can be described and measured.

Websites are one set of examples; they are immediately accessible by people worldwide and offer design challenges of “localization” that go beyond translation (Marcus and Gould, 2000). Some years ago, Jordanian Website Arabia.On.Line used English for North American and European visitors, but the layout read right to left as in Arabic because the local designers were too influenced by their own culture.

Localization goes beyond languages and translation. If one were to examine the home page of Yahoo.com in English and Maktoob.com, one of the Arabic world’s most popular portals in Arabic, one would find not only language differences, but differences in color, imagery, organization, and topics of primary interest. There may be geographic, historical, political, aesthetic, and language differences.

Small-scale communities with preferred jargon, signs, and rituals can constitute a “cultural group.” This definition is different from traditional definitions of culture that more typically refer to longstanding historical differences established over many generations and centuries. The modern cultural group may be considered more a social group or “lifestyle group,” including affinity groups, social groups, and geographically dispersed groups communicating through the Internet. Today, “digital natives” vs “digital immigrants” may constitute significant differences in “culture.”

The challenge for business is how to account for different user experiences that are culturally differentiated in a cost-effective manner. Developers may need to rely on market and user data to achieve short-term success and to avoid wasting money and time on too many variations. Paying attention to culture models and culture dimensions can assist.

Culture Models and Culture Dimensions

Analysts have written about culture for many decades. Geert Hofstede’s (1997) cultural dimensions are well known and well established, although controversial for some anthropologists and ethnographers. Hofstede examined IBM employees in more than 50 countries during 1978–1983 and was able to gather data and analyze it in a statistically valid method. His definition of culture (in his model, each country has one dominant culture) concerns patterns of thinking, feeling, and acting that are “programmed” by a particular group in their children by the time they reach pubescence. The differences of culture manifest themselves in specific rituals, symbols, heroes/heroines, and values. Hofstede’s five dimensions of culture are the following:

 Power-distance: High vs low—differences between powerful people in the society and others

 Collectivism vs individualism: being raised in a group and owing allegiance, or not

 Femininity vs masculinity: roles that different sexes play within the society

 Uncertainty avoidance: High vs low—the degree of discomfort about things not known

 Long-term orientation vs short-term orientation: Confucian values of perseverance, virtue, etc., or other values.

For each culture dimension, Hofstede noted differences of attitudes toward work, family, and education.

Cautions, Considerations, and Future Developments

Although Hofstede’s model is well established, and many studies have been based on it, there are also criticisms of the model:

ent Old data, pre-postmodern (no emphasis on media, sociology of culture, politics of culture)

ent Corporate subjects only, not farmers or other laborers

ent Assumes one culture per country

ent Assumes fixed, unchanging relationships

ent Gender roles, definitions debatable

ent Seems too general, stereotypical

Studies have shown that even the concept of usability may be biased. A study published in CHI 2009 Proceedings (Frandsen-Thorlacius et al., 2009) showed that Chinese users found fun and visual appeal to be related more closely to usability than for Danish users.

At the very least, awareness of culture models and culture dimensions enlarges the scope of issues. For example, these models challenge the professions of UI development to think about appropriate metaphors for different cultures, user profiles that are culture sensitive, differing mental models, and their influence on performance, not only preference, alternate navigation strategies, evaluation techniques, attitude toward emotions, etc. An additional challenge is introducing culture considerations into corporate and organization frameworks for product/service development and into centers of user-centered design. There are additional sources of insight into UX and culture, each of which has formulated models and seven plus or minus two dimensions. Each of these gives rise to further issues and interactions with culture: persuasion, trust, intelligence, personality, emotion, and cognition.

With the rise of India and China as sources of software and hardware production, innovation, and consumption, it becomes more obvious that computer-mediated communication and interaction occur in a context of culture. It is inevitable that user-experience development must account for cultural differences and similarities. Models, methods, and tools do exist, but many research issues lie ahead. Future development of tools, templates, and treasure chests of patterns will provide a body of knowledge in the future for more humane, cultured design of computer-based artifacts.

References

1. Hofstede G. Cultures and Organizations: Software of the Mind. New York: McGraw-Hill; 1997.

2. Frandsen-Thorlacius O, Hornbæk K, Hertzum M, Clemmensen T. Non-Universal Usability? A Survey of How Usability Is Understood by Chinese and Danish Users. 6 April 2009, In: Proc., CHI 2009. Boston, MA 2009;41–50.

3. ACM Publisher Marcus A, Gould EW. Crosscurrents: Cultural Dimensions and Global Web User-Interface Design. Interactions. 2000;7(4):32–46 www.acm.org; 2000.

Anticipating modeling needs in contextual inquiry: Create contextual data “bins”

There is a spectrum of approaches to contextual data collection from data driven to model driven. We draw on the best of both but lean toward the model-driven approach. A data-driven approach operates without any presuppositions about what data will be observed. There are no predefined data categories to give hints about what kind of data to expect. The data-driven approach simply relies on data encountered to guide the process of data gathering and subsequent analysis. Whatever arises in contextual inquiry observations and interviews will define the whole process.

Data Bin

A data bin is a temporary repository—for example, a labeled pile of notes on a table—to hold data—raw contextual data at first and, later, synthesized work activity notes. Each bin corresponds to a different data category or contextual data topic.

Alternatively, a model-driven contextual inquiry process means that instead of just gathering relevant data as you encounter it in observations and interviews, you use your experience to help guide your data collection. In particular, you use the known general categories of design-informing models (Chapter 6) as a guide for kinds of data to watch for, looking forward to the data needs of modeling so that at least some of your data collection in contextual inquiry can be guided by these future needs.

Data-Driven Inquiry

Data-driven inquiry is led entirely by the work activity data as is presents itself, forestalling any influence from the analyst’s own knowledge, experience, or expectations. The idea is to avoid biases in data collection.

From your knowledge you will have a good idea of which models will be needed for your project and what kind of data will be needed for your models. Using this knowledge, you create some initial “bins” for data categories and models into which you can start putting your contextual data, in whatever form it has at this point. A bin is a temporary place to hold data in a given category. As you collect data, you will think of other categories to add more bins.

Model-Driven Inquiry

In model-driven inquiry, contextual data gathering is informed by knowledge and expectations from experience, intelligent conjecture, and knowledge of similar systems and situations. The idea is to be more efficient by using what you know, but it comes at the risk of missing data due to biases.

For example, we will cover construction of what we call a physical model (Chapter 6), which includes a diagram of the physical layout of the working environment. So, if a physical model is relevant to your project, then you will need to make a sketch and/or take photos of the physical layout, where equipment is located, and so on while you are still on-site doing contextual inquiry. In order to meet those modeling needs later, you will also need to take notes about the physical layout and any problems or barriers it imposes on the work flow and work practice.

In the next chapter we will extend and complete the creation of bins for sorting and organizing your data in contextual analysis.

3.3.2 During the Visit: Collecting User Work Activity Data in the Domain-Complex System Perspective

When you first arrive

Begin your first field visit by meeting the manager, administrator, or supervisor through whom you arranged the visit. Continue with the building of trust and rapport that you started previously. Make it clear that you are doing this for the purpose of helping make a better design. It is a big advantage if, at the beginning, you can briefly meet all customer personnel who will be involved so that you can get everyone off to the same start by giving the overview of your goals and approach, explaining what will happen during your visits and why.

Remember the goal

Often in field visits for “talking with our users,” people ask users what they want or need. In a contextual inquiry, we do not ask users what they want or need. We observe and interview users in their own work context about how they do their work as they are doing the work that later will be supported by the system you hope to design.

And, by all means, do not forget that we are on a quest for a quality user experience. The techniques of contextual inquiry and contextual analysis will not necessarily take care of searching out the special aspects of user experience; you have to be tuned to it. So be especially aware of user stories about particularly good or bad usage experiences and ferret out the reasons, both in design and in usage context, that contribute to those experiences.

Establish trust and rapport

The interviews with users should also start with trust building and establishing rapport. Help them understand that you have to ask many questions and “get in their face” about the work. Interviewing skills are learned; observe users doing their work and ask many questions about why they do something, how they do certain things, how they handle certain cases, and get them to tell specific stories about their work and how they feel about the work and the way it is done in the existing environment. Follow them around; do not miss a part of the work activity by staying behind if they have to move as part of their work.

Form partnerships with users

In most consulting situations the person who comes in from outside the customer organization is considered the “expert,” but it is quite the opposite in contextual inquiry. The user is the expert in the domain of work practice and you are the observer trying to understand what is happening.

The process works best if you can form a partnership with each user in which you are co-investigators. Invite the user into the inquiry process where you can work together in close cooperation. You need to establish an equitable relationship with the user to support information exchange through a dialog.

As the observations and interviews proceed you can feed the partnership by sharing control of the process, using open-ended questions that invite users to talk, for example, what are you doing? Is that what you expect? Why are you doing that? Be a good listener and let the user lead the conversation. Pay attention to nonverbal communication.

Task data from observation and interview

One of the most important kinds of contextual data to collect is task data. You will need this to build your task structure models and task interaction models in Chapter 6. This is where a combination of observation and interview can work especially well. Get task-related data by observing actual sessions of users doing their own work in their own work context.

At the same time, you will interview the users, but asking only about the task they are doing, not about possible tasks or tasks that other users do. The interview component is used to interpret and assign meaning to what is observed. To have necessary data for task models later, ask questions to clarify anything not obvious in what you observe. Ask questions about the purposes and rationale for each task and each important step; why do they do certain actions?

On the observation side of things, be sure to notice the triggers for tasks and steps; what happens to cause them to initiate each task or step? For example, an incoming phone call leads to filling out an order form.

Learn about your users’ task barriers by observing tasks being performed and by think-aloud verbal explanation of underlying information about the tasks, such as task goals. Notice hesitations, problems, and errors that get in the way of successful, easy, and satisfying task or step completion. Probe for what was expected and reasons why it did not turn out well. You will need these answers to model barriers in the upcoming analysis and modeling.

It takes a certain skill to key in on occurrences of critical information in the flow of observation and interviews. With practice, you will acquire an awareness and ability to detect, discern, and discriminate the wheat from the chaff of the flow.

The output of this process is raw user work activity data in the form of lengthy observation and interview notes or transcripts of recorded sessions.

Recording video

Video recording is an effective way of comprehensively capturing raw contextual data where conditions and resources permit. Video recording can help you capture important nonverbal communication cues in contextual data.

However, factors such as the time and effort to produce and review the recordings can weigh against the use of video recording in contextual inquiry data collection. Confidentiality, privacy, and other such concerns can also preclude the use of video.

In addition, video recording can interfere with a close user relationship. The feeling that what they say is permanently captured may prevent them from being forthcoming. They may not be too willing to say something negative about existing work practice or complain about policies at work. Informal note taking, however, can provide a more intimate conversational experience that may encourage honest expression. Despite all these possible barriers, video clips can provide convincing evidence, when linked to the contextual notes, that an issue is real.

Note taking

Regardless of whether you use video or audio recordings of observation and interview sessions, you should consider note taking your primary source of usable raw data. Manual paper note writing may be the most commonly used contextual inquiry data collection technique used in the real world. It is unintrusive, not intimidating to the user, and fits most naturally into what should be a more or less low-key interaction with the user. Alternatively, a laptop is acceptable for note taking, if you can do it inconspicuously.

When taking notes, you must incorporate some scheme that preserves information about the source of each note. We recommend that you use:

ent quotations marks to denote what a user says

ent plain text to describe observations of what users do

ent parentheses to delimit your own interpretations

A small handheld digital audio recorder used inconspicuously, but not trying to be covert, might be beneficial to augment note taking, especially when things are moving fast. One way to use audio recording is as the audio equivalent of written notes.

In this mode, after hearing user comments or observing user behavior, you dictate a short summary into the recorder, much as a medical doctor dictates summaries for patient charts during daily rounds. This mode of operation has the additional benefit that if the user can hear you paraphrase and summarize the situation, it is a chance to correct any misconceptions.

Use a numbering system to identify each point in data

It is important to use a numbering system to identify uniquely each note, each point in the raw data, or each sequence in a video or audio recording or transcript. This last item is necessary to provide a way to reference each note. Later, in analysis, each conclusion must be linked to the associated raw data note or else it cannot be treated as authentic. Some of the ways to tag your raw data for reference in analysis include the following:

ent If you record sessions, you can use video frame numbers or time codes on the recording as identifiers of sequences and points in raw data.

ent If you record sessions, you definitely should assign line numbers to the transcripts, just as it is done for legal documents.

ent If you take manual notes, each note should be tagged with a note identification number tied to the person or persons who are the data source.

How to proceed

Record raw data by expressing it in the user’s voice. Because these data are raw, it makes sense to express points in the interview transcripts generally as they occurred during the interview, which would usually be by using the words of the user. For example, the statement: “I like to add an extra egg when I make a cake” naturally reflects the fact that a user is speaking. If you record your interviews, the transcripts will mostly appear as the exact words of the user anyway.

Switching to an expression such as “the user likes to add an extra egg when baking a cake” unnecessarily introduces an analyst flavor that is not useful this early in the process. Moreover, the user’s voice describes much closely the user’s experience, and subtle use of adjectives and expressions can provide clues on designing for enhancing that experience.

It is your job to capture data about the user’s work. Do not expect users necessarily to tell you what they want or need; this is just about how they work and how they feel about it. Your team will deduce needs later, after they understand the work. Also, do not expect users to do design, although they might occasionally suggest something they would like to see in the system.

ent Be a listener; in most cases you should not offer your opinions about what users might need.

ent Do not lead the user or introduce your own perspectives.

ent Do not expect every user to have the same view of the work domain and the work; ask questions about the differences and find ways to combine to get the “truth.”

ent Capture the details as they occur; do not wait and try to remember it later.

ent Be an effective data ferret or detective. Follow leads and discover, extract, “tease out” and collect “clues.” Be ready to adapt, modify, explore, and branch out.

Part of being a good detective, the latter point above, is being willing to deviate from a script when appropriate. Be prepared to follow leads and clues and take the interview and observations where you need to go, tailoring questions to meet the goal of learning all you can about their work practice, work environment, and work issues and concerns.

As an example of following leads, this is a real story told by a team doing a project for one of our classes. The client was in retail sales and the conversation of the interview had centered on that concept, including serving their customers, making the sale transaction, and recording it.

However, during this conversation the word “inventory” was mentioned once, in the context of point-of-sale data capture. No one had asked about inventory, so no one had mentioned it until now.

Our good ethnographic detectives, recognizing an entree to another area of work activities, pounced on that word and pursued a new train of thought. What about inventory? What role does it play in your point-of-sale data capture? Where does it go from there? How is it used and for what? How do you use inventory data to keep from running out of stock on items in demand? Who orders new stock and how? Once an order is sent, how do you keep track of it so it does not fall through the cracks? What happens when the new stock is delivered? How do you know when it arrives? Who works in receiving and what do they do? How do you handle partial shipments?

As an example of dialogue that violates the point above about not introducing your own perspectives, consider this user comment: “I want privacy when I am buying tickets.” You might be tempted to say: “You mean, when you are looking for events and buying tickets, you do not want other people in line to know what you are doing?” To which the user might respond: “Yes, that is what I mean.” A better way to handle the user’s comment here would have been with a follow-up question such as “Can you elaborate what you mean by wanting privacy?”

Pay attention to information needs of users

As you talk with users in the work roles, try to identify their information needs in the context of the work activities and tasks, as they do their jobs in the work domain. Do the current work practices and the current software systems provide information needed by users to do their jobs? Is the needed information provided at the time it is needed and in the form it is needed? And beware of “information-flooding screens.”

When designers do not know what users need, they often fall back on the unjustifiable excuse that the users themselves will know what they need. These designers then create designs that display all information available or all the information users might need, in an “information flooding screen,” and let the users sort it out. The designer’s assumption is that all the information needed is presented—the “it is all there” syndrome—and the users are in the best position to know which parts are needed for which functions/tasks and what format is best for the job. This is a thinly veiled copout for not doing the necessary upfront analysis to inform the design.

What about design ideas that crop up?

Contextual inquiry is not about design, but you do not want to lose any good ideas, so you should make note of design ideas from users as they come up and then get back to asking about work practice. It is normal for users to suggest design ideas, often very specific and sometimes not very practical. It is the interviewer’s responsibility to take note of these suggestions, but to ask more questions to connect them back to work practice. Ask “why?” How does that suggestion fit into your workflow? What part of your work leads to a need for this?

What about analyst and designer ideas that crop up?

Similarly, make note of design ideas from your own team and tag them as such. Just as with users, it is normal for analysts to get design ideas during interviews or during subsequent analysis activities.

Because such suggestions can introduce analyst bias into what is supposed to be all about user data, “righteous” analysts may want to ignore them. But even analyst ideas generated in the action of contextual inquiry are real data and it would be a shame to lose them. So to include analyst and designer data in contextual inquiry, we suggest getting user confirmation by asking about these ideas and keeping clear the source; be sure to label or tag such notes as analyst ideas.

Questions not to ask

Do not expect you can ask customers and users for direct answers to the questions that will lead you straight to design. Remember that contextual inquiry is often called the process for discovering what users cannot tell you. In his “column” on the User Interface Engineering Website, Jared Spool (2010) advises us about three specific questions not to ask customers or users during field visits. We summarize the three no-no questions here:

ent Do not ask about the future; do not ask users what they would do in a given circumstance. The answer will probably not reflect the reality of what they might do if in the same situation but all alone at work or at home.

ent Do not ask for design advice, how they would design a given feature. Users are not designers and do not usually have a design-thinking mind-set. You are likely to get off-the-wall answers that will not fit in with the rest of your design; although their idea might work in the present situation, it might not fit other usage conditions.

ent Do not ask a question by trying to state what you think is their rationale. You just put ideas in their heads and they might give answers they think you want. Users often do not think about their usage in terms of a logical rationale for each action.

Collect work artifacts

During site visits collect as many samples of work artifacts, such as paper forms, templates, work orders, and other paperwork, as you can. Work artifacts include not just paperwork, but all items used in the work practice and photos of the same.

For example, consider the keys to a customer’s car in an auto repair facility. First, they may be put in an envelope with the work order, so the mechanic has the keys when needed. After repairs, the keys are hung on a peg board, separate from the repair order until the invoice is presented to the customer and the bill is paid. Artifacts include physical or electronic entities that users create, retrieve, use or reference within a task, and/or pass on to another person in the work domain. This passing of artifacts should also show up in the flow model.

Example: Work Artifacts from a Local Restaurant

One of the project teams in our user experience class designed a system to support a more efficient workflow for taking and filling food orders in a local restaurant, part of a regional chain. As part of their contextual inquiry, they gathered a set of paper work artifacts, including manually created order forms and “guest checks,” shown in Figure 3-3.

image

Figure 3-3 Examples of work artifacts gathered from a local restaurant.

These artifacts are great conversational props as we interview the different roles that use them. They provide avenues for discussion given the fact that almost every restaurant uses these artifacts over and over again. What are things that work with this kind of artifact for order taking? What are some breakdowns? How does a person’s handwriting impact this part of the work activity? What is the interaction like between the wait staff and the restaurant’s guests?

Other forms of data collection

Other kinds of contextual data are also essential in representing work context, including:

ent Copious digital pictures of the physical environment, devices, people at work, and anything else to convey work activities and context visually. Respect the privacy of the people and ask for permission when appropriate.

ent On-the-fly diagrams of workflow, roles, and relationships; have people there check them for agreement.

ent On-the-fly sketches of the physical layout, floor plans (not necessary to be to scale), locations of people, furniture, equipment, communications connections, etc.

ent Quantitative data—for example, how many people do this job, how long do they typically work before getting a break, or how many widgets per hour do they assemble on the average?

Wrap it up

Do not overstay your welcome. Be efficient, get what you need, and get out of their way. Limit interviews to no more than two hours each; everyone is tired after that much concentrated work. At the end, you may wish to give interviewees something as a thank you. Although cash is always welcome, sometimes employers will not like you to pay their employees since in principle they are already being paid for being there. In these cases a “premium gift” is appropriate, such as a T-shirt or coffee mug inscribed with something catchy about the situation.

3.3.3 During the Visit: Collecting User Work Activity Data in the Product Perspective

Roles of users will be different with commercial products. In most cases, work in a domain-complex system context is performed by people in roles that make up the organization, which we will be calling “work roles.” In the setting of a system with a complex work domain, a work role is defined and distinguished by a corresponding job title or work assignment representing an area of work responsibility. For a commercial product, a work role may just be the user.

Work Role

A work role is defined and distinguished by a corresponding job title or work assignment representing a set of work responsibilities. A work role usually involves system usage, but some work roles can be external to the organization being studied.

Usage location will also be different for commercial products. The work or play by individual users of commercial products is not usually connected to an organization. This kind of work or play happens wherever the product is used. For example, if the product is a camera, the work happens pretty much anywhere.

The challenge therefore is being able to collect work activity data as it happens, in the context and location in which it happens, without influencing the user’s behavior. What are the things users do when taking a photograph? With whom do they interact? What do they think about? What concerns and challenges do they have while taking pictures? What are the barriers to, or inconveniences in, doing it the way they want to?

Emotional impact and phenomenological aspects are more prominent with commercial products. A product such as a digital camera is much more likely to generate a strong emotional component within the user experience and even an emotional attachment to the device. What does it mean to the user emotionally to have a compact camera handy at all times?

A product like a digital camera also has more of a chance to be the object of long-term phenomenological acceptance into one’s life and lifestyle. The more people carry the camera with them everywhere they go, the stronger the phenomenological aspects of their usage.

Phenomenological Aspects of Interaction

Phenomenological aspects (deriving from phenomenology, the philosophical examination of the foundations of experience and action) of interaction are the cumulative effects of emotional impact considered over the long term, where usage of technology takes on a presence in our lifestyles and is used to make meaning in our lives.

What does the camera’s brand mean to people who carry it? How about the style and form of the device and how it intersects with the user’s personality and attire? What emotions do the scratches and wearing of edges in an old camera invoke? What memories do they bring to mind? Does the user associate the camera with good times and vacations, being out taking photos with all his or her worries left behind? What does it mean to get one as a gift from someone? What about reuse and sustainability? How can we design the camera to facilitate sharing among friends and social networks?

You may have to observe longer-term usage. It usually takes longer to address these emotional and phenomenological factors in contextual inquiry because you cannot just visit once and ask some questions. You must look at long-term usage patterns, where people learn new ways of usage over time.

MUTTS

MUTTS is the acronym for Middleburg University Ticket Transaction Service, our running example for most of the process chapters.

Example: User Data Gathering for MUTTS

We performed contextual inquiry sessions, interviewing MUTTS employees and customers. We had three analysts separately interviewing several groups of one or two users at a time and came up with a fairly rich set of raw data transcripts. At the end, we also expanded the inquiry by asking customers about experience with other kiosks they might have used.

In most examples throughout this book, we cannot include all the details and you would not want us to. We therefore call on the reader for a kind of dramatic suspension of disbelief. The point of these examples is that it is not about content, especially completeness, which we deliberately abstracted to reduce the clutter of details. It is about simple illustrations of the process.

For simplicity, in most of our examples we will focus on MUTTS customers, whom we interviewed in the context of using the ticket office. Here are paraphrased excerpts from a typical session with a MUTTS customer:

Q: We want to begin with some questions about your usage of the ticket service, MUTTS. What do you do for a living? Tell us about your typical day.

A: I have a 9 to 5 job as a lab technician in Smyth Hall. However, I often have to work later than 5PM to get the job done.

Q: So do you use MUTTS to buy tickets for entertainment?

A: I work long hours and, at the end of the day, I usually do not have the energy to go to MUTTS for entertainment tickets. Because this is the only MUTTS location, I cannot buy tickets during normal working hours, but the MUTTS window is not open after 7PM.

Q: How often and for what have you used the MUTTS service?

A: I use MUTTS about once a month for tickets, usually for events on the same weekend.

Q: What kinds of events do you buy tickets for?

A: Mostly concerts and movies.

Q: Describe the ticket buying experience you just had here at the MUTTS ticket office.

A: It went well except that I was a little bit frustrated because I could not do the search myself for the events I might like.

Q: Can you please elaborate about that?

A: My search for something for this weekend was slow and awkward because every step had to be a series of questions and answers through the ticket seller. If I could have used her computer to browse and search, I could have found what I wanted much sooner. Also, it works better if I can see the screens myself and read the event descriptions. And I also felt I need to answer quickly because I was holding up the line.

Q: Did you know you could search for some of these events on Tickets4ever.com?

A: No, I did not know they had local events.

Q: While you were looking at the seating chart, you seemed unsure about what the ticket seller was expecting you to do with it. Can you please walk us through what you were thinking and how that fit in with the way the seating chart was laid out.

A: Yeah, that was a problem. I could see it was a seating chart but I did not understand what seats were still available and could not quite put the layout of the seats in perspective. I had to ask her what the colors meant on the chart, and what the price difference was for each of those colored regions.

Q: Walk us through a couple of other experiences you have had at the ticket office and do not skip any details.

A: Last week I bought two movie tickets and that was very smooth because I knew what movie I wanted to see and they are usually the same price. Generally, buying movie tickets is very easy and quick. It is only with concerts and special events that things get somewhat complicated. For example, a couple of months ago, I wanted to get tickets to a concert and I could not get to this office for a couple of days because I was working late. When I eventually got here, the tickets were sold out. I had to fill a form over there to get added to a waitlist. I do not know how the waitlist works, and that form was very confusing. Here, let me show you…

Q: What do you like most about MUTTS?

A: Because I am an MU employee, I get a discount on tickets. I also like that they feature the most popular and most current local events.

Q: What do you like least about MUTTS and what concerns do you have about using MUTTS to buy tickets?

A: MUTTS seems to have a limited repertoire of tickets. Beyond the most popular events they do not seem to handle the smaller events outside the mainstream.

Q: What improvements, if any, would you like to see in MUTTS?

A: It would help me if they were open later at night. It would be great if I could get football tickets here, too!

Q: Do you buy football tickets regularly?

A: Yes, I go to about four to five games every season.

Q: Do you buy tickets to any other athletic events? Can you describe a typical transaction?

A: Yes, I also get MU basketball tickets for at least a few games every season. For athletic tickets I have to be on the lookout for the dates when the lottery opens for the games I care about. I sign up for the lottery during the three days they are open and if I win, I have to go all the way to the other side of campus to the MU Athletics Tickets Office. When I am looking to buy tickets to MU basketball, I like to look at different seating options versus prices; I sometimes look for an option allowing several friends to sit together. But that process is very complicated because I have to coordinate lottery signup with some friends. We get to buy only two guest tickets if we win the lottery.

Q: What difficulties do you experience in using MUTTS as the main source of tickets for events?

A: The main problem is that it is too far away from where I live and work.

Because the envisioned kiosk-based ticket system is so different from the existing MUTTS ticket window, we also wanted to get their thoughts on the proposed kiosk system.

Q: Now we want you to imagine a new service where you can buy tickets at public kiosks located across campus and the town. In particular we are planning to have ticket kiosks conveniently located at major bus stops in Middleburg. Have you had any experience with ticket kiosks in other places, other towns?

A: That is interesting! I never bought tickets at a kiosk before.

Q: Have you had any experience with other kinds of ticket kiosks at places like bus stops or in Metro-type commuter train stations in any big city?

A: Yes, I lived in New York for a couple of years and I used the MTA kiosks to buy metro cards all the time.

Q: If we were to put kiosks at places such as university parking lots, the university mall, and other public locations across campus to sell tickets that you get at this office, would you use them?

A: I would be willing to at least try a ticket kiosk located at the Burruss Hall bus stop because I take the bus there every day. I would also try one near the University Mall because I live near there.

 Most of my free time is outside normal business hours, after many businesses are closed, so a kiosk might be convenient.

Q: What type of information would you like to see in such a kiosk?

A: When I look for entertainment options, I want to see the most current events (top picks for today and tomorrow) on the first screen so I can avoid searching and browsing for those.

Q: In your transaction here at the MUTTS office today, you asked if Unspoken Verses is like the Middleburg Poet Boys band. How do you envision getting information like that at a kiosk?

A: That is a good question! I am not sure. I guess the kiosk should have some sort of related items and good description of events. Perhaps even recommendations of some sort.

Q: Can you envision yourself using a kiosk to do what you did today at this office?

A: Yes, definitely. I guess I would expect some form of detailed description of the events. I should be able to look for different types of events. If there are pictures, that would help. I should be able to see a seating chart.

Exercise

See Exercise 3-2, Contextual Inquiry Data Gathering for the System of Your Choice

3.4 Look for emotional aspects of work practice

Look for the impact of aesthetics and fun in work practice, and look for opportunities for more of the same. When you are visiting workplaces, observing work practice, and interviewing people in work roles, you may find that customers and users are less likely to mention emotional aspects of their work practice because they think that is about personal feelings, which they might think is inappropriate in the context of technology and functional requirements.

As a result, you must try harder to uncover an understanding about emotional and social aspects of work practice. For each different work or other role studied in contextual inquiry, try to get at how emotion might play a part. You have to be diligent and observant of little details in this regard.

Look for ways to fight job boredom. Does anyone intimate, even obliquely, that they would like their job to be less boring? What about the work is boring? Where and when are people having fun? What are they doing when they have fun? Where do people need to have fun when they are not?

Where is there stress and pressure? Where can job stress be relived with aesthetics and fun? Where would it be distracting, dangerous, or otherwise inappropriate to try to inject fun or surprise?

What are the long-term phenomenological aspects of usage? What parts of usage are learned over longer times? Where is it appropriate for users to give the system or product “presence” in their lives?

Presence

Presence of a product is a kind of relationship with users in which the product becomes a personally meaningful part of their lives.

3.5 Abridged contextual inquiry process

The full rigorous process for contextual inquiry and analysis is appropriate for domain-complex systems. But the fully rigorous contextual process is not always necessary. Contextual inquiry calls for using good sense and not slavishly following a predefined process. Minimize overlap in raw data collection across interviews. Use your experience to focus on just the essentials.

Another way to abridge your contextual inquiry is to limit your scope and rigor. As an example, we were part of one small project where less than a day’s worth of talking to users about their work practice made a big difference in our understanding of the work domain to inform the design.

One of the most obvious and direct ways to abridge the full contextual inquiry process to save resources is to not make audio or video recordings of the user interview sessions. This also saves resources later in contextual analysis because you do not have to transcribe the recordings.

3.6 Data-driven vs. model-driven inquiry

Beyer and Holtzblatt (1998) take an approach to contextual inquiry and analysis for HCI based on pure ethnographic field research. That is, their process is led entirely by work activity data. Simply stated, letting data do the driving means that if you encounter any information that seems relevant to the work practice and its milieu, collect it. This approach means forestalling any influence from your own knowledge, experience, or expectations and just gathering data as they present themselves.

Data-driven contextual inquiry results in voluminous raw data describing a wide variety of topics. To digest this mass of disparate data points, make sense of them, and put these data to work in informing design, practitioners must apply contextual analysis to extract the concise and meaningful points and issues and then sort and organize them into piles or affinity diagrams. Then the sorted categories must be converted into design-informing models such as flow models, user models, and task models. In the purely data-driven approach, these categories and models are dictated by the data content.

In effect, Beyer and Holtzblatt (1998) recommend not thinking of data categories in advance, but letting data suggest the categories and subsequent models. This will help avoid biasing the process by replacing data from users with analysts’ hunches and opinions. Their “contextual design” approach to contextual inquiry and contextual analysis has proven itself effective.

However, Constantine and Lockwood (1999) show that there is more than one effective approach to gathering contextual data to inform design. They promote a method they call model driven, which is in important ways the reverse of the Beyer and Holtzblatt data-driven approach. In their “use what you know” approach, Constantine and Lockwood advocate using knowledge and expectations from experience, intelligent conjecture, knowledge of similar systems and situations, marketing analysis, mission statements, and preliminary requirements to focus your contextual inquiry data gathering to anticipate preconceived data categories and target the most useful data and to get a head start on its organization and analysis.

From this experience, most practitioners know what kinds of models they will be making and what kinds of data feed each of these models. This knowledge helps in two ways: it guides data collection to help ensure that you get the kinds of contextual data you need, but at the risk of analyst bias in those data. It also helps with analysis by giving you a head start on data categories and models.

Certainly not all of this anticipatory information will be correct for a given work practice or system, but it can provide an advantageous starting point. Experienced professional practitioners, having gone through the contextual inquiry process and having done similar analyses in other work contexts, will learn to get through the chaff efficiently and directly to the wheat.

Although their process might seem that it is about modeling and then finding just the data to support the predefined models, it really is about starting with some initial “exploratory” models to guide data collection and then focused data collection to find answers to questions and outstanding issues, to refine, redirect, and complete the models. This “model-driven inquiry” approach also has a solid real-world track record of effectiveness.

The Beyer and Holtzblatt contextual design approach works because, in the end, data will determine the truth about work practice in any specific real-world customer or user organization. However, the Constantine and Lockwood approach works because it encourages you to use your experience and what you know to anticipate data needs in contextual inquiry and contextual analysis. While data-driven inquiry assumes a “blank slate” and a completely open mind, model-driven inquiry acknowledges the reality that there is no such thing as a blank slate (Constantine & Lockwood, 1999).

The Beyer and Holtzblatt approach is rooted in real data untainted by guesswork or analyst biases. But the Constantine and Lockwood approach claims advantages in lower cost and higher efficiency; they claim the search for data is easier if you know something about what you are looking for. To them, it is about pragmatically reducing the ratio of data volume to insight yielded.

The Value of Contextual User Studies in Understanding Problem Causes

Jon Meads, President/Principal Consultant, Usability Architects, Inc.

Introduction

This is a case study that exemplifies the integration of user studies with agile development. In an agile development environment, the usability engineer may be the user’s representative, part of the design team, and, often enough, both.

In order to inform the design and represent the user, the usability engineer needs to understand not only the user requirements, but also the business requirements and process. Also, it is very possible that neither the business requirements nor the current process is as well known as it should be. This case study depicts such a situation.

The Problem

I was working with an insurance claims processing company as a consultant. Their problem was that there was a high turnover of adjudicators. The adjudicators had the responsibility of reviewing all claims that the automatic processing had rejected and making a final determination on rejection or payment. The adjudicators were skilled workers and required about 6 weeks of training followed by several months of experience to get to the level of performance that the company required of them.

However, the work was both tedious and demanding, and the turnover of adjudicators was relatively high, as was the case for most of their clerical staff. The company asked me if I could redesign the user interface to make the process easier to learn so that new adjudicators could be brought on in less time.

Understanding the Problem

As is usually the case for consultants, you work for a variety of clients in a variety of business sectors. The insurance business was completely new to me and there was a lot to learn. Management was able to explain to me what the responsibilities of the adjudicators were and what the management problem was. The company’s business analysts provided me with an overview of the adjudication issues and pointed me to the policy manual, an online reference document with much more information than I could possibly absorb in a reasonable amount of time. The policy manual was the adjudicator’s “bible” and becoming familiar with it was essential for them to do their work.

Management thought that something might be done to make finding the desired information in the bible easier, which would help the adjudicators. But discussions with the adjudicators did not reveal any significant problems in finding the information they needed and did not mention it as being a problem for either doing their work or in becoming proficient at their work. At this point I had no idea of what could be designed that would reduce the amount of training required.

There were two things still to do. One was to talk with the trainers to find out what they perceived to be the reason it took new adjudicators 6 months to become proficient. The other was to spend some time observing the adjudicators doing their work. Discussions with the trainers provided the first indication of what the problem really was and where the solution might lie and the observations confirmed it.

The trainers stated that the actual task of adjudicating the claims was something that was learned in a couple of weeks. The remainder of training time was spent learning where to get the information relative to the claim that would support a decision on whether to allow the claim. Watching the adjudicators doing their work showed that they were constantly pulling up new data screens, switching back and forth among the screens, and making notes on scraps of paper.

After spending some time observing the adjudicators at work, the real problem became evident. The current user interface was based on the structure of the underlying database of claims, referrals, subscribers, and providers. To resolve an issue, adjudicators needed to immerse themselves in the database, searching for information about past claims (“encounters”), referrals, provider/clinic associations, and other pieces of data that would allow them to determine if the claim was covered or not. To do this, they were constantly navigating a complex database, pulling up screens full of data to find one or two items of interest.

It was not the training that was a problem or difficulty with the policy manual. The root problem was that they were doing the work of the computer system, sifting through large amounts of data to find those items that were pertinent to resolving the claim. Contextual research showed that the information needed to resolve a claim could be diagrammed as an object model. This model showed the needed information as well-defined objects and what the data relationships were from the perspective of the adjudicators.

I was also able to determine that the process of adjudicating a claim had three basic activities:

i. determining if a referral is needed

ii. matching a referral to a claim

iii. paying, denying, or forwarding the claim

Design and Iteration

Although the process was usually fairly linear, the adjudicator would sometimes need to switch from one activity to another to check up on an item or resolve a minor issue. However, the recognition of these activities as constituting the process allowed for development of a simple conceptual model:

image

where the information the users were previously writing down as notes was consolidated and kept visible as “Encounter Data.” Selecting the tabs in the upper right would bring up tools and data needed for the specific activity the adjudicator was currently engaged in.

The conceptual model, above, was validated (“tested”) by several adjudicators and adjusted to make access to data being sought during the referral matching activity easier and more straightforward.

It was at this point that we entered the agile phase of development. We developed an initial working prototype that fleshed out what data should be presented along with where and how it was presented and then went through several iterations of programming and designing of the prototype, changing data that were presented, and adjusting the placement of data and the mechanisms used to present it. These intermediate prototypes were reviewed with a select group of adjudicators until we had a final version that most everyone was satisfied with. At this point, we let the graphic designer clean it up and make it more attractive. Being an in-house application, our graphic design goals were aesthetic: to provide a display that was clean in appearance and comfortable to view and work with.

Success Measures

The final check was to validate the design with measures on time to train and productivity. We checked expected training time by simply allowing novice adjudicators to use the new design to adjudicate a number of claims with only a simple introduction to it. We first measured their performance using the current system and then measured their performance with the new system. During the first 30 minutes of using the new system, claim resolution time was approximately 20% longer than their performance with the old system. During the second 30 minutes with the new system, they were averaging 20% less time than with the old system. By the end of 90 minutes use of the new system, adjudicators were resolving claims in about one-third of the time that they did with the old system.

Since it was the task of finding the information needed to resolve a claim that required 6 months of experience to become proficient, we were comfortable that the new system would not only improve productivity but reduce the time it took to train adjudicators and bring them to an acceptable level of proficiency.

3.7 History

3.7.1 Roots in Activity Theory

First of all, we owe a great acknowledgment to those who pioneered, developed, and promoted the concepts and techniques of contextual design. Early foundations go back to Scandinavian work activity theory (Bjerknes, Ehn, & Kyng, 1987; Bødker, 1991; Ehn, 1988). The activity theory work was conducted for quite some time in Scandinavia, in parallel with the task analysis work in Europe and the United Kingdom. More recent conferences and special issues have been devoted to the topic (Lantz & Gulliksen, 2003). Much of the initial work in this “school” was directed at the impact of computer-based systems on human labor and democracy within the organizations of the affected workers. This singular focus on human work activities shines through into contextual inquiry and analysis.

Work Activity Theory

Work activity theory in HCI stemmed from a democratic movement that flourished in Scandinavia during the 1980s. It emphasized human labor and human activities as complex, goal-directed and socially situated phenomena mediated by tool usage.

3.7.2 Roots in Ethnography

A second essential foundation for contextual inquiry is ethnography, an investigative field rooted in anthropology (LeCompte & Preissle, 1993). Anthropologists spend significant amounts of time living with and studying a particular group of humans or other possibly more intelligent animals, usually in social settings of primitive cultures. The goal is to study and document details of their daily lives and existence.

In a trend toward design driven by work practice in context, quick and dirty varieties of ethnography, along with other hermeneutic approaches (concerned with ways to explain, translate, and interpret perceived reality) (Carroll, Mack, & Kellogg, 1988), have been adapted into HCI practice as qualitative tools for understanding design requirements. Contextual inquiry and analysis are examples of an adaptation of this kind of approach as part of the evolution of requirements elicitation techniques.

The characteristics that define ethnography in anthropology are what make it just right for adaptation in HCI, where it takes place in the natural setting of the people being studied; it involves observation of user activities, listening to what users say, asking questions, and discussing the work with the people who do it; and it is based on the holistic view of understanding behavior in its context.

In contrast to long-term field studies of “pure” ethnography, with its cultural, anthropological, and social perspectives, the “quick and dirty” version of ethnography has been adapted for HCI. Although involving significantly shorter time with subjects and correspondingly less depth of analysis, this version still requires observation of subjects in their own environment and still requires attending to the sociality of the subjects in their work context (Hughes et al., 1995). For example, Hughes et al. (1994) describe application of ethnography in the area of computer-supported cooperative work (CSCW), a sub-area of HCI.

Lewis et al. (1996) describe an ethnographic-based approach to system requirements and design that parallels much of the contextual inquiry process described here. Rogers and Belloti (1997) tell how they harnessed ethnography as a research tool to serve as a practical requirements and design process. Blythin, Rouncefield, and Hughes (1997) address the adaptation of ethnography from research to commercial system development.

3.7.3 Getting Contextual Studies into HCI

The foundations for contextual design in HCI were laid by researchers at Digital Equipment Corporation (Whiteside & Wixon, 1987; Wixon, 1995; Wixon, Holtzblatt, & Knox, 1990). By 1988, several groups in academia and industry were already reporting on early contextual field studies (Good, 1989) in the United States and the United Kingdom (notably the work of Andrew Monk). Similar trends were also beginning in the software world (Suchman, 1987). Almost a decade later, Wixon and Ramey (1996) produced an edited collection of much more in-depth reports on case studies of real application of contextual studies in the field. Whiteside, Bennett, and Holtzblatt (1988) helped integrate the concept of contextual studies into the UX process.

3.7.4 Connections to Participatory Design

Contextual inquiry and analysis are part of a collection of collaborative and participatory methods that evolved in parallel courses over the past couple of decades. These methods share the characteristic that they directly involve users not trained in specialized methods, such as task analysis. Among these are participatory design and collaborative analysis of requirements and design developed by Muller and associates (1993a, 1993b) and collaborative users’ task analysis (Lafrenière 1996).

1 The term “missionary” referred to his commitment to educate his customers about their own needs. While he aimed to serve his clients’ needs, he felt he was the only authority on determining those needs.

2 Study of the problem of measurement in quantum mechanics has shown that measurement of any object involves interactions between the measuring apparatus and that object that inevitably affect it in some way. − Wikipedia.com

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.141.47.51