The existing system: The Middleburg University Ticket Transaction Service

Middleburg, a small town in middle America, is home to Middleburg University, a state university that operates a service called the Middleburg University Ticket Transaction Service (MUTTS). MUTTS has operated successfully for several years as a central campus ticket office where people buy tickets from a ticket seller for entertainment events, including concerts, plays, and special presentations by public speakers. Through this office MUTTS makes arrangements with event sponsors and sells tickets to various customers.

The current business process suffers from numerous drawbacks:

Organizational context of the existing system

The desire to expand the business coincides with a number of other dynamics currently affecting MUTTS and Middleburg University.

The proposed new system: The Ticket Kiosk System

The Middleburg University Ticket Transaction Service (MUTTS) wants to expand its scope and expand to more locations, but it is expensive to rent space in business buildings around town and the kind of very small space it needs is rarely available. Therefore, the administrators of MUTTS and the Middleburg University administration have decided to switch the business from the ticket window to kiosks, which can be placed in many more locations across campus and around town.

Middleburg is home to a large public university and has reliable and well-used public transportation provided by its bus system operated by Middleburg Bus, Inc. There are several bus stops, including the library and the shopping mall, where there is space to add a kiosk for a reasonable leasing fee to the bus company.

A number of these bus stops seem good locations for kiosks; buses come and go every few minutes. Some of the major stops are almost like small bus stations with good-sized crowds getting on and off buses.

In addition to an expected increase in sales, there will be cost savings in that a kiosk requires no personnel at the sales outlets. The working title for the new system is Ticket Kiosk System, pending recommendations from our design team. The Ticket Kiosk System will have a completely new business model for the retail ticket operation.

Learn about your customer organization before the visit

Preparation for the visit means doing upfront planning, including addressing issues such as these about the customer:

Learn about the domain

While designing for complex and esoteric domains, working first with subject matter experts helps shorten the actual contextual inquiry process by giving you a deeper understanding of the domain, albeit from a non-user perspective. Your contextual inquiry process can now include validating this understanding. In cases where time and resources are at a premium (not an insignificant portion of projects in the real world), you may just have to make do with just interviewing a few subject matter experts instead of observing real users in context.

Issues about your team

In addition, there are issues to address about your team:

Your visit-group size can depend on how many are on your initial project team, the number of different user roles you can identify, the size of the project overall, the budget, and even your project management style. Practitioners report taking as many as two to eight or more people on visits, but three to four seems to be typical.

A multidisciplinary team is more likely to capture all necessary data and more likely to make the best sense of data during subsequent analysis. We have found using two people per interview appealing; one to talk and one to take notes.

Lining up the right customer and user people

Among the things to do to prepare for a site visit for contextual inquiry, you should:

This latter item, selecting the people to meet, observe, and interview, is especially important. Your fieldwork should include all work roles, selected other stakeholders who impact work directly or indirectly, and (depending on the project) possibly grand-customers (customers of the customer) outside the user’s organization. You want the broadest possible sources to build a holistic and multi-perspective picture of the contextual data.

Get access to “key” people

For projects in a domain-complex system context, you might also be told by your customer that users of the system in question are scarce and generally unavailable. For example, management might resist giving access to key people because they are busy and “bothering” them would cost the organization time and money.

If you sense reluctance to give access to users, you need to step up and make the case; establish the necessity for gathering requirements that will work and the necessity for firmly basing requirements on an understanding of existing work activities. Then explain how this extra work upfront will reduce long-term costs of reworking everything if analysts do not get the right requirements. Ask for just a couple of hours with key users. Persevere.

At the other end of the spectrum, for consumer software, such as shrink-wrap word processors, users abound and you can recruit users to interview via a “help wanted” ad posted in the local grocery store.

Do not interview only direct users. Find out about the needs and frustrations of indirect users served by agents or intermediaries. And do not forget managers. Here is a quote from a team that we worked with on a project, “It was eye-opening to talk with the managers. Managers are really demanding and they have different kinds of requirements from those of the users, and they see things from a totally different viewpoint than the other users.”

Sometimes you may have access to the users for only a small period of time and therefore cannot observe them doing work. In such cases, you can ask them to walk you through their typical day. You must work extra hard to ask about exceptions, special cases, and so on. This approach suffers from many of the problems we described earlier regarding not observing users in context but at least provides some insights into user’s work.

What if you cannot find real users?

In the worst case, that is, when you have no access to real users (this has happened in our consulting and work experience), the last resort is to talk to user proxies. User proxies can be business experts or consultants who are familiar with the user’s work.

This approach suffers from many disadvantages and often results in hearing about high-level functional needs and crude approximations of what a broad class of users need in the system. The accounts of such proxies are often tainted by their own opinions and views of the work domain. They also suffer from serious omissions and simplifications of often nuanced and complex user work activities.

Setting up the right conditions

The environment, the people, and the context of the interview should be as close a match to the usual working location and working conditions as possible. We once found ourselves being ushered into a conference room for an interview because, as the employer put it, “it is much quieter and less distracting here.”

The employer had even arranged for time off from work for the worker so that he could focus his complete attention on the interview. But, of course, the conference room was not anything like the real work context and could not possibly have served as a useful source of information about the work context. We had to convince them to move the whole thing back into the active workplace.

Make sure that the observations and interviews are conducted without undue political and managerial influences. You want to create the right conditions for observation and interviews, conditions in which users feel comfortable in telling the “real” story of the everyday work practice. We once had to deal with the supervisor of a person we wanted to interview because the supervisor insisted on being present during the interview. His reason was that it was a rare opportunity to learn more about what his workers did and how.

However, we also suspected that the supervisor did not want the employee to be complaining to strangers about working conditions or the organization. However, from the worker’s view, having a supervisor present looked a lot like an opportunity for the supervisor to evaluate the user’s job performance. It meant not being able to be open and candid about much of anything. Instead, the employee would have to pay very close attention to doing the job and not saying anything that could be interpreted in a way that could be used against him. It would be anything but a sample of everyday work practice.

How many interviewees at a time?

It might work out that, via a group interview, multiple users can work together and produce data not accessible through a single user. However, group interviews can also mask individual thoughts. Each user may have a very different view of how things work and what the problems are, but these differences can be sublimated in an unconscious effort to reach “consensus.” Additionally, group dynamics may be influenced by hidden agendas and turf battles.

Preparing your initial questions

Script your initial interview questions to get you off to a good start. There is no real secret to the initial questions; you ask them to tell you and to show you how they do their work. What actions do they take, with whom do they interact, and with what do they interact? Ask them to demonstrate what they do and to narrate it with stories of what works, what does not work, how things can go wrong, and so on.

We found that instead of asking them generally “What do you do here?” it is sometimes more helpful to ask them to walk us though what their work specifically entailed the day before and if that was typical. This kind of a specific probing gives them an easy point of reference to make their descriptions concrete.

Before the visit: Preparation for the product perspective

While the aforementioned guidelines for preparing a visit in a domain-complex system context generally also apply to a product perspective, there are a few differences. For one, the context of work in a product design perspective is usually much narrower and simpler than that in an entire organization. This is primarily because organizations contain numerous and often widely different roles, each contributing to a part of the overall work that gets accomplished.

In contrast, the work activities within a product design context are usually centered on a single user in a single role. To observe the full range of usage patterns by a single user of a product, you usually have to observe their usage over a long time. In other words, to do this kind of contextual inquiry, instead of observing several users in a work role for a relatively short time, you have to “shadow” single users over a longer time.

For example, the work, or play, activities associated with a portable music player system sometimes include searching for and listening to music. At other times the same user is trying to manage music collections. Even in cases where the design needs to support multiple users, say the user’s family, the complexity of the interaction among different roles is usually much lower in the product perspective, and often more homogeneous than in a domain-complex system perspective.

Where do we start with our contextual inquiry process for such products? The best place to start is by understanding the complete usage context of this kind of product, including desirable features and limitations.

We also have to ask about things such as branding, reputation, and competition in this product segment. To find unbiased information about these issues, instead of looking online for the customer’s organizational policies and culture, we need to look for user groups or blogs about using this kind of product and check out reviews for similar products.

Do some initial brainstorming to see what kinds of user communities are primary targets for this product segment. College students? Soccer moms? Amateur photographers? Then think of good places to meet people in these user classes. If necessary, use marketing firms that specialize in recruiting specific target populations.

Anticipating modeling needs in contextual inquiry: Create contextual data “bins”

There is a spectrum of approaches to contextual data collection from data driven to model driven. We draw on the best of both but lean toward the model-driven approach. A data-driven approach operates without any presuppositions about what data will be observed. There are no predefined data categories to give hints about what kind of data to expect. The data-driven approach simply relies on data encountered to guide the process of data gathering and subsequent analysis. Whatever arises in contextual inquiry observations and interviews will define the whole process.

Alternatively, a model-driven contextual inquiry process means that instead of just gathering relevant data as you encounter it in observations and interviews, you use your experience to help guide your data collection. In particular, you use the known general categories of design-informing models (Chapter 6) as a guide for kinds of data to watch for, looking forward to the data needs of modeling so that at least some of your data collection in contextual inquiry can be guided by these future needs.

From your knowledge you will have a good idea of which models will be needed for your project and what kind of data will be needed for your models. Using this knowledge, you create some initial “bins” for data categories and models into which you can start putting your contextual data, in whatever form it has at this point. A bin is a temporary place to hold data in a given category. As you collect data, you will think of other categories to add more bins.

For example, we will cover construction of what we call a physical model (Chapter 6), which includes a diagram of the physical layout of the working environment. So, if a physical model is relevant to your project, then you will need to make a sketch and/or take photos of the physical layout, where equipment is located, and so on while you are still on-site doing contextual inquiry. In order to meet those modeling needs later, you will also need to take notes about the physical layout and any problems or barriers it imposes on the work flow and work practice.

In the next chapter we will extend and complete the creation of bins for sorting and organizing your data in contextual analysis.

When you first arrive

Begin your first field visit by meeting the manager, administrator, or supervisor through whom you arranged the visit. Continue with the building of trust and rapport that you started previously. Make it clear that you are doing this for the purpose of helping make a better design. It is a big advantage if, at the beginning, you can briefly meet all customer personnel who will be involved so that you can get everyone off to the same start by giving the overview of your goals and approach, explaining what will happen during your visits and why.

Remember the goal

Often in field visits for “talking with our users,” people ask users what they want or need. In a contextual inquiry, we do not ask users what they want or need. We observe and interview users in their own work context about how they do their work as they are doing the work that later will be supported by the system you hope to design.

And, by all means, do not forget that we are on a quest for a quality user experience. The techniques of contextual inquiry and contextual analysis will not necessarily take care of searching out the special aspects of user experience; you have to be tuned to it. So be especially aware of user stories about particularly good or bad usage experiences and ferret out the reasons, both in design and in usage context, that contribute to those experiences.

Establish trust and rapport

The interviews with users should also start with trust building and establishing rapport. Help them understand that you have to ask many questions and “get in their face” about the work. Interviewing skills are learned; observe users doing their work and ask many questions about why they do something, how they do certain things, how they handle certain cases, and get them to tell specific stories about their work and how they feel about the work and the way it is done in the existing environment. Follow them around; do not miss a part of the work activity by staying behind if they have to move as part of their work.

Form partnerships with users

In most consulting situations the person who comes in from outside the customer organization is considered the “expert,” but it is quite the opposite in contextual inquiry. The user is the expert in the domain of work practice and you are the observer trying to understand what is happening.

The process works best if you can form a partnership with each user in which you are co-investigators. Invite the user into the inquiry process where you can work together in close cooperation. You need to establish an equitable relationship with the user to support information exchange through a dialog.

As the observations and interviews proceed you can feed the partnership by sharing control of the process, using open-ended questions that invite users to talk, for example, what are you doing? Is that what you expect? Why are you doing that? Be a good listener and let the user lead the conversation. Pay attention to nonverbal communication.

Task data from observation and interview

One of the most important kinds of contextual data to collect is task data. You will need this to build your task structure models and task interaction models in Chapter 6. This is where a combination of observation and interview can work especially well. Get task-related data by observing actual sessions of users doing their own work in their own work context.

At the same time, you will interview the users, but asking only about the task they are doing, not about possible tasks or tasks that other users do. The interview component is used to interpret and assign meaning to what is observed. To have necessary data for task models later, ask questions to clarify anything not obvious in what you observe. Ask questions about the purposes and rationale for each task and each important step; why do they do certain actions?

On the observation side of things, be sure to notice the triggers for tasks and steps; what happens to cause them to initiate each task or step? For example, an incoming phone call leads to filling out an order form.

Learn about your users’ task barriers by observing tasks being performed and by think-aloud verbal explanation of underlying information about the tasks, such as task goals. Notice hesitations, problems, and errors that get in the way of successful, easy, and satisfying task or step completion. Probe for what was expected and reasons why it did not turn out well. You will need these answers to model barriers in the upcoming analysis and modeling.

It takes a certain skill to key in on occurrences of critical information in the flow of observation and interviews. With practice, you will acquire an awareness and ability to detect, discern, and discriminate the wheat from the chaff of the flow.

The output of this process is raw user work activity data in the form of lengthy observation and interview notes or transcripts of recorded sessions.

Recording video

Video recording is an effective way of comprehensively capturing raw contextual data where conditions and resources permit. Video recording can help you capture important nonverbal communication cues in contextual data.

However, factors such as the time and effort to produce and review the recordings can weigh against the use of video recording in contextual inquiry data collection. Confidentiality, privacy, and other such concerns can also preclude the use of video.

In addition, video recording can interfere with a close user relationship. The feeling that what they say is permanently captured may prevent them from being forthcoming. They may not be too willing to say something negative about existing work practice or complain about policies at work. Informal note taking, however, can provide a more intimate conversational experience that may encourage honest expression. Despite all these possible barriers, video clips can provide convincing evidence, when linked to the contextual notes, that an issue is real.

Note taking

Regardless of whether you use video or audio recordings of observation and interview sessions, you should consider note taking your primary source of usable raw data. Manual paper note writing may be the most commonly used contextual inquiry data collection technique used in the real world. It is unintrusive, not intimidating to the user, and fits most naturally into what should be a more or less low-key interaction with the user. Alternatively, a laptop is acceptable for note taking, if you can do it inconspicuously.

When taking notes, you must incorporate some scheme that preserves information about the source of each note. We recommend that you use:

A small handheld digital audio recorder used inconspicuously, but not trying to be covert, might be beneficial to augment note taking, especially when things are moving fast. One way to use audio recording is as the audio equivalent of written notes.

In this mode, after hearing user comments or observing user behavior, you dictate a short summary into the recorder, much as a medical doctor dictates summaries for patient charts during daily rounds. This mode of operation has the additional benefit that if the user can hear you paraphrase and summarize the situation, it is a chance to correct any misconceptions.

Use a numbering system to identify each point in data

It is important to use a numbering system to identify uniquely each note, each point in the raw data, or each sequence in a video or audio recording or transcript. This last item is necessary to provide a way to reference each note. Later, in analysis, each conclusion must be linked to the associated raw data note or else it cannot be treated as authentic. Some of the ways to tag your raw data for reference in analysis include the following:

How to proceed

Record raw data by expressing it in the user’s voice. Because these data are raw, it makes sense to express points in the interview transcripts generally as they occurred during the interview, which would usually be by using the words of the user. For example, the statement: “I like to add an extra egg when I make a cake” naturally reflects the fact that a user is speaking. If you record your interviews, the transcripts will mostly appear as the exact words of the user anyway.

Switching to an expression such as “the user likes to add an extra egg when baking a cake” unnecessarily introduces an analyst flavor that is not useful this early in the process. Moreover, the user’s voice describes much closely the user’s experience, and subtle use of adjectives and expressions can provide clues on designing for enhancing that experience.

It is your job to capture data about the user’s work. Do not expect users necessarily to tell you what they want or need; this is just about how they work and how they feel about it. Your team will deduce needs later, after they understand the work. Also, do not expect users to do design, although they might occasionally suggest something they would like to see in the system.

Part of being a good detective, the latter point above, is being willing to deviate from a script when appropriate. Be prepared to follow leads and clues and take the interview and observations where you need to go, tailoring questions to meet the goal of learning all you can about their work practice, work environment, and work issues and concerns.

As an example of following leads, this is a real story told by a team doing a project for one of our classes. The client was in retail sales and the conversation of the interview had centered on that concept, including serving their customers, making the sale transaction, and recording it.

However, during this conversation the word “inventory” was mentioned once, in the context of point-of-sale data capture. No one had asked about inventory, so no one had mentioned it until now.

Our good ethnographic detectives, recognizing an entree to another area of work activities, pounced on that word and pursued a new train of thought. What about inventory? What role does it play in your point-of-sale data capture? Where does it go from there? How is it used and for what? How do you use inventory data to keep from running out of stock on items in demand? Who orders new stock and how? Once an order is sent, how do you keep track of it so it does not fall through the cracks? What happens when the new stock is delivered? How do you know when it arrives? Who works in receiving and what do they do? How do you handle partial shipments?

As an example of dialogue that violates the point above about not introducing your own perspectives, consider this user comment: “I want privacy when I am buying tickets.” You might be tempted to say: “You mean, when you are looking for events and buying tickets, you do not want other people in line to know what you are doing?” To which the user might respond: “Yes, that is what I mean.” A better way to handle the user’s comment here would have been with a follow-up question such as “Can you elaborate what you mean by wanting privacy?”

Pay attention to information needs of users

As you talk with users in the work roles, try to identify their information needs in the context of the work activities and tasks, as they do their jobs in the work domain. Do the current work practices and the current software systems provide information needed by users to do their jobs? Is the needed information provided at the time it is needed and in the form it is needed? And beware of “information-flooding screens.”

When designers do not know what users need, they often fall back on the unjustifiable excuse that the users themselves will know what they need. These designers then create designs that display all information available or all the information users might need, in an “information flooding screen,” and let the users sort it out. The designer’s assumption is that all the information needed is presented—the “it is all there” syndrome—and the users are in the best position to know which parts are needed for which functions/tasks and what format is best for the job. This is a thinly veiled copout for not doing the necessary upfront analysis to inform the design.

What about design ideas that crop up?

Contextual inquiry is not about design, but you do not want to lose any good ideas, so you should make note of design ideas from users as they come up and then get back to asking about work practice. It is normal for users to suggest design ideas, often very specific and sometimes not very practical. It is the interviewer’s responsibility to take note of these suggestions, but to ask more questions to connect them back to work practice. Ask “why?” How does that suggestion fit into your workflow? What part of your work leads to a need for this?

What about analyst and designer ideas that crop up?

Similarly, make note of design ideas from your own team and tag them as such. Just as with users, it is normal for analysts to get design ideas during interviews or during subsequent analysis activities.

Because such suggestions can introduce analyst bias into what is supposed to be all about user data, “righteous” analysts may want to ignore them. But even analyst ideas generated in the action of contextual inquiry are real data and it would be a shame to lose them. So to include analyst and designer data in contextual inquiry, we suggest getting user confirmation by asking about these ideas and keeping clear the source; be sure to label or tag such notes as analyst ideas.

Questions not to ask

Do not expect you can ask customers and users for direct answers to the questions that will lead you straight to design. Remember that contextual inquiry is often called the process for discovering what users cannot tell you. In his “column” on the User Interface Engineering Website, Jared Spool (2010) advises us about three specific questions not to ask customers or users during field visits. We summarize the three no-no questions here:

Collect work artifacts

During site visits collect as many samples of work artifacts, such as paper forms, templates, work orders, and other paperwork, as you can. Work artifacts include not just paperwork, but all items used in the work practice and photos of the same.

For example, consider the keys to a customer’s car in an auto repair facility. First, they may be put in an envelope with the work order, so the mechanic has the keys when needed. After repairs, the keys are hung on a peg board, separate from the repair order until the invoice is presented to the customer and the bill is paid. Artifacts include physical or electronic entities that users create, retrieve, use or reference within a task, and/or pass on to another person in the work domain. This passing of artifacts should also show up in the flow model.

Other forms of data collection

Other kinds of contextual data are also essential in representing work context, including:

Wrap it up

Do not overstay your welcome. Be efficient, get what you need, and get out of their way. Limit interviews to no more than two hours each; everyone is tired after that much concentrated work. At the end, you may wish to give interviewees something as a thank you. Although cash is always welcome, sometimes employers will not like you to pay their employees since in principle they are already being paid for being there. In these cases a “premium gift” is appropriate, such as a T-shirt or coffee mug inscribed with something catchy about the situation.