Chapter 5. User and Customer Research

Doctor: What are you doing here, honey? You’re not even old enough to know how bad life gets.

Cecilia: Obviously, Doctor, you’ve never been a thirteen-year-old girl.

—The Virgin Suicides

As a designer, you have an enormous, exciting responsibility. You define the human world, one object or system at a time. Every delightful and every frustrating artifact, every unseen algorithm that governs interactions, every policy constraining choices, exists because of a series of design decisions.

Design as a job is similarly delightful and frustrating. Whatever you create or specify has implications for a diverse array of people who may not be anything like you. Your work must be sufficiently novel to attract attention or provide value while fitting into each user’s existing world of objects and situations, over which you have no control. How do you identify one design that solves a problem for an endless combination of people and environments?

You do user research to identify patterns and develop empathy. From a designer’s perspective, empathy is the most useful communicable condition: you get it from interacting with the people you’re designing for.

When we talk about user research as distinguished from usability testing, we’re talking about ethnography, the study of humans in their cultural and social contexts. We want to learn about our target users as people existing in their habitual environments. We want to understand how they behave and why.

This is very different from gathering opinions. It isn’t just surveying or polling. And it’s definitely not focus groups.

Ethnographic design research allows your design team to:

• understand the true needs and priorities of your customers/readers/target audience/end users;
• understand the context in which your users will interact with what you’re designing;
• replace assumptions about what people need and why with actual insight;
• create a mental model of how your users see the world;
• create design targets (personas) to represent the needs of your users in all decision-making; and
• hear how real people use language to develop the voice of the system and ensure that the interface is meaningful to them.

Everything in Context

For you to design and develop something that appeals to real people and reflects their priorities, you’ll need to talk with or observe representative users directly in their contexts—their physical environments, mental models, habits, and relationships. This reduces your risk of making bad assumptions based on your own experiences, hopes, or subjective preferences.

Physical environment

This is the physical context in which someone will use your product or service. This could be in an office at a standing desk, at home on the sofa, outside at a job site, or on the train in an unfamiliar city. Is your target user likely to be alone, or surrounded by others, subject to interruptions? Needs—and the best ways to meet them—can change vastly with setting.

Mental model

A mental model is an individual’s preexisting internal concept of, and associations with, any given institution, system, or situation. Every one of us has an imperfect, idiosyncratic map of reality in our head. Without it, we would be utterly lost. With it, we rely on assumptions based on previous experiences we consider analogous. The better the analogy, the more useful the map. This is why interfaces that strive for novelty are often unusable. With no hooks into an existing mental model, we have to figure things out from scratch. And that is very difficult. Designers often make it unnecessarily so.

Habits

How does the user already solve the problem you are trying to solve for them (if indeed they do)? What are their physical and mental habits around the problem, and their relevant beliefs and values? We frequently hear from entrepreneurs who are trying to create a habit around a new product. Habits are hard to change, as anyone trying to kick one will attest; inserting a new hook into an existing habit is much easier.

Relationships

Social networks are merely the most obvious intersection of human relationships and digital products. People are social animals and every interactive system has an interpersonal component. Despite the solo nomenclature of “user experience” design, the use of your product or service will likely involve a web of human relationships.

Assumptions Are Insults

There are over seven billion people on the planet. According to a 2018 report by the International Telecommunications Union, about half of them have no internet access (http://bkaprt.com/jer2/05-01/). Wrap your head around that. That’s nearly four billion people who have never even received the $500 chocolate chip cookie recipe. You probably start getting itchy two minutes after your iPhone dies at TED and you can’t text your husband anymore.

See what I did there? I just made some assumptions about you. If they were correct, maybe you nodded slightly and moved on without noticing. However, if you don’t have an iPhone or don’t go to TED or don’t have a husband with whom you’re constantly exchanging messages, or if you have no idea what the $500 cookie recipe is, you probably got a little annoyed.

When you make assumptions about your users, you run the risk of being wrong. When you embed wrong assumptions in the design of your product or service, you alienate people—possibly before they even have a chance to hear what you have to offer. The more obvious that wrong guess is, the more annoying it is.

“Annoying” might be a generous description. By designing for yourself or your team, you are potentially building discrimination right into your product. Your assumptions about the age, gender, ethnicity, sexuality, and physical or cognitive abilities of your users might lead to barriers you don’t actually intend—barriers that don’t serve your business goals or ethics.

Not all products need to be all things to all people. However, every design decision should be well-informed and intentional, welcoming your intended users rather than alienating or upsetting them. That’s why identifying and understanding your target audience or user base is the most important design research you will do.

As the original motivational speaker, Dale Carnegie used to say (while getting rich saying it):

You can close more business in two months by becoming interested in other people than you can in two years by trying to get people interested in you.

Getting Good Data from Imperfect Sources

It seems like a simple formula:

1. If your goal is to make things people buy and use, you should design what people want.
2. To do that, you need to know what people want.
3. So just find some people and ask them what they want.
4. Then go off and make what they tell you.

No. This does not work. The first rule of user research: never ask anyone what they want.

You know what people want? People want to be liked. When you ask someone directly what they want, it is very possible the answer you receive will be what they think you want to hear, or the answer that reflects how they like to think of themselves. And because it’s impossible to want what you can’t imagine, you risk the scope of your ideas being limited by the imaginations of others.

The television show House M.D. actually makes a terrific case for ethnographic research, as long as you ignore certain ethical and medical realities. In each episode, Dr. Gregory House and his diagnostic team tackle a mysterious, challenging, life-or-death case. Examining and directly questioning the patient leads only to one false diagnosis and subsequent dramatic defibrillation after another, until finally a couple of comely physicians resort to breaking into the patient’s home and snooping around to discover evidence of undisclosed habits and behaviors. They return with artifacts. House has an epiphany. Patient lives! Awkward conversation with loved ones about habitual talcum powder huffing or previous traveling circus career ensues.

“Everybody lies” was the perennial theme and occasional tagline of the show. Not only are most people straight-up craven dissemblers, but even those we would call perfectly honest lack sufficient self-knowledge to give a true account.

It may seem a harsh maxim for the designer who genuinely wants to empathize with users, but it is far more succinct and memorable than “most people are poor reporters or predictors of their own preferences and behavior when presented with speculative or counterfactual scenarios in the company of others.”

Your challenge as a researcher is to figure out how to get the information you need by asking the right questions and observing the right details.

You won’t be breaking into anyone’s house. You need to figure out how to break into their brain—only after being invited, of course, like a vampire of the mind. If you go in through the front door, asking direct questions, you’ll run into defenses and come up with pat, and potentially useless, answers.

The questions you ask directly and the questions you want answered are two different sets of questions. If you ask “Would you spend $50 on my product?” you’ll be truly misled.

If you want to know what people might do in the future, you need them to tell you true stories about what they’ve done in the past. To create a good fit between what you’re designing and what your target users need, you have to know about the aspects of their habits, behaviors, relationships, and environment that are relevant to your work, and then turn that knowledge into insights you can act on. These insights will allow you to design with more confidence and less guesswork.

What Is Ethnography?

Ethnography is a set of qualitative (descriptive rather than measurable) methods to understand and document the activities and mindsets of a particular cultural group who are observed going about their ordinary activities in their habitual environment.

Radically simplified, the fundamental question of ethnography is, “What do people do and why do they do it?” In the case of user research, we tack on the rider “...and what are the implications for the success of what I am designing?”

We already observe people regularly, if only to determine how we should interact with them ourselves. (“Is that guy on the bus trying to get my attention or having a loud conference call on his headset?”) And many of us are quite experienced at reporting interesting behaviors. (“You should have seen this guy on the bus...”) To do user research, you’ll need to make a slight mental shift to “How should what I’m designing interact with this person?” and then do your best to be totally nonjudgmental. That’s all it takes to stoke the human data-gathering machine.

The four Ds of design ethnography

Humans and their habits and material culture are endlessly complex. Ethnography is an equally deep and nuanced field. The practices outlined in this chapter are merely a pragmatic simplification of a few core ideas intended to help you apply useful insights about people to your product design.

It’s easy to get caught up in the specific techniques and terminology, so try to keep the following key points in mind for more successful user research.

Deep dive

You want to get to know a small but sufficient number of representative users very well. We’re typically talking a Vulcan mind meld with a handful of individuals, not a ten-question survey of a thousand families. Walk in their shoes. live in their skins, see through their eyes…choose the creepy spiritual possession metaphor that works for you.

Daily life

Fight the urge for control and get into the field where things are messy and unpredictable. (The field is wherever your target users generally are, anywhere from a cube farm to the London Tube.) As you’re probably well aware from how your day is going so far, life for everyone is messy and unpredictable in ways both good and bad. It’s very easy to think up ideal scenarios in which everything is smooth and simple. These are as useful to your work as a game of SimCity is to allocating actual resources in New York City.

Participant observation, whether done in person or remotely, is the name of the game. Everyone’s behavior changes with the context and the circumstances. Soak in your subject’s actual environment. It’s of limited utility to learn how people behave in your conference room. No one is going to act naturally in there. Even calling them in their own home or office is better. The most interesting insights will come when you keep your eyes open and go off script.

Data analysis

Gathering a lot of specific observations in the field is just the first part. Once you have all of this data you need to do a thorough job of sifting through it to figure out what it means. Systematic analysis is the difference between actual ethnography and just meeting interesting new people at a networking event. You can use a light touch and a casual approach, but take enough time to gain some real understanding, and get your team involved in creating useful models.

Drama!

Lively narratives help everyone on your team rally around and act on the same understanding of user behavior. From the mundane realities of real people, personas emerge—fictional protagonists with important goals—along with scenarios, the stories of how they use the product you’re designing to meet those goals. Personas keep you honest. You design for them, not for you or for your boss.

Interviewing Humans

The goal of interviewing users is to learn about everything that might influence how they will use what you’re creating. Good interviewing is a skill you develop with practice. The great myth is that you need to be a good talker. Conducting a good interview is actually about shutting up.

Remember, the people you’re interviewing want to be liked. They want to demonstrate their smarts. When you’re interviewing someone, you know nothing. You’re learning a completely new and fascinating subject: that person.

Preparation

Once you have established whom you want to talk to and what you want to find out, create your interview guide. This is a document you should have with you while you’re interviewing to ensure that you stay on topic and get all of the information you need.

The interview guide should contain:

• The goal and description of the study: This is for you to share with the participant and use to remind yourself to stay close to the topic.
• Basic demographic questions: These are for putting the participant’s answers in context. They will vary depending on the purpose of the interview, but often include name, gender, age, location, and job title or role.
• Icebreaker or warm-up questions: These are to get the participant talking. Most people know this as “small talk.” Feel free to improvise based on the demographic information.
• The actual questions or topics: You know, the primary focus of the interview.

You should also gather a bit of background information on the topic and people you’ll be discussing, particularly if the domain is unfamiliar to you.

Interview structure: three boxes, loosely joined

An interview has three acts, like a play or a spin class: the introduction and warm-up, the body of the interview, and the conclusion.

Introduction

Introduce yourself with a smile, expressing genuine gratitude that the person you are interviewing has taken the time to talk (even if they’re getting a large incentive and especially if it’s a busy staff member who has taken time out of their workday).

Describe the purpose of the conversation and the topic without going into so much detail that you influence the answer. Explain how the information will be used and shared. Obtain their explicit permission to record the conversation.

Ask whether they have any questions about the process.

Move on to the demographic information or facts you need to verify. Use the collection of this information as the basis for the warm-up questions. “Oh, you live in San Diego. What do you like to do for fun there?”

Body

Once you’ve covered the formalities and pleasantries, it’s time to dig into the interview. With a sufficiently talkative subject, you might get all of the answers you wanted and then some without asking more than the initial question directly.

Ask open-ended questions that encourage the subject to talk, not closed questions that can be answered with “yes” or “no.” (Closed question: “Do you communicate with the marketing department often?” Open question: “Tell me about the internal groups you communicate with as part of your job.”)

If the subject doesn’t offer enough information on a topic, ask a follow-up or probing question, such as “Tell me more about that.”

Allow pauses to let the story through. Silence is uncomfortable. Get used to it and don’t rush to fill gaps in the flow of conversation. You want your subject to do that.

Use your list of questions more as a checklist than as a script. If you read the questions verbatim, you’ll sound like a robocall survey.

Conclusion

Once you have the information you were looking for, and ideally even more, make a gentle transition to the wrap-up. Say something like “That’s it for my questions. Is there anything else you’d like to tell me about what we discussed?”

Thank them for their time and cover any administrative topics such as incentives or next steps on the project.

Don’t be afraid to shut it down early if you find yourself in an unproductive interview situation. Sometimes an interview subject goes taciturn or hostile. The best thing you can do is move on to the next one. No rule says you need to hang in there until you’ve attempted to have every single one of your questions answered. Just do your part to remain friendly and respectful to the end.

Conducting the interview

You, the interviewer, play the dual role of host and student. Begin by putting the participant at ease with your demeanor. The more comfortable a participant feels, the more and better information you will get. A relaxed participant will open up and be more honest, less likely to worry about putting on a good impression.

Once you’ve done your part to get the subject talking, get out of the way. You should strive to be a nearly invisible, neutral presence soaking up everything the other person has to say. Think of them as the world’s foremost expert on themselves, which is the all-absorbing matter at hand. Insert yourself only when necessary to redirect back on topic or get clarification. You will know when your interview is going particularly well because you won’t be able to get a word in, but you will be getting answers to all your questions.

Breathe

It’s easy to feel like you’re on stage and tense up without realizing it. Your own tension can be contagious, so remind yourself to breathe and remain relaxed and observant.

Practice active listening

As long as you’re breathing, make interested “mm-hmm” sounds. If you’re interviewing in person, make sure to look at the speaker directly and nod. Unrelated thoughts might start to pop up, especially if an answer goes on at length. Stay alert and focused on the other person.

Keep an ear out for vague answers

You want details and specifics. Always be ready to bust out a probing question such as “Tell me more about that.”

Avoid talking about yourself

Sometimes, what starts as active listening turns into “Let me tell you about a similar experience I had…” The interview isn’t about you or your opinions. This can be very hard to remember and takes practice to avoid. So, if you find that you’ve inserted yourself into their narrative, just stay relaxed and steer the conversation back on track.

Handy checklist

This checklist for effective user research was adapted from the Ethnography Field Guide produced by the Helsinki Design Lab, powered by Sitra, the Finnish Innovation Fund (http://bkaprt.com/jer2/05-02/):

• Create a welcoming atmosphere to make participants feel at ease.
• Listen more than you speak.
• Try to capture the thoughts and behaviors of your participants accurately.
• Go to your participants’ environments—wherever they’ll be engaged in the behavior you’re studying.
• Tell them the goal of your study briefly, being careful that it doesn’t direct their responses.
• Encourage participants to act naturally and share their thoughts aloud.
• Do not ask leading or yes-or-no questions. Follow up with more questions to clarify their responses.
• Write out your questions in advance, but feel free to deviate in the moment.
• Take pictures to create visual notes of your observations.
• Pay attention even after you stop recording—you might be surprised by what gets said.

Try to be as conversational and natural as possible. If the user volunteers the information in the course of your conversation without you having to ask, that’s terrific. Your questions are just prompts to help the participant tell you a story that reveals situations, attitudes, and behaviors you didn’t even think to ask about. Offer enough information to set the scope for the conversation, but not so much that you influence the responses.

Here is a sample set of questions for you to modify to meet your needs:

• Walk me through your day yesterday. Tell me about your job.
• Walk me through your last day off from work.
• How do you stay in touch with people close to you?
• What computers or devices do you use?
• Tell me about the last [category] thing you bought?
• Tell me about your household.

What to do with the data you collect

The interview is the basic unit of ethnographic research. Once you’ve completed your interviews, analyze them all together to find themes, including user needs and priorities, behavior patterns, and mental models. Note the specific language and terms you heard so you can better reflect the way users think and talk in the actual interface.

If you are doing generative research, look to the needs and behaviors you discover to point out problems that need solving. Turn the clusters around user types into personas that you can use for the life of the product or service you’re working on. (See Chapter 8 for detailed examples.)

Contextual Inquiry

Once you’re comfortable doing ethnographic interviews, you can take your skills into the field. If you like watching reality shows, you will love contextual inquiry, also called site visits or consensual home invasion—except instead of Project Runway, you’ll be enjoying Project Conference Call, Home Office Experience, or Saturday Morning Grocery Shopping. You enter the participant’s actual environment and observe as they go about the specific activities you’re interested in studying. By doing this, you will be able to see actual behaviors in action and learn about all of the small things you might not hear about in an interview, such as a janky workaround so unconscious and habitual the individual has completely forgotten it.

Contextual inquiry is a deeper form of ethnographic interview and observation. It is particularly useful for developing accurate scenarios, gathering stories about how users might interact with potential features, and identifying aspects of the user’s environment that will affect how someone might use a particular product.

Scott Cook, the founder of financial software giant Intuit, started the “Follow Me Home” practice very early in the company’s history (http://bkaprt.com/jer2/05-03/). He would quite literally hang out in Staples office supply stores waiting for someone to purchase Quicken, and then follow them home to observe them using the software. He learned where they had difficulty setting up the program, which allowed him to make improvements to the initial experience.

Things to keep in mind with contextual inquiry:

• Travel. Allow plenty of time to get to the site and set up.
• Get situated. Find a comfortable spot that allows you to talk to the participant without interrupting their normal routine.
• Interview. Establish trust and learn about what you will be observing. Find out when it will be least disruptive to interrupt and ask questions.
• Observe. It’s a show. You’re watching. Note everything in as much detail as possible. The relevance will become apparent later. Pause to ask questions. Stay out of the way.
• Summarize. Conclude by summarizing what you learned and asking the participant to verify whether your observations were correct. Note: even if the participant disagrees with your assessment, you might still be correct; the contradictory description makes for an interesting data point.

Contextual inquiry can be very inspirational. You might observe problems and opportunities you had no idea existed and open the door to some innovative and surprising ideas. Be ready to learn that people don’t need what you thought they need at all, but that they do need something totally different. Joyfully release all of your preconceived plans and notions.

Focus Groups: Just Say No

Scene: a handful of “ordinary” people around a conference table engaged in a lively discussion about how various brands make them feel. A cheerful but authoritative moderator. Observers wielding clipboards behind a two-way mirror. Focus groups are synonymous with qualitative research in popular culture, and it isn’t uncommon to hear all user research reduced to “focus groups.”

Focus groups evolved from the “focused group interview” developed by American sociologist Robert K. Merton. Merton himself deplored how focus groups came to be misused:

There’s so much hokum in focus groups, at times bordering on fraud. There are now professional focus-group subjects who get themselves on lists. Even when the subjects are well selected, focus groups are supposed to be merely sources of ideas that need to be researched. (http://bkaprt.com/jer2/05-04/, subscription required)

Focus groups are the antithesis of ethnography. Unlike interviewing participants individually or observing people in their natural environment, the focus group creates an artificial environment that bears no resemblance to the context in which what you’re designing would actually be used. The conversation is a performance that feeds social desirability bias and blocks true insight into what people need and how they behave outside of this specific, peculiar group dynamic. And one bad recruit in the group can take down the entire session.

Some group activities may yield useful insights as part of the design process. However, focus groups are simply wildly expensive research theater. And your research time and budget are too precious to squander on a sideshow.

The Talking (and Watching) Cure

Accept no substitute for listening to and observing real people who need to do the things you’re designing a thing to help people do. A few phone calls might completely change how you approach your work. Or, maybe you’ll find out your instincts were right all along. In any case, the information you gather will keep paying dividends as you continue to gather and examine it, grounding your design decisions in real human needs and behaviors.

And as you develop the skill of stepping out of yourself to become an effective design ethnographer, you will develop powerful empathy that can inspire you to find creative, effective solutions.