Chapter 7: Interviewing

In this chapter, we’ll show you how you can get better at running user research sessions. We’ll cover interviewing techniques, managing tasks, and how to deal with tricky situations that can throw you off balance. As we mentioned in Chapter 6, interviewing is only one of the activities that takes place while you’re running the session. But because it’s so important, it gets a whole chapter to itself.

Research cycle: fieldwork

Show Participants How to Help You

Research interviews are a strange, artificial social interaction. If they’re in a purpose-made research lab with cameras and recording equipment, they’re even stranger.

Fundamentally, an interview is an exchange of information between two people. But think of how many variants that includes: job interviews, police interviews, dates, quizzes, journalistic interviews… the list goes on. In each case, there are subtle differences in the rules and dynamics. Through personal experience – or from watching television – most people have an understanding of how that social situation works.

Think now of research interviews. In many cases, participants have no idea what to expect. Is it a test? Is it a chat? Will I be expected to give away intimate information? Because they don’t know what the rules are, they’ll be nervous, and they’ll look for analogies in other interview-like situations that they’re more familiar with to make sense of it. They want to help you, but they don’t know how.

This is where you can show the way. By clearly communicating the rules, process and expectations of the session, you can show them how to help you. They’ll be relieved, and you’ll get a better, more open, session.

The rules of a research session:

1. There are no right or wrong answers.
2. Negative answers are OK.
3. We’re testing the product, not you.
4. If you don’t want to answer, you don’t have to.
5. If you don’t want to stay, you can leave at any point.
6. The researcher isn’t an expert on this subject either: they’re here to learn from you.
7. If you’ve got a problem with your account or a complaint about the client company, the researcher won’t be able to help with it.
8. Phones should be turned off.

By stating the rules at the beginning (and reminding the participant about them as necessary later on), you can build trust, and avoid misunderstandings. Sometimes, we find it helpful to post the rules on the wall of the interview room for further emphasis.

If you want a session to run smoothly, the first thing you can do is be clear about what you need:

• As you’re briefing the participant about the session and introducing them to the consent form, ask them what they’re expecting. Tell them what will happen. Make it a conversation, not a speech.
• At transition points in the session, remind them. Typically this would be just before you introduce the first main task, and in the final five minutes, before you ask them to sum up.
• If the participant is doing something you don’t want, say so. Be firm (but pleasant and polite) in your moderating. If they’re rambling or veering off track, say something like: "That’s not a topic we’ve got time for today. I need to ask you about this instead." You’ll be surprised how receptive participants are to this. Don’t forget: you’re giving them clarity about what they need to do and how they can be helpful.

Observing vs. Asking

When we think of interviews, we tend to think of a series of questions and answers between two people. In reality, the words are just the beginning – you have many more tools at your disposal.

• Your observations of the participant as they complete tasks and activities. The classic scenario is to ask someone to carry out a task on a website, but it could be a game, a card-sorting exercise, a worksheet, or many other things.
• Nonverbal communication: body language, gestures and the like. Does it echo what’s been said, or contradict it?
• The participant’s tone of voice, energy, mood, cadence, and how these are changing over time.
• If there’s more than one participant, the dynamic and interaction between them.

Because this kind of concentrated listening goes beyond just the words that the participant voices, we call it ‘paying deep attention’.

Paying Deep Attention

When we train researchers, they’re often most nervous about how to ask questions. Actually, the core skill is simple: stop worrying about your next question, and instead pay attention to the participant. You’ll find your next question comes naturally from what they say or do.

Obviously there’s a bit more to it than that. Firstly, paying deep attention is harder than it looks, especially when you need to sustain it over time and avoid the temptation to judge. Secondly, there are better and worse ways to ask a question (which we’ll cover below). Thirdly, as moderator you’ll normally be juggling several other tasks: making sure the prototype is working, keeping an eye on the clock, ensuring you’re covering off the topic areas in your guide.

There’s no shortcut to mastering deep attention. You may find it easier if you’ve practiced mindfulness, or if you’re gifted with strong powers of concentration or empathy. The best way to develop this skill is to practice it:

• In your research interviews, try to catch yourself if your mind is wandering. Recognise why this is happening: are you bored, distracted, hungry? Maybe you disagree with something the participant has said, or find their personality objectionable. Maybe you’re thinking about solutions. In each case, you’ve stopped paying deep attention. Mentally acknowledge what’s interrupted you, and refocus.
• After interviews, watch one or two recordings. Pay attention once again to the participant, and critique your questions. Could you have followed up on what they said or did more effectively? Also, watch out for the ratio between listening and talking. In a depth interview, you should be speaking for approximately 15% of the time, compared with the participant’s 85%. If it’s closer to the 50/50 split you’d expect in an everyday conversation, then you’re talking too much. Next time, give the participant more space to talk and think.
• In your everyday life, practice paying attention in your interactions: really focus on people’s words, body language, tone and the relationship between what they say and what they do. Practice listening without judging. Apart from the boost this will give your interviewing, it’s also a great transferrable skill, applicable in many areas of your life.
• Try to become more comfortable with pauses and silence in conversation (we’ll say more about this below).

Starting the Session

If we’re doing lab testing, we always introduce ourselves outside of the testing room and explain what is going to happen during the session. We’ve found that this makes people more comfortable than taking them straight into the test room, because it puts them at ease while they’re getting used to you and the rules of the session.

Our waiting area

In our waiting area we have a comfortable sofa, some magazines, and we offer everyone a cup of tea or coffee. The interviewer gives the participant a few minutes to settle in and then goes to say hello. We introduce ourself and thank them for taking the time to come in and help us out. We explain what we’re doing and what they can expect from the session, and give them a chance to ask any questions.

Make Sure You Get Consent

You need to explain your consent form to your participants and be ready to answer their questions. You can read about writing a consent form in Chapter 3. They have to understand what is happening and provide their consent before you can start the session with them. At this stage, you should also make them aware of any codes of ethics or professional standards that apply. For example, at cxpartners we abide by the MRS Code of Conduct.

At the end of the session, you may also want to re-confirm the participant’s consent by asking if they’re still OK for you to keep a recording.

Starting Guerrilla Testing Sessions

If you’re guerrilla testing, you’ll have less time with people to introduce yourself. Also, while you’re introducing yourself you’re also trying to explain what the research is about, and persuade them to help you. That’s tricky and needs some consideration.

What you shouldn’t do:

• Avoid opening by asking a question that could be answered with a "no". For example, "Have you got 10 minutes?" People will have no idea what you will be expecting them to do, so this is going to make them reluctant to commit their time. They’re also likely to be busy. Why should they give you anything?
• There is also a chance that you will be mistaken for a charity collector, which will make it even more likely that people won’t stop.

What you should do:

• Ask a question related to the subject or product you’re researching, to start the conversation. So if you’re developing an app that will allow people to buy bus tickets, you could ask something like, "Hi, sorry to trouble you. I was just wondering if you bought a bus ticket in the past week?"
• If they answer yes, you could say, "I’ve been working on something to try and make it easier to buy tickets. Would you mind taking a look at it for me and telling me what you think?"

This gives people context, establishes their eligibility, and starts a more natural conversation.

How to Ask a Question

In this section we’re going to look at how questions work:

• Individually, questions have their own syntax. If you follow these rules, it’ll be easy for participants to understand what you’re asking them.
• In sets, questions have a combined power. If you use questions effectively in combination, you’ll get to the answers you need more quickly and with less effort.
• Good questioning relies on timing. Choosing the right moment makes your questions more thought-provoking and revealing.

Here are 10 pointers to making you a better interviewer. Try to focus on one or two of these each time you run a session, and ask your colleagues for feedback on how you did.

1: Clearly State the Subject and the Query

Think of a question as having two parts:

• The subject (what the question’s about: football, ice cream, holidays).
• The query (what you want to know about it: what’s your favourite? what was your most recent experience?).

On their own, neither is much use:

• Subject: “Let’s talk about ice cream.”
• Query: “What’s your favourite flavor?”

Yet when we combine them, it’s clear what we’re talking about and what we want to know: “Let’s talk about ice cream. What‘s your favourite flavor”

Asking the subject first, then the query, makes it slightly easier to process the question. But you can introduce them in either order: “What’s your favourite flavor of ice cream?”

2: Don’t Use Leading Questions: Establish Relevance Instead

A question like, "What’s your favourite flavor of ice cream?" introduces a new issue. With this structure, we’ve assumed that the participant has a favourite ice cream flavor, and therefore the query is relevant to them. We’ve asked a leading question. This is bad news, because the participant may feel under pressure to pretend to like ice cream, and invent a favourite flavor. That’s worse than no answer at all.

To avoid this problem, we should use a qualifying question to establish the relevance of our query:

• “Do you like ice cream?”
• “What’s your favourite flavor?”

3: Keep Your Question As Short As It Can Be

It’s easy to ramble on about the subject and let it turn into a kind of introductory statement, as in the example below. Apart from wasting time and patience, it’s possible that the participant will forget what you were asking about!

• Subject:“The next thing I’m going to ask you about is ice cream, which is something that we’ll talk about now, and then later on in the interviews, but in the meantime I want to ask you about it initially.”
• Query: “What’s your favourite type?”

It’s best to avoid this kind of rambling, but if you do need use an introductory statement, always restate your subject immediately before or after the query:

• Introductory statement:“The next thing I’m going to ask you about is ice cream. We’ll talk about it later on in the session too. But I’ve got some initial questions that I need to ask you first”
• Query and subject reminder together: “So, what’s your favourite ice cream flavor?”

4: Closed Questions and Open Questions Have Complementary Strengths and Weaknesses

Closed and open questions both have their role, and it’s best to use them together.

• Closed questions (eg, “Do you like ice cream” or “Which do you prefer, cupcakes or donuts?”) are easy to phrase and understand. They require less cognitive effort from the participant to answer. They’re better for putting people at ease than more challenging open questions. And as we saw above, they’re good for communicating the subject and establishing relevance.
• Open questions (eg,“What do you like about ice cream?”; or “What’s your favorite flavor?”) do most of the real work. They give participants the liberty to explore and talk about the subject in their own way, revealing their thought processes, needs and preferences in the process. The disadvantage is that they’re harder to answer, can be more intimidating, and can be a cue to ramble for some people. Open questions tend to begin with words like ‘what’, ‘who’, ‘when’ or ‘how’. It’s generally best to avoid ‘why’, because it can lead to post-rationalised answers.

5: Look Out For ‘Value Words’

There’s a certain kind of word or phrase that you should be particularly attentive to when a participant is speaking. We call these ‘value words’: adjectives or descriptions that appear meaningful, but are actually open to wide interpretation. Some examples: “corporate”, “cluttered”, “gritty”, “feels like the real thing”, “my kind of place”.

When you hear value words, you should do one of two things:

• Follow up with a probe there and then. You do this by repeating back the exact phrase or word, and then asking an open clarification question:
  “You said the website felt corporate. What do you mean by ‘corporate’?”
  “You said the shop ‘feels like my kind of place’. What makes it feel that way to you?”
• It might not be the right moment to follow up. Make a note of the value words on a pad or Post-It note, and keep it in reserve to return to later.

Value words are your doorway to real insight in the interview. Don’t waste them!

6: Listen For Contradictions and Gaps

Imagine you’re a detective. The participant will be wanting to present a certain version of him or herself to you: perhaps someone who exercises every day and eats plenty of fruit and vegetables. Whether or not this is a conscious effort on their part, it’s unlikely to be the whole picture.

If you can be attentive to contradictions in the research session, you’re more likely to get under the skin of the topic, and find out what’s really going on. This might be a contradiction between what someone said and what they did: “Earlier on, you said your preferred holiday destination was Mexico, but when you went onto the website, the first page you visited was South Africa. That’s interesting isn’t it? Tell me more about that.” Or it might be an inconsistency between two different statements that they made: “Earlier on, you said that your favourite ice cream flavor was rum and raisin, but just now you said it was coconut. Do you have more than one favourite flavor, or is there another factor at play here?”

When you’ve spotted the contradiction, follow it up. It’s absolutely essential that you’re respectful in your phrasing, but participants are generally curious about their own internal processes, and intrigued by what you’ve shown them.

Sometimes it can be as interesting to note what’s not happened. Perhaps the participant had mentioned a subject they were interested in, but flicked straight past it in the material you were asking them to look at. Or perhaps you were expecting them to comment on a particular design, but they said nothing.

It’s ok to draw attention to this. Your expectations (your rolling hypotheses, in other words) need checking, so ask:“A moment ago, I was wondering if you were going to click on to the information about Mexico, but you didn’t. Looks like I was wrong! Can you help me to understand your thought process?”

7: Use Question Chains

Now that you’ve been introduced to the main question types, you can learn to use them in sequence, as ‘question chains’. Here’s an example:

• Subject: “Let’s talk about going on holiday.”
• Relevance:“Have you been away this summer?”
• Query: “Where to?”
• Participant responds: Sri Lanka
• Open question: “What drew you to Sri Lanka?”
• Participant responds: It was a great deal, and it’s a bit unusual.
• Probe: “When you say ‘unusual’…?”
• Participant responds: I like to go to places that are a bit unusual. Somewhere a bit different. Not just the usual places that everybody goes to. And it’s a resort, so you’ve got your home comforts too, there’s a bit of predictability.
• Query contradiction: “Let me check then. So you’re after somewhere that’s a bit unusual, a bit different. And you also want your predictability, your home comforts?”
• Participant responds: “I hadn’t really thought of it like that. I suppose what I want is a taste of adventure, without really going off the beaten track. Something to show off about at the water cooler when I get back to the office.

In that example, we’ve gone from using closed questions to establish relevance and initial facts, before using an open question to really kick off the conversation. We jumped on the value word ‘unusual’, and then explored the contradiction between adventure and home comforts. And in doing so, the participant has shared something that really matters to them: it’s important to them to be able to show off about their holiday afterwards. In other words, we’ve gone from some very neutral facts, to a statement about feelings and self-perception.

Once your chain runs out of steam, you can start another one. As you get more practiced, you’ll be able to run several question chains at once, spotting the contradictions between them as you go, and reviving them when the time is right.

8: Prompt, But Don’t Lead

When you want the participant to keep talking, but you don’t want to ask a new question, you can use prompts. It’s helpful to have a repertoire to use. These can include:

• Short phrases such as “OK”, “tell me more” or “carry on”. It’s important to keep these neutral so they don’t lead the participant. If you were to say “yes!” or “that’s right”, then you’d be tempting them to tell you more about what you wanted to hear.
• Non-word noises like “um-hm” or “uh-huh”. These are better because they’re less of an interruption to the participant’s flow. On the other hand, they can feel awkward, so experiment to find one you’re comfortable with.
• Body language. A nodding head can lead participants in the same way as“yes”, so go for something more subtle. One way is to tap your foot in time to the rhythm of their speech, and then keep it going if they stall. Or raise your eyebrows to indicate “go on”.
• Silence is a great prompt (see below).

9: Choose The Right Moment

The timing of your question is as important as the phrasing. If you choose the right moment, you’ll barely have to give any explanation: the participant will understand what you’re interested in and give you a long, detailed answer. Choose the wrong moment and your question will fail, or get a radically different response.

Here’s an example. Imagine you wanted to get feedback about a new website design from your participant. Your plan is to ask them: “I’d like to know what changes you’d like the designers to make to this website. What are your top three suggestions?” If you asked this question after the participant had used the site for just 60 seconds, then you’d be likely to get comments that focused on visual design: “the colours are too garish”, “I don’t like the photography”, etc. If, on the other hand, you asked them at the end of the session, they might concentrate on difficulties with completing tasks, or missing functionality. Be flexible about the order in which you tackle subjects, and don’t feel obliged to follow the sequence in the discussion guide.

10: Be Quiet

Most people who are learning to interview talk too much. They don’t give the participant enough space to talk or think, and as a result they miss valuable insights. Get used to silence – become comfortable with it. Some participants (particularly more introverted personalities) appreciate a moment to gather their thoughts before or after answering a question. What’s more, silence is a great prompt.

As you get more accustomed to interviewing, you’ll create your own methods and favourite questions. You’ll develop your own style. And because you’re comfortable with the technical side of interviewing, you’ll be able to pay deep attention at the same time as juggling your various other tasks. Once you’re up and running, interviewing participants can be fascinating, and great fun. Enjoy it!

How To Get Out Of Trouble

The vast majority of problems that occur with participants during interviews relate to your setup. If they’ve been correctly recruited, you’ve piloted the session and prepared your material, warmed them up, and been clear about your expectations, you’ll find that sessions run smoothly most of the time.

Occasionally, though, you’ll find yourself in a difficult situation. See it as a challenge: there are plenty of techniques you can use to get yourself out of trouble. If you’re finding a participant uncooperative, select from the escalating scale of tactics below to help turn it around.

Getting out of trouble: an escalating scale

The main point to remember is: never panic. You’re in charge. Easier said than done, perhaps, but don’t forget, it’s your interview and you’re the one setting the rules. Even if you can’t control how the participant acts, you do get to choose how the session is run, and whether to terminate it. Just having that knowledge in mind can be enough to avert a drama.

Summary

• Be clear about your expectations with participants, and they’ll be better able to help you.
• Practice paying deep attention, and you’ll find your questions flow from participants’ answers and behaviour.
• Get to grips with question syntax, and learn to use question chains, so the structure of your interviewing comes more easily. That’ll leave you with extra capacity for paying attention.
• Dig deeper into participants’ responses by using value words.
• Sometimes you might get into trouble, but there’s always a way out. Never panic.