Appendix A. Questions That Work

Customer development doesn’t always have to be scheduled, or part of a structured interaction.

Trying to craft an effective question while you’re carrying on a conversation can be challenging. It’s easy to inadvertently ask a leading question (How often do you think you would use X?) or a yes-or-no question that elicits a one-word response (Do you think Y is a good lunch option for your family?). Yes-or-no questions are not only ineffective, but they throw the ball right back in your court since they do not elicit a more open-ended response. You have to have another question ready immediately. Since you want to listen more than you talk, that’s not a good strategy.

In this appendix, I give you a list of questions you can use. For each question, we’ll talk about how the question is constructed, when it makes sense to ask it, and what you can learn by asking it.

Warning

Do not ask questions that you could have answered yourself using a search engine. People enjoy feeling helpful by providing information that is specific to them. Treat them as experts, not research assistants.[76]

Questions for Any Customer Development Interview

Whether you work for a startup or an established company, these questions can help you.

Tell me about the last time you ___

The situation

The customer is talking about a specific task, either complaining about it or expressing the wish to do it faster, better, or not at all.

This isn’t a question as much as an invitation to speak freely. It doesn’t make sense to ask questions that measure time, effort, cost, or value because you don’t yet know whether any of those attributes are interesting.

Another important feature of this question is that you’re asking about a tangible past action, not a potential future action. When people talk about things they may do in the future, they tend to be more aspirational, more positive, and less accurate.

What you’ll learn

When you don’t imply that you’re looking for a specific type of information, the customer will simply talk about what is most significant to her about that task. It may be related to who, what, why, when, or how; it may be a positive or negative emotion.

Note

What the customer chooses to talk about first is a guide to how you should continue the conversation.

If you could wave a magic wand and change anything about how you [perform this task], what would it be?

The situation

This question is an effective way to help a customer who is fixated on a specific solution or hampered by a real or perceived limitation.

What you’ll learn

This question forces customers to identify their biggest pain point. It’s often a way to learn more about a customer’s environment: would his magic wand help him do something faster or better, or eliminate obstacles or bureaucracy?

What tools do you use for _____?

The situation

This question elicits specifics. It helps to prompt a customer to move beyond generalized descriptions of how he completes a task or deals with a situation.

Even when you have made it clear that you are interested in a customer’s specific experiences, she doesn’t always believe you: “Oh, I didn’t think you wanted me to mention individual websites or apps that I use—I figured you’d want a general answer that might apply to other people too.”

What you’ll learn

Specific websites, equipment, software programs, apps, or methods that the customer uses. If the customer is using tools to try and solve a problem, that’s a good sign. It shows that she recognizes the problem and is already committed to trying to solve it. (A customer who intends to try a solution to a problem but hasn’t gotten around to it yet is usually a red herring.)

When you started using [tool], what benefit were you expecting?

The situation

Use this question if the customer thinks his problem is solved. He’s using a product that at least partially addresses his pain points. He may not be actively seeking a better solution.

What you’ll learn

The customer’s initial expectations around solving his problem. If the tool is a commercial product (as opposed to something like pencil and paper), it’s useful to look at that product’s marketing. Compare the value promised by the product’s marketing with the customer’s expectations.

The whole problem may not be solved

It’s common for customers to start using a product or service for one purpose and then discover it’s useful for a different purpose. The customer may be completely satisfied with the product’s new value but still have an unsolved problem. A good follow-up question is:

Is there anything you expected to be able to do with [tool name] that you aren’t able to?

How often do you do _______? Let’s say, how many times in the past month?

The situation

Usually I find myself asking this question when the customer has expressed interest in a product or feature to help her solve a problem with a task or routine.

“How often” sounds appropriately conversational so it doesn’t put the person on the defensive. Since people tend to be bad at estimating the frequency of events, following it up with a specific window of time in the past (not too long a window; here I used a month) makes it easier for them to give you an accurate count. Using a “how many times” framing is also helpful if you are asking about behaviors that are less socially acceptable or that the customer is embarrassed about.[77]

What you’ll learn

Obviously, you’re asking how often a situation occurs, which helps you figure out whether this is a significant enough problem to invest in solving.

You’re also implicitly asking about priority. It’s not uncommon for someone to describe a situation that happens all the time and then admit that it hasn’t happened in the past month. How does the customer then react? Does she laugh and admit that, “OK, maybe it’s not that big a deal after all,” or does she say defensively, “Well, it seems like it happens all the time.” The latter is a clue that this is a particularly severe problem or one that causes constant low-level stress. That’s a problem worth looking into!

When this occurs, how much additional time or money does it cost you or your company?

The situation

You’ve identified a problem but you’re not sure if you can fix it profitably. This question is also helpful if you’re uncertain of the severity of the customer’s pain point because the customer is not showing much emotion. (Of course, in a public space such as a conference or sales meeting, people tend to exhibit more reserved emotional responses.)

What you’ll learn

How this customer thinks about time and money. Is he the one responsible for budgeting either of them? Is this customer the primary user of your product but not the person who makes a purchasing decision?

Helping them quantify

Many people (especially consumers but also businesspeople) are not accustomed to quantifying time or money wasted. People make a lot of irrational decisions. However, it’s useful to guide a customer through this calculation:

Interviewer: You just described a problem that happens in your household, realizing that you’ve run out of groceries. When this occurs, how much additional time or money is it costing you?

Customer: Well, I just run to the store, or maybe call for a pizza.

Interviewer: So, if you make an unplanned trip to the grocery store, how long does that take?

Customer: Oh...maybe 20 minutes. Well, 30, if it’s crowded in the store.

Interviewer: How is the rest of your night affected, taking that extra 30 minutes to get to the store and shop and come back?

Customer: Well, then we’re eating dinner later, so there’s a rush to get the kids’ homework done and then bath and bedtimes go late. That’s why it’s tempting to just order a pizza.

Interviewer: How does the cost of pizza for dinner compare to what you would have cooked if there were groceries?

Your goal is not to get to a precise dollars and cents (or hours and minutes) calculation, but to get a more concrete sense of the value you’re providing.

Who else experiences this problem?

The situation

You’re looking for additional segments of customers to target, or you’re not sure if your initial target customer profile is accurate.

What you’ll learn

Who else you could be talking to. Another benefit to this question is that your customer may have a better sense than you of what he has in common with other customers. For example, if you were building a product for babies, you might assume that your target customers are parents. But parents know that they are not the only purchasers of baby things—buyers include grandparents, doting aunts, neighbors, and anyone invited to a baby shower. A more accurate target customer description may be people who buy presents for babies.

When you do (or use) ______, is there anything you do immediately before to prepare?

The situation

You can always ask this question—almost any task or routine includes preparation steps that we don’t always think about. This is also a useful ice-breaker question when you’re talking to a customer who doesn’t seem to have anything to complain about, such as an existing customer who has fallen into a routine. Nothing is explicitly broken, so she doesn’t think about what could be improved. Or perhaps you asked the “tell me about how you ____” question and you sense that the customer’s answer was too narrow in scope.

What you’ll learn

There may be existing preconditions required for the customer to use your product (I only use X when I’m on a business trip). If there are related activities or tasks that someone always does before using your product, that’s an opportunity to extend your feature set or partner with a complementary product or service.

Online bill pay is surprisingly manual

When I was at Yodlee, my design team partnered with a major consumer bank to figure out how to create a significantly better online bill-pay experience. The project was canceled before engineering resources were committed, but I’ve never forgotten a surprising insight that I kept hearing from the customers I interviewed:

Customer: I don’t have any real complaints about using online bill pay—it just works, and it’s more convenient than writing checks and remembering to get stamps.

Me: You told me about the steps you go through when you pay your bills online. Is there anything that you do before you come to the bank’s website to prepare?

Customer: No... well, I know the bank shows me my balance, but that’s not accurate because I’m going to get money from the ATM between now and when these bills are due. And I have a couple of bills that automatically debit. So I need to figure out what’s going to really be left.

Me: How do you do that?

Customer: I usually grab some scratch paper and a pen and just subtract, you know, I have this much money, and then subtract what I think I’m going to withdraw, and then subtract the auto debit amounts. Then I know what my real balance is going to be and I can start filling in the amounts I want to pay online.

This customer defined paying her bills online as going to the website, but it actually included this preparatory offline step. I heard variations of this story from multiple people!

Now, would it make business sense to build an additional calculator tool into an online banking website? Maybe not; banks do make an awful lot of their revenue from customers who accidentally overdraft their accounts. As someone who’d been designing financial software for years, it was amazing to me to discover this common user behavior that I had no idea existed.

Here’s a related question:

When you do (or use) ______, is there anything that you do immediately afterward?

Similar to the question above, this is a good way to learn more about a situation from a wider angle. I talked in Chapter 4 about abstracting up a level when you ask questions—asking what people do before and after a specific task or habit is another way of doing that.

Would you be willing to help us by participating in user research or beta testing?

The situation

Always ask this question at the end of every interview! Don’t wait until you have a formal beta testing program in place. It’s extremely useful to have a bank of email addresses of people who’ve already agreed to help you out in the future.

What you’ll learn

If you’re solving customer problems, people will say yes to this question. If you get a lot of “no” answers, view that as invalidating your hypothesis.

How asking this question helps with future product development

Once you have people willing to answer questions, you can cut down dramatically on the number of assumptions that your team has to make in future product development. You can quickly take simple assertions (I assume most of our customers are using this feature while they’re traveling) and send them to a handful of people via email for feedback:

Quick question: Is there a specific occasion or situation occurring when you’re using [feature]? (If so, what is it?)

We’d like to make sure we understand how people are using [feature] before we decide which changes to prioritize. Thanks for your help!

It’s worth taking a moment to wordsmith your question to prompt something beyond a yes-or-no answer. (For example, “Do you run out of groceries in the middle of the week?” should become “When do you typically run out of groceries?” If the customer’s answer is “I don’t,” he’ll tell you so.)

This is quite different from the Question of the Week described in Chapter 9. That’s a random question, asked of anyone you happen to be in contact with that week, just to keep learning. In that case, you want a very short and consistent answer so you can later search, for example, your trouble, ticketing system for responses.

For this question, you’re choosing specific customers you trust and asking them (in individual emails) to elaborate on the question so that your team doesn’t waste time arguing over unproven opinions.

Questions for an Existing Product

When you have existing products, both you and your customers may have some preconceived notions to overcome. Customers may have seen a previous product roadmap presentation and be expecting certain features. You may have your own biases about what you think customers need and want.

When you use [our product], what’s the first thing you do with it?

The situation

You’d like to learn more about customers’ subjective experience. Procedural questions sometimes get you a more detailed answer.

What you’ll learn

Depending on how you measure your product usage, you may get the factual answer to this question from your quantitative analytics. I’m often surprised, though, by how frequently a customer’s subjective recollection of what she does differs from our objective logging of the actions she takes.

What’s the most useful thing that you regularly do with our product?

The situation

You have a hypothesis about what customers value the most. You’d like to validate it to determine where to invest in improving your product.

What you’ll learn

People who build products tend to believe that customers will derive the most value from the most difficult or technologically advanced features. This is often not true! I’ve often heard customers explain that the most used and useful features relate to easy actions like sharing or exporting.

If you had [requested feature] today, how would that make your life better?

The situation

A customer has just asked for a specific feature or change to your product. Maybe you don’t think it’s aligned with your product vision or maybe you just aren’t sure you understand what the customer is asking for.

You may need to adjust the level of language formality, depending on your customers and your existing relationship. There’s a small risk of customers perceiving this question as glib or dismissive. I do recommend keeping it deliberately personal and vague, though—“make your life better” or “make your job easier.” If you ask about something more specific, like saving time or money, you’ll get a more constrained answer.

What you’ll learn

The problem that this customer is trying to solve!

Other customers have told me that they experience [problem]...

The situation

As a last resort question in a customer development interview. If you are trying to validate that a problem affects other customers, it’s a stronger signal if someone mentions the problem unprompted. But sometimes you have a strong intuition: you just know this customer experiences a problem, even though she hasn’t mentioned it yet. When this happens, a nudge may help get her talking.

When you don’t have time for an extended interview. You can also use this question if you don’t have enough time for an extended interview, in which case it serves as a useful standalone conversation starter. (Note that it’s not a yes/no question but an invitation for the customer to start talking.)

When you see a pattern and want to challenge it. As patterns begin to emerge, you can use this question by stating the opposite of the pattern and claiming that mythical other people do it another way. See Chapter 6 for details.

What you’ll learn

For some reason, the mention of other customers seems to trigger people to give more thoughtful, honest responses than blind agreement or disagreement.

This may be because it is perceived as being granted permission to complain. Those of you who’ve been on the other end of an angry customer phone call may find this hard to believe, but most customers don’t actually feel comfortable complaining, which means that their problems go unresolved.

If It Works, Keep Asking It

I’ve provided these questions to ensure that you have a good foundation for both your structured customer development interviews and any ad hoc incremental customer conversations you have. However, you’ll find that you won’t need this list for long. Once you’ve started talking to customers regularly, you’ll develop a good sense for which types of questions get customers talking. If you asked a question spontaneously and it elicited a five-minute enthusiastic response, by all means keep asking it!



[76] Don’t ask someone when her industry’s annual conference is or ask for a list of schools in her city—those are facts that you should be willing to look up on your own. Don’t ask for information that you could find if you read her corporate website or watched her marketing videos. The Let Me Google That for You website was created to reflect this feeling of insulted indignation. From their About text: “This is for all the people who find it more convenient to bother you with their question rather than google it for themselves” (http://www.lmgtfy.com).

[77] One of my friends is a nurse practitioner for a free clinic in a low-income urban neighborhood. She was trained to ask “How often have you used drugs in the past week?” instead of “Do you use drugs?” because patients respond (more or less) honestly to the first question and lie in response to the second one. Without an accurate picture of the patient’s habits, it’s more difficult to provide effective medical care.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.149.254.48