What will you hear when people are too eager to please? There’s often a difference between politeness and honesty. You have to learn to read between the lines, and the subtext of the conversation can be slightly different depending on whether your product is aimed at consumers or at businesses.

Generally speaking, consumers don’t want to disappoint you. If you’ve done a good job in building rapport and sounding human, your interviewee wants to cooperate and make you happy. She may say that she’s experiencing a problem or behaving in a certain way even though it’s not true. She may say that she would pay for a solution that allowed her to accomplish something, but in reality she would never hand over her credit card.

For those of you who work with enterprise products, this takes on a different tone. Your interviewee may feel that being professional means valuing politeness over honesty. He may think he is being diplomatic by agreeing that something is possible when he knows it’s highly improbable. He may also believe that these conversations are an invitation to verbal horse-trading: by agreeing with you in one area, he may feel that he has earned the right to ask for something that he wants later.

In both of these situations, you may hear responses that align a little too perfectly with what you were hoping to hear. When you hear an interviewee say something that confirms your assumptions, don’t stop there. Be skeptical.

I often say, “If someone says maybe, write it down as no.”

It’s not enough for someone to say that she wants something—a product, a service, the ability to accomplish a task. It’s free and easy to want.

Actually changing behaviors, spending money, or learning something new has a cost. You need to figure out the difference between want and will, and uncovering that difference requires discipline in how you talk to your target customers. If you ask a question and the person says “yes,” that carries significantly less weight than if he brought up the subject unprompted.

In Chapter 4, I talked about the importance of asking open-ended questions. It’s almost always better to avoid yes-or-no questions entirely, but in the beginning that’s a hard habit to form. You’ll slip up occasionally and ask something like “Would you do X?” or “Would you like X?” That’s fine, as long as you recognize that you can’t take the customer’s response at face value. You need to challenge it.

Years ago, I worked with a large bank on some in-person customer research. An executive with the bank insisted on asking the question “Are you concerned with the security of your financial information?” I pointed out that no one would ever say “no” to that question, but to no avail. As I predicted, 10 of the 10 people I spoke with answered that, of course, they were very concerned with the security and privacy of their financial information. As one of them was leaving, I thanked him and said, “In order to get your $50 gratuity, I’ll need you to write down your mother’s maiden name and social security number.” Without hesitation, the man grabbed a ballpoint pen and reached for my sheet of paper. I stopped him before he could write anything, but my point was made. Very concerned about security... until $50 was on the line.

A better way to phrase that initial question would have been to ask about existing behavior, such as “What actions do you believe people can take to keep their financial information secure?” If you’ve already asked a yes-or-no question and gotten a “yes,” you’ll want to use a more open-ended question in subsequent interviews.

A reliable answer should include some tangible actions, such as “choose secure passwords” or “always log out after using a bank website.” In contrast, consider a response like “I don’t know” or “I always read about someone hacking into somewhere and getting credit card numbers, but there’s really nothing you can do to stop that.”

Even if you’ve phrased your questions well, sometimes customers will sound extremely enthusiastic even though they’re unlikely to buy or use your product. I experienced this firsthand when I ran design and research for a personal finance product. When I did customer interviews to select people for our beta program, almost everyone was passionate about problems with managing their money. They described their frustrations in detail. When they saw a demo, they loved the idea of the product. Most said that they would definitely use such a product, even pay for it. And yet somehow, we didn’t end up with anything approaching 100% usage. (You’re shocked, I know.)

Over time, I began to see differences in the specific words people used. The people who became customers used a more active voice and described specifics; others used a more passive voice and talked in hypothetical terms. If you’ve taken verbatim notes, look for patterns like those in Table 6-1.

It turned out I was hearing from a lot of people who were smiling and passionate and talking about what they wished they would do. Managing your finances well is one of those goals, like losing 10 pounds or flossing daily, which everyone feels they ought to do.

The best predictor of future behavior is current behavior. Focusing on specific examples of current behavior is the best way to defuse aspirational speak. If an interviewee talks about wanting to lose 10 pounds, ask what exercise she’s done in the past week. If someone talks about wanting to streamline the project management process, ask which meetings or bottlenecks she’s eliminated so far.

If your interview notes contain lots of exclamation points but little real-world behavioral evidence, you’re not validating your hypothesis. We’ll talk more about validating (or invalidating) your hypothesis later in this chapter. Remember that the most important thing is to do one or the other quickly. If your hypothesis is invalid, the sooner you know that, the sooner you can adjust it and make progress. If you validate your hypothesis quickly, you can get on with building an MVP (which I’ll cover in more detail in Chapter 7).

But before we get too far ahead of ourselves, let’s talk about your notes, which will help you in validating or invalidating your hypothesis.

For projects where I am the primary interviewer, I keep a single Word document with all of the notes from all customer interviews. Right before I talk to a new person, I add a few blank lines, type in the person’s name and company in bold, and copy and paste my simple interview template (which was presented in Chapter 5).

If you have multiple people interviewing, Google Docs is a better choice because it supports co-editing. You won’t need to worry about accidentally overwriting each other’s notes if two or more people are conducting interviews at the same time.

Having all of your notes in a single document makes it easier to search later on. If you remember that someone previously mentioned “email integration,” you could search for that phrase instead of having to remember who said it. It’s also easy to see the number of times a specific word or product name was used. You can do this across multiple documents, but it’s much slower.

Some people I’ve worked with recommend Evernote for customer development. One advantage of Evernote is its availability across multiple platforms. If you think of something later, you can look it up on your phone or iPad and make a quick note about it. But personally, I use Word or Google Docs.

I use a second Word document when creating summaries. Using a separate document for summaries makes it easier to skim and look for patterns. This document is lightweight enough that I can share it with my team without burying them in information.

The first step is to summarize what you learned from each interview. But don’t stop there. You shouldn’t be simply shortening what the interviewee told you, but drawing some conclusions and recommended next steps. Customer development is only effective if it drives your team to focus and avoid spending time on ineffective activities.

If the customer disagrees with the team’s beliefs, a summary document probably won’t be enough to convince people to change their minds. This is why I emphasized the practice of pair interviewing in Chapter 5. A team member who has participated in a few interviews, even if only as a note-taker, will be more receptive to summarized feedback. Pull out the full version of your notes for people who need more proof of what customers are saying, and see if you can persuade some of those folks to act as note-takers in future interviews.

As you remember from Chapter 5, the most important areas to capture in note-taking are:

Those areas are where most of your learning will happen. When I’m summarizing, I force myself to boil each interview down to five to seven bullet points containing the most interesting information. This makes me prioritize what I heard and pick out the most valuable things I learned from each interview.

When you have multiple interviewers, each person can prioritize her own notes into a total of five to seven bullets for three sections: Validates, Invalidates, and Also Interesting.

If you’re not sure how to prioritize, let the interviewee’s emotion be the deciding factor. A matter-of-fact statement that supports one of your assumptions is interesting, but not nearly as interesting as an enthusiastic five-minute tangent on a competitor’s product.

Here’s a five-point summary from a KISSmetrics interview:

Joanne, marketing director, midsized company

In Chapter 5, we talked about taking a few minutes after your first customer interview to figure out which elements were most successful. Most likely, the first changes you made to your interviews related to your tone and the phrasing of your questions.

Once you’ve completed two or three interviews, you’ll have to make a more critical assessment. Are you learning what you need to learn?

Within the first five interviews, you’ll encounter at least one person who is really excited about your idea. At minimum, if you’ve identified a good problem to solve, someone will give you a direct referral (“This doesn’t affect me, but I know exactly who you should talk to”). If you’ve heard something like that, great! Keep going.

If you haven’t, one of the following is likely true:

Either way, this invalidates your hypothesis (remember that the hypothesis consists of two parts: that this type of person has this type of problem).

It’s possible to talk to one outlier. It’s possible to talk to two outliers. It’s highly unlikely to talk to five in a row. If five people aren’t interested, it’s probably because you’ve invalidated your hypothesis.

“But wait!” you might say. “Maybe I wasn’t asking the right questions.” That’s unlikely. When you’re talking to a person who really cares about solving her problem, her passion will shine through and she’ll answer the question you should have asked.

Once you’ve heard your first really excited person, you may want to think about how quickly you can assemble an MVP. I generally recommend starting with 10 interviews, but if you can put together an MVP in less time than it would take to do 5 more interviews, why wait?

No matter how excited someone sounds about your product, the real test is whether or not he is willing to throw down money (or some other personal currency, such as his time, a pre-order, or an email address). If you can figure out a way to collect currency from a user within a couple of hours, go for it!

Within 10 interviews, you’re likely to have heard some repetition. This may come in the form of two or three people expressing similar frustrations, motivations, process limitations, or things they wish they could do.

As I mentioned earlier, the real answer is “it depends.” Remember that the aim of customer development is to reduce risk. As such, the number of interviews you need to do is inversely correlated with:

The best indicator that you’re done is that you stop hearing people say things that surprise you. You’ll feel confident that you’ve gotten good enough insights on your customers’ common problems, motivations, frustrations, and stakeholders.

Typically, it has taken me 15–20 interviews to feel confident that the problem and solution have potential. Between recruiting participants, preparing questions, taking notes, and summarizing, that equates to about two weeks of work. That may sound like a lot of effort, but if you learn that you can cut a single feature, you’ve already justified the investment in customer development.

This process can and should happen in parallel with developing your MVP.

One of my recent customer development projects involved learning about Yammer and how our customers are measuring success with it.

We knew that many customers were reporting on Yammer usage and successes to their bosses. (Any employee can start a free Yammer internal social media network for their company without top-down permission. As a result, there is often internal selling by advocates of Yammer to executives who need to learn more about its value.) They expressed frustration with our existing reporting tool, which we built based on “what customers asked for” as opposed to controlled research into their needs and behaviors. Before we committed to building more functionality, I wanted to make sure that we fully understood their problems so that we could provide a more effective solution.

Over a two-week period, I conducted 22 short customer interviews. By the last few interviews, I wasn’t hearing any comments that were both new and significant. I could confidently make some assertions:

Based on that information, I was able to identify that a couple of requests were tied to genuine pain points and recommend not building the others. More importantly, I was able to summarize what I’d learned from the interviews and share that back with our customers. This was an extra step that is particularly helpful with existing customers. Closing that loop—showing that we had taken their concerns seriously, identified their problems, and pointing to the changes we had implemented in response—strengthened our relationship with those customers.^[52]