THE NEXT STEP OF the shipping process is making your product idea understandable and specific. If you’ve defined a mission and a strategy, then you have an understanding of who your customer is and what that customer needs. You also know what you need to do better and differently than your competition. With this knowledge and some inventiveness, you should be able to brainstorm a rough product idea. Or, if you’re like the vast majority of us, your management said, “Go build X,” and now you have to use more than one letter to communicate your objective to your team. In other words, how can you make the product real enough in words that designers can make mocks, recruiters can hire engineers to build it, and you can get funding to buy donuts and servers?
As you try to make your product understandable and specific, you will uncover assumptions that you’ve made about customer problems. These assumptions were baked into your strategy and your mission, because both your strategy and your mission followed from customer needs. I hate to break it to you, but you might be wrong about what customers need. We all know that Amazon, Google, and others have been wrong many times. So you’re probably right, but the best way to prove you are is to give customers a product and see what they say.
Serial software entrepreneur Eric Ries seems to agree with this approach, and makes a compelling case for building what he calls the minimum viable product in his book The Lean Startup (Crown Business). Ries defines the minimum viable product as the smallest fraction of your product that a sufficient number of customers will use in order to validate an assumption. You may only need a handful of customers to know you’re on the right track, and you may only need to validate one assumption at a time. Regardless of how big your minimum viable product is, you can still follow the product definition process. You will want to repeat it quickly to test assumptions and deliver great incremental progress to your customers. If your iterations are smaller and faster, you’ll spend less time guessing about what customers need and more time acting on what customers tell you—and that will lead to greatness.
There are 10 major steps to the product definition process. Each step builds on the step before. Ten steps may seem like a lot of process, but some of these steps are easy, and you can choose to do some of them (like writing a press release) only once at the beginning of a series of small iterations, rather than for each small product update. Step 1 of the product definition process begins after you have figured out your strategy. The process ends at step 10 with a fully defined and clearly articulated product that you can start to code up with your engineering team.
Write a press release. An unusual way to start, this is a less-than-one-page document that drives understanding and clarity and follows from your strategy. Amazon loves this approach. You can probably hammer this out in a couple of days.
Create a living Frequently Asked Questions (FAQ) document. This running document collects objections and details that must be addressed. You can start this document in an hour and then add to it in your “spare” time before and after the release. It’s very inexpensive to build and maintain, particularly if you build it in a wiki or Google Doc.
Make wireframes or flowcharts. Wireframes and flowcharts describe your product visually and help make discussions and answers more concrete. You might spend a day or a week on these drawings. They’re one of the most powerful communication tools you have, and they’re worth the effort.
Write a one-pager or 10-minute pitch deck. This single-page document describes your product in enough detail for a senior executive or most venture capitalists (VCs). This deck will have the same content as the one-pager, but is used when you’re presenting. A draft one-pager will take a couple of hours to build, and the same goes for the pitch deck. I find that it takes one to two weeks to refine the one-pager and pitch deck after I have a draft because I have to test them out on people and collect a lot of different opinions. Sifting through this data and figuring out how to make these documents sing takes some brain time.
Add application programming interfaces (APIs) to your FAQ. APIs are the first technical tentacles of your product, and you’ll fully integrate them into your requirements in step 6. You can probably draft a rough cut of your APIs in a few hours and refine them over time with the help of your engineering team.
Write the functional specifications document. This document is also known as a product requirements document (PRD, at Google), or marketing requirements document (MRD, at Microsoft). Regardless of its name, this is the big document. It’s the bible that describes in detail how everything will work and why it works like that. You’ll fill in sections by copying from your press release, FAQ, wireframes, one-pager, and APIs. To these major ingredients, you’ll add spices like your capacity plan, nongoals, and clear use cases that shed light on all the corners exposed in the FAQ.
The functional specifications document can take anywhere from a couple of days to a couple of weeks to finish, depending on your product scale and how mature it is. If your product is immature, you want to make the product as small as possible so you can test your assumptions. If your product is larger and more mature (e.g., Apple’s iPhone), you will need a more robust and complete functional specifications document.
Review the product with design and engineering leadership. The goal of this step is to get buy-in from the individual contributors and solicit their advice so you expose all potential edge cases. If you can pull everyone together into an offsite meeting, you can get this review done in a day, although you won’t have the more nuanced feedback you’ll get from people who have had a chance to scratch their heads for a bit about what you’re proposing. If you have to nag your team to read and review your document—well, you know how that goes.
Test the product concept on customers. At this stage, you need to make sure that you’re solving the problem you set out to solve. You can get a good cognitive walkthrough done in a day, and you can get online feedback in a few days.
Name it, price it, and forecast your revenue. While you can sometimes delay these items and operate on faith that you’ll be successful, I find I sleep better when I know that there’s a solid upside for investors in the product. Also, while some MBAs will spend two weeks on a pricing or revenue model, I think you can (and should!) complete this step in less than a few hours. If you spend more time than that, you’re probably trying to be too fancy.
Pitch your product to the execs. You can now take your product to your executives or VCs for final approval. You’ll use your 10-minute pitch, your FAQ, and your wireframes, and leave them with the functional requirements. If you can sell your product upstairs, you can start building it downstairs. It should take you 30 minutes to pitch your 10-minute deck (more about this later).
These steps can look overwhelming, but don’t worry, they’re not super-hard to complete. This detailed product definition process is a very granular and linear breakdown of an otherwise messy experience. If you complete the steps one at a time and celebrate each milestone you pass, you can actually have a pretty good time. Most of these steps, except for steps 6 and 7, are fast and fun (if you’re a geek like me). So let’s start at step 1 and build a product.
An unorthodox but otherwise great way to start defining your product is by writing a press release. Jeff Bezos and company pioneered the “write the press release first” approach at Amazon. The concept is that you have one page in which to make the marketing announcement. A great press release or blog post communicates critical information that succinctly describes the product. The benefit of starting with a press release instead of the FAQ or one-pager is that it is inherently brief, readable, and focused on what the real product will mean to real users.
A good press release or blog post contains six things:
What your product is named.
When it will ship.
Who it’s for.
What problem it solves.
How it solves that problem. Briefly!
What’s so great about it that the CEO will go on record espousing its virtues.
Note that a press release or blog post doesn’t go into deep details. It rarely includes graphics and never includes financials. A press release is a crisp summary of what, when, and why, from the customer perspective. If you’re following the earlier advice, you should already be thinking about your product from the customer perspective, and that will make the press release easy to write. In fact, all of these items follow directly from your strategy. For example, you should already know what the CEO will rave about in the press release: it’s your unique approach to the market.
When I worked on Google Apps, I helped write the blog post that follows. There are a couple of prefatory paragraphs that speak to general business pain points (in 2009 it was harder to deploy Google Apps), but in general you’ll see how it conforms to the requirements of a good blog post. Since it’s a blog post, you’ll see that instead of a quote from the CEO, I used a testimonial from a major customer. We were extremely happy with the performance of this post and how well aligned our product was with customer needs, and that is precisely what you’re trying to accomplish at this stage of your development.[1]
TUESDAY, JUNE 9, 2009
Over the last year, we’ve had a razor sharp focus on making it as easy as possible for businesses to deploy Google Apps. In the last few months you’ve seen some of the results, from offline Gmail to user directory synchronization to full BlackBerry® interoperability.
Today we’re excited to remove another key barrier to enterprise adoption of Google Apps with Google Apps Sync for Microsoft Outlook. Google Apps Sync for Microsoft Outlook lets you use Microsoft Outlook seamlessly with Google Apps Premier or Education Editions.
Many business users prefer Gmail’s interface and features to products they’ve used in the past. But sometimes there are people who just love Outlook. For them, we’ve developed Google Apps Sync for Microsoft Outlook. It enables Outlook users to connect to Google Apps for business email, contacts, and calendar. And they can always use Gmail’s web interface to access their information when they’re not on their work computer.
Key features include:
Email, calendar, and contacts synchronization. For email, the plug-in uses the offline Gmail protocol, which is much faster than IMAP or other methods.
Free/Busy lookup and Global Address List functionality, which makes it easy to schedule meetings with your colleagues, regardless of whether they use Outlook’s calendar or Google Calendar.
A simple, two-click data migration tool that allows employees to easily copy existing data from Exchange or Outlook into Google Apps.
If you read this post and said, “If I were Eric Schmidt, I’d understand why these guys invested a couple years in this engineering effort—good job!” then starting with the blog post is right for you. If you said, “This guy is a ding-dong. When can we get to the APIs?” then you might want to skip step 1. But don’t skip any of the other steps, OK?
If you’re going to skip the press release, then complete the next two steps. You’ll end up building a one-page document that says roughly the same things that the press release does—it’ll just be harder because you didn’t start with a customer-facing message. Other situations where you may skip the press release can include internal systems development, feature improvements, and corporate environments where something novel like a predevelopment press release won’t be met with applause.
As I move forward in the product definition process, I find that I accrete tons of questions, many of which are extremely important because they inevitably point to product gaps. I immediately write down these questions in an internal FAQ document and answer them if I can. I like dumb questions because when I answer them, I feel like I’ve made progress without expending much effort. It’s a rare and delightful feeling for me.
If I think a user may ask the same question that I receive, I write the question in an “External” section of the same document. I also continue to update the document with new questions as they arrive, so the document becomes a “living” source of truth for people with questions.
When I get a question I can’t answer, it goes into the FAQ too, along with the hope that someone else will answer it. Worst case, you can use the FAQ just like a personal bug list or source of topics for discussion with your team. When the number of open issues approaches zero, you’re ready to write a quality one-pager or product requirements doc.
There are two major benefits of building an FAQ document. First, the FAQ is great because it saves you a lot of email and can put out internal fires. It answers obvious questions without you having to send a long email. More important, when you get heckled about minor points, simply pointing folks to the FAQ is enough to indicate that you have done your homework and will douse most hand-waving complaints.
Second, the FAQ becomes a valuable resource for your support and tech writing team since they can organize all public content around it. And because you’ve organized it into Internal/External sections, they already know what they can say publicly. Building the FAQ is a great idea because it will save you time during the most time-crunched part of your product development process—right as you’re getting ready to release and your team needs to finalize the support content.
As you’ve been answering the questions in the FAQ, you’re likely encountering questions that are best answered visually, particularly when it comes to delicate parts of the user experience (UX). It helps immensely to draw flowcharts for user workflows and system interactions. Similarly, drawing coarse wireframes that walk through the user experience can make the product more concrete and will be incredibly valuable in the 10-minute pitch that we’ll discuss later. Sketches you make on the whiteboard or on plain paper and then photograph with your mobile phone are also a wonderful way of communicating your ideas.
I think the process and techniques involved in drawing wireframes or flowcharts is so important that I’ve dedicated a whole section to it in Chapter 4, so I won’t dwell on it here.
At this point in the product development process, you should have a good idea of your customer, the problem you’re trying to solve, and how you’re going to solve it. The next step is generally to get preliminary support from your engineering team, your executive management, VCs, or other stakeholders. You want to check the temperature of your investors at this stage because step 7, the big document, is right around the corner, and you don’t want to overinvest in that document if you can help it. The other benefit of building the pitch at this point is that you can state the assumptions from which you started more clearly; you’re likely much more erudite at this juncture.
A one-page summary of your product and/or a 10-minute pitch deck are all you need. Feel free to include your wireframes or flowcharts and not count the space they take up in the one-pager.
At Amazon it’s critical to have a one-pager because that’s how the business operates—the senior vice presidents (SVPs, a.k.a. the “S-Team”) all sit around a table and read your document quietly and then when they’re all done, they discuss it. This is a strange dynamic to experience; it’s kind of like sitting in an SAT where everyone wants to be the first to put his or her pencil down. For better or worse, it’s how Amazon’s SVPs have worked for years now.
Google operates differently than Amazon. At Google you need a pitch deck, even if you plan to speak without slides, because you will need to present in person and Google hasn’t established the SAT system that Amazon has. You can find instructions on how to build a great 10-minute pitch in Chapter 5.
For VCs, you’ll need both a one-page summary and a pitch because you’ll need to send something in email (the one-pager) and present something (the pitch deck). Regardless of where you work, both of these artifacts are an extension of the press release and they contain the same basic content. The five major elements you must include in your summary are:
What it’s called
Who it’s for + how many of these users exist
What problem it solves + how valuable that solution is to a user
How it solves that problem + what that solution looks like, and why your approach is durably different than the competition
When it will ship + what the major milestones are
For VCs only: team profile
The one-pager and pitch deck extend the press release to explain the market opportunity (number of users), the revenue opportunity (value of the solution), and sustainable competitive advantage (durable differentiation). If you can’t make a clear statement about these topics in one page, keep working on it.
It may seem challenging to fit all these points into a 10-minute presentation or a one-page document. You’re right; it is challenging. Most team leads fail the first few times they try to create such a concise, powerful document or presentation. I’ve seen hundreds of decks from companies looking for investment or to be acquired, and very few have built a coherent pitch that can be presented quickly and clearly. Therefore, building a clear, concise pitch is an essential task for the team lead. It will get you instant respect and attention, and you’ll get your message across to boot. There are detailed instructions on how to build the 10-minute pitch in the section How to Build and Give a Great Presentation in Chapter 10.
One final note on this particular 10-minute pitch: most of the people to whom you’re going to pitch, whether they are inside your company or outside, or engineering or business focused, are going to be very smart and knowledgeable about your industry. However, they won’t have any context on your specific business. The best way I’ve seen to present to these groups is to start with the user and move outward (see the aforementioned outline). Do this quickly and then allow the smart folks to whom you are presenting dig into areas that they care about. Your audience needs to do this level of investigation not because you don’t have the data but because it’s their way of interviewing you. Embrace the interview!
The goal of adding APIs to your product description is to explain how your team will interface with other teams. APIs can also explain how external developers might work with your systems and what kind of data you’ll store. Another one of the advantages of articulating APIs in advance is that it helps you build a great platform; service-oriented architectures (see Chapter 3) hinge on these APIs, so writing them up front helps everyone a great deal.
Most important, APIs make the boundaries of your system very clear and concrete. Concrete boundaries help people understand where the responsibility for various functions or outputs lie. That level of understanding between parties enables a good conversation about your product requirements because you start from a shared understanding and vocabulary.
APIs can be very useful but can also backfire because the engineers you work with may feel that APIs are their territory. Be careful—feel out the team first. If they understand that the idea behind you writing APIs is so that high-level management can agree on which teams will own which data, and which interfaces must be maintained as part of your partnership, they’ll likely acquiesce. If they don’t, don’t fight for it. I like to remind myself in these times that I serve at the pleasure of my engineering team, and I find a different way of getting the message across.
One example of writing APIs up front comes from a project at Amazon, when my team built the content scoring system for Amazon’s customer reviews. As part of the product definition process, I needed to tell the customer reviews team how they’d get scores from our system. So I wrote this simple API into the FAQ:
float getContentQualityScore(string reviewId,
string userId){}This example is simplistic (and is written in some unknown programming language), but it shows a few important things:
We’ll assume that the index is not the ASIN (the Amazon product ID) but rather the Reviews’ ID system.
We’ll assume that the score will be a noninteger number.
We had the idea that we could support a Netflix-style review ratings system for people like you, so that we can give you the reviews that are most relevant. We figured it was best to put that in the API.
You may want to provide less detail than I show in this API. But if you’re building anything developer facing, even if those developers are other engineers within your business, it’s worth considering digging deep so that you don’t expose a problem later. For example, if Reviews were assuming we’d give the content a letter grade, its sort algorithm might be completely different!
You’ve crafted a product idea that solves a real need for a real group of customers. You’ve pitched and received buy-in from your essential stakeholders. You’ve worked with your dependencies to define how you’ll interface. It’s now time to get into the implementation details and build your big document.
At Microsoft, this document is called a marketing requirements document (MRD) because the market research folks put together a list of requirements from the customers they interviewed. At Google, it’s referred to as a product requirements document (PRD) because anyone can make them, but product managers generally write them. Amazon calls these documents functional specifications because they describe how the product is supposed to function for the user. These documents are all effectively the same; they describe in detail how users are supposed to experience the product. None of these documents includes technical details on how the systems operate behind the scenes. These details are covered in a technical specification or a design document that your engineering lead will write.
The audience for a functional specification is your engineering team, your design team, and occasionally your marketing team. Your functional spec will have nine sections, which I’ll cover in detail. They are, in order of the most general information to the most specific information, as follows:
Introduction (Mission and Strategy)
Goals and Nongoals
Use Cases or User Scenarios
Mocks or Wireframes
APIs
Capacity Plan
Dependencies
FAQ and Open Issues
Key Milestones
Your introduction is your one-pager. You may think it’s unnecessary to include this information in a detailed functional spec, but your engineering team will appreciate it. It describes why you’re doing what you’re doing. It provides necessary context for people new to the project and establishes terminology that you might have forgotten you are using.
Your goals are a more detailed description of your objectives from the introduction. It’s worth calling these out, and the nongoals, so that the engineering team can organize around them. List your goals in priority order. Make them clear and brief. Prioritizing your goals will also help the engineering team make good design decisions.
If a goal is not 100% obvious, you need to take the time to explain why it is a goal. If you don’t, the goal, and the ensuing specific requirements, will seem arbitrary. Engineers like arbitrary requirements just as much as they like arbitrary dates, and pay just as much attention to them.
Nongoals are useful for addressing objections or clearing the air when your constituents have assumptions. For example, if your design team is concerned that your device assumes the existence of a keyboard, you can address their concern by saying, “Mobile and nonkeyboard support are nongoals.”
Sometimes I see use cases and user scenarios broken out into two sections. Use cases are succinct statements about actions that users must be able to perform, while user scenarios are more narrative stories about how users experience the product.
For example, a use case for Hangouts on Google+ was:
User can share screen.
A more interesting use case, which would be added to the list of use cases that includes the previous example, is:
User is prompted to take over screen sharing when the user tries to share his/her screen and another user is already sharing.
You can see how these use cases can get a little challenging to read. They are nicely specific, though, and make it easy to figure out which pieces of engineering work are required. Agile development makes good use of use cases (a.k.a. “stories”) by describing each core task as a use case, following a framework like:
As a participant in a hangout, I want to be able to [share my screen with others in the hangout].
This agile model emphasizes the user type and action and works reasonably well. But when the actions become complicated, user scenarios can become very powerful. For example, if we were to rewrite the Hangouts use cases as a user scenario, you can see how a developer would end up with a better sense of the intended user experience.
Jody wants to share her screen. She clicks the “share screen” button. She is prompted to select the window she wants to share, and she can also choose to share her whole desktop. Pictures and labels describe the options she has, and the pictures are updated in real time, like little videos. When Jody clicks on an option, she starts presenting her screen, unless someone else is already presenting. If someone else is presenting, Jody is prompted to take over: “Rick is currently sharing his/her screen. Do you want to take over and share your screen instead?” If Jody says No, she returns to her initial state. If Jody says Yes, Rick’s video switches back to his camera and Jody’s screen is shown.
Regardless of whether you write use cases, user scenarios, or both, prioritizing them is important. Prioritizing your use cases early will help your engineering team prioritize engineering tasks and optimize designs. I once heard an Amazon exec say, “Prioritization is failure!” Run, don’t walk, from these people. Prioritization is a critical tool that enables your resource-constrained engineering team to trim a feature set to meet a date. Prioritization at Google and Amazon typically works the same way, and there are four levels:
- P0
Can’t demo without it.
- P1
Can’t ship without it.
- P2
Nice to have.
- P3
Ha ha ha!
P3 is where features go to die, and even P2 features are likely candidates for the culling. Because this priority schema applies to bug triage as well, your team will develop a common vocabulary and calibration. Calibrating importance and urgency is hard, so it’s good to establish priorities and start calibrating early. See the section Triage Bugs Properly in Chapter 5 for more information on how this works in the later stages of your shipping process.
In some cases, you might want to help your team by prefacing a use case with “[V2]” or some other label that lets them know they can deal with it later. In reality, P3 implies V2, but even so, it’s nice to know that nobody expects that use case to be supported in V1. It’s still best to articulate the use case, even if it’s in the next version, because it helps your engineering and design team envision the system they must build in the long term, and it will save you from a mind-bending mass of “yeah, but what if…” type of questions.
Because you’re following a tried-and-true product definition process, you already have some rough mocks or wireframes. Paste them into your functional spec, and they’ll reinforce the story articulated in your user scenarios.
A capacity plan is a rough estimation of how many users will use your software over time, and it’s important for your engineering team. Your team will use your estimates to figure out where to add caching, what kind of servers and storage you’ll need to provision, and what licensing concerns there might be, among other things.
It’s very hard to estimate usage. I once heard one of the principals on Xbox Live talk about their capacity planning for launch. He said they picked the biggest number they thought they’d achieve in their first year, and then they doubled it. Even so, they were so successful that they had to add capacity in a rush.
The Xbox approach seems excessive, but it isn’t completely wrong. Amazon and Google generally do capacity planning in a similar way. You start by building a spreadsheet that plans your capacity by years or quarters. Estimate both storage usage (number of posts, images, image size, etc.) and traffic usage (visitors, visitor dwell time, number of page views per user). At Google, you need to go a level deeper and estimate egress traffic (data leaving the datacenter) and ingress traffic (requests to your servers). For most applications, such as a website like Amazon, egress traffic dwarfs ingress. However, if you’re building an application that uploads photos, videos, or other user-generated content, your ingress bandwidth will be much larger, so plan accordingly.
Build in buffer and communicate that buffer as an assumption. For example, you might say, “I am assuming 100% buffer for unexpected growth.”
Assume your daily peak is three to four times the average usage. This number is a safe assumption because it represents the overlapping peak of US users across time zones. If your product is significantly different, like a software update system designed to run when computers start up, your peak may be even larger than three to four times the average usage, so adjust accordingly.
If you are a global business, you may want to take into account how you’re going to alleviate latency issues. Will you deploy to multiple datacenters? Will you use a content delivery network (CDN, also known as an edge cache) such as Akamai?
Plan for spikes in usage. What happens at launch? Spikes generally have very different user behaviors, so you can estimate usage differently. For example, if 60 Minutes runs a story on your product, what happens? This happened to me at one startup and we got lucky. Latency increased and we got paged, but we continued to serve traffic. It was a suboptimal situation, but we lived to code another day.
Plan a fallback strategy for the worst-case scenario. Your worst case could be a distributed denial-of-service attack or a Wall Street Journal article, or it could just be a datacenter failure. It doesn’t matter what catastrophe you imagine. You need to have some kind of draconian crisis management system in place, such as a throttling system, a “we’re busy” page, or a static version of the application that you can serve out of a CDN.
The capacity planning part of the document is a great place to spawn discussions with your engineering team about systems design. You should understand what happens when you run out of capacity. Will parts of the system fail completely, or will they slow down? Do your systems scale horizontally (meaning that for each computer you add, you get one computer’s worth of capacity) or nonlinearly?
Ultimately, you are responsible for making reasonable predictions, and your engineering lead is responsible for building a system that scales to this prediction. Like most things that are predictions, you need to be careful to spend the right amount of time on this analysis. Spend a few hours working on a draft, run it by a few people on your team, double your estimates, and move on.
Call out all of your dependencies, and if you have contingency plans, call them out too. The functional spec is a good place to aggregate owners for each dependency. When you circulate your functional spec, you can include your dependency owners so they know they are on the hook to support you. They may only read your introduction, but that’s OK, since it’s one page long and very clear!
Your dependencies don’t need to be excessively detailed. I try to provide a simple description of why we have the dependency and what the impact on the dependency will be, such as traffic and exceptional conditions. For example, when I worked on Google Pack, I had the following dependency:
Download Service (owned by dl-eng@, contact m_@)—We need the download service to host the signed third-party binaries. We will update through biweekly pushes and signed clients will request the payloads over HTTP so we don’t need high-volume SSL traffic. Instead, we’ll just request a manifest over SSL and the manifest will include the signatures of the binaries. We expect that emergency binary updates will be rare but likely—we estimate 1–3 per year.
Link your FAQ into your functional spec. You could also copy it into the document, but I prefer to keep it independent so that there is always a single FAQ and you don’t run into a revision problem where some readers will see the old version.
Link your open issues document into your functional spec.
If you have hard dates you need to meet (e.g., Apple has its World Wide Developer Conference, you might be running out of funding, etc.), add those to the doc. While it’s great to call out major milestones for feature complete, trusted tester release, and other dates, you have to tread carefully since you haven’t sized the engineering work yet. Focus instead on hard dates, not engineering milestones, and link to your project plan (see Chapter 5).
The hard part is over! You’ve written this big document that nobody will read completely, but each section is relevant to some stakeholder, and the act of writing it has brought laser-like clarity to your product vision. Also, it’s a good career artifact, so pat yourself on the back now that you’re more employable. Good job.
Your next step is to tear the document apart and find all the problems. If you’re not prepared for the searing critique and course changing that will likely ensue, you may end up disheartened. Don’t fret. This next step, in which your engineering, design, and business teams pull at all the edges of your product, is a tempering phase. Nobody creates a perfect product on his or her first try. That’s why you have a team. Deep breath now.
Your team is going to find edge cases, or corner cases, which are product behaviors and scenarios that occur rarely. Don’t be irritated by these details, because like most edges and corners in the real world, they can hurt if you’re not looking out for them. As in the real world, the best thing to do with edges and corners is to cover them just enough so that nobody loses a finger. But before you can even do that, you need to discover them.
Aaron Abrams, an experienced program manager from Motricity who’s managed large-scale projects for clients like Motorola, says that the best way to discover edge cases is by “taking a slow walk through the functionality.” The slow walk is a great way to approach the problem because you really need to take time to reflect creatively on ways that users will break your software or use it in a manner other than you intended. As you take this slow walk, write down all the potential edge cases and address them either in the FAQ or in the product requirements document.
In addition to making sure you’ve addressed the edge cases, you need to ensure that your engineering and UX team has bought into the plan. The best way to do this is by initially circulating your product requirements with your development lead, your test lead, and your UX lead. If you have customer support, legal folks, PR, or other individual-contributor-level leaders who might be interested in the product, now is a great time to have them raise a red flag.
Next, since you’re doing the slow walk anyway, include your leads in the process. Treat the meeting like a design review meeting (more on this meeting in Chapter 10). The review meeting will give the leads who were too busy to actually read your document (and there will be more than one) a chance to comment. Your leads will likely provide some great perspective that you don’t have and identify further edge cases for which you need to adjust.
At this stage of the project, you have one more tightrope to walk: you must acknowledge and incorporate all the edge cases that you and your team expose, and you must defend the core principles behind the product. If the product is deeply flawed, you’ll find that it’s really hard to get the engineering team onboard. Do you think you can sell the product to customers if you can’t sell it to your engineering team? Engineers are a special bunch, all pajama-clad and dismissive of patents, but that doesn’t mean they are not savvy consumers. Consider their feedback very carefully and spend as much time as you need to convince the team that this will be an amazing product.
If your team has now bought into the idea, you’re in good shape. If they’re not, the solution is simple: repeat step 7 again, starting with your mission, and figure out where you lost the team. Make adjustments to your product plan until it sings and the team has bought in. Everyone on your team doesn’t need to agree that the plan is perfect, but they do need to agree to go in the same direction and treat this product plan as an experiment that has a reasonable chance of success. When the team gets to that point, move on to step 8.
“Test on customers” seems like a bad idea, but the reality is that both Amazon and Google effectively test production software on customers. True, some of the tests are “experiments” released on a subset of customers. And true, Amazon does staff some test teams, and Google believes adamantly in unit testing (but not in functional testing), but the reality is that lots of bugs make it to production. That said, I’m not actually advocating that you now go build your software and throw it out there to see if it sticks. Instead, bring your deck and your mocks to a group of users or potential users and see what they think about your idea.
The reason you want to perform this level of testing is because you want to avoid building a product that nobody wants, or shipping a product with one essential feature missing. As noted by Google SVP Alan Eustace, teams can easily get wrapped around the axle building a perfect solution for a customer who doesn’t exist. Previewing your product with customers will validate your goals, nongoals, and prioritization.
Some people might call these tests “focus groups.” Others would call them “presales,” or a “roadmap presentation” for existing customers. Your UX team may think about this process as a cognitive walkthrough, wherein you walk users through sketches of the user experience to get their feedback on features and utility (more on that later). How you perform this level of user tests doesn’t really matter, but doing it does. So organize three to five meetings per week for three weeks as soon as you have your deck together and pitch your product to potential customers.
If you don’t have customers yet, you can ask your UX lead to do some basic user research; the lead will bring in potential customers and interview them to see if your product fits. UX researchers sometimes find volunteers through Craigslist and compensate them with $100 Amazon gift certificates. If you have a marketer who you can work with, try to leverage him or her to pull in a few groups of customers. Family and friends work well too, but be careful not to skew your test customer base too far to the geek end of the spectrum.
A classic example of why testing your product idea on users is critical is the Real Names™ project that I managed at Amazon.[2] Our goal was to mitigate the problem of people writing reviews that artificially made products look good or bad. We wanted to introduce accountability into the process of writing reviews. The idea was that reviewers would have to provide their real name and have it verified by entering a credit card. It’s still running today, so feel free to go write a review on Amazon.com and check it out.
There was some debate within the company about the right way to approach the problem. For example, should we allow users to continue to post with pseudonyms or not? Ultimately, the way that I resolved this debate was by bringing the idea to a group of our top reviewers, under an NDA (nondisclosure agreement). They reacted strongly and negatively; one customer sent a flaming email directly to Jeff Bezos himself. Jeff is great about responding to customer mail, and that customer missive drove a rapid change in our approach, allowing users to have real names, pseudonyms, or a combination of both. It’s clear to me that without that customer feedback, we would have had a painful product launch on our hands. That’s not to say that the launch was smooth—it wasn’t—but at least we avoided the even bigger crisis that would have ensued had we required everyone to have a credit-card-verified Real Name.
Your work up until this point has been focused on the customer, the customer’s problem, and what you need to do to solve the customer’s problem. This is exactly the right approach. If you’ve followed the guidelines of solving a big problem that a lot of people share, your next tasks are a cakewalk. If, however, you picked a tiny niche problem, these are the steps in which you’ll finally realize you’ve made a mistake.
The basic business stuff you need to know at this point is the name of your product and how much money you can make from it. You need the name so that when you pitch to your execs or investors, you can all speak about the same thing. And you need to know how much money you think you can make so that they’ll take you seriously. To determine how much money you could make, you’ll need to figure out a price. Together, these are the only business elements you need at this point. Don’t worry about sales training, marketing campaigns, or launch tricks because you’ll have plenty of time to address those topics later, and they’re not relevant for your current audiences.
First, you need a name that can pass trademark and copyright review by your lawyers and that customers like. The reality is that time spent working on the product name is time you’ll never get back. The only thing more contentious and less objective than naming is pricing. My suggestion is to pick something descriptive and go with it.
Here’s an example of how naming sometimes works at Google. Google’s Hangouts is a product of the Google Talk team. That team of product and engineering leads probably spent a dozen meetings getting derailed about whether we should call the product “Google Voice” or “Google Talk” or something new, like “GVC.” In the end, Google’s SVP of Social, Vic Gundotra, named it Hangouts—he felt very passionately about the name. And there you have the second way of choosing a name at this stage: delegate to someone. The name of your product might seem precious, but a great name will not make or break a product, so spend the right amount of time on naming—very little.
Pricing is even worse and more painful to establish than naming because it looks scientific—there are numbers involved, after all—but it ends up largely being guesswork. So you and your teammates can spend a ton of time debugging opaque formulas in your Excel model only to find that customers won’t buy your product at half the price, or that execs want to give it away and make money on ads. I’m optimistic that if your product is good, your customer base is large, and the need is real, the initial price of your product will have little impact on your long-term success.
But you do have to come up with an initial price, and you need to be able to discuss your rationale. Rather than spend time reading 300 pages of deep economic analysis that would be largely useless for most of your software products, you should understand the three basic ways that products are priced: price to cost, price to value, and price to competition.
If you write software, pricing to cost is a bad option unless you offer technical support or a software license agreement (SLA), and even then pricing to cost is generally a bad idea. A major disadvantage of pricing to cost is that very few businesses truly understand their costs. It is challenging to account for your investment, frontline support, engineering support, future legal support, marketing costs, and more. You can get accurate costs from accounting—but those were yesterday’s costs, and you don’t know what tomorrow’s costs will be.
If you want to try to price to value, you can survey your customers. Ask customers at which price points they’d strongly consider your product, and at which price points they wouldn’t buy it even if they really needed it. Using this data, you can ask a couple of MBAs or a high school student to triangulate what the optimal price is. This approach is reasonable but useless because your product doesn’t exist yet. Customers don’t really know if they need it, and therefore your data is invalid. Also, customers are rarely honest with answers to pricing questions, and they’re not even consistently dishonest in a particular direction. Let’s move on.
Pricing to competition is pretty much the only reasonable approach but it requires two things: 1) that there’s a reasonable alternative to which you can compare your product, and 2) an assumption that the market is elastic. In other words, you must assume that more people will buy if the price is lower, and fewer people will buy if the price is higher (which is true of many products). Look at the alternatives in the marketplace. If your product offers a superset of functionality, charge more. If it’s a simpler product that offers a subset of functionality, charge less. If there are no alternatives, see the first point: you must have two alternatives.
Like the first two pricing models, pricing to competition is useless when you have a substantially new product. So what should you do? Here are some broad guidelines that work well for modern products:
Analyze your competition. If you can price to competition, you have a good starting point.
Ask customers what they’d pay even though you don’t believe them. This data will either reinforce your competitive pricing guesses or give you a place to start for a completely new price.
After you have a starting place, adjust the price so that it is easy to understand. Google Apps chose two price points: $50/user/year and free (for fewer than 10 users). Compare this to Microsoft’s many-tier Live365 offering, which is deeply confusing. Perhaps that’s part of its strategy?
Pick a price that is higher than you initially need. It’s always easier to drop the price for customers than it is to raise it.
Don’t fight for a price. Often someone bigger and badder than you has a very strong opinion on pricing. I suggest giving in really quickly and getting on with the business of shipping. Pricing is not shipping—it’s just one step on the journey. Move on!
Now that you have a price, you can model revenue. I’ve seen many team leads and senior sales folks end up stuck when it comes to modeling revenue. After dealing with this for a few years, I’ve come up with a theory of why: they are afraid to guess, and all revenue models are composed of at least 50% guesses. The remainder is mostly 25% market research culled from free Gartner Research executive summaries and 25% instinctual hand waving. It’s the 25% market research, and the fact that there are numbers on the page, that turns your forecast from a wild-ass guess into a scientific wild-ass guess (SWAG). A SWAG is what you’re trying to achieve at this stage of your product’s development. But if your SWAG is mainly guesswork, why model revenue at all?
First, VCs and many business unit leads are going to need to have some sense of whether you’re building a meaningful business. So you’ll need a model that forecasts revenue by month over a three-year time period.
Second, building a revenue model exposes your assumptions and validates your opportunity. The 25% market research you blended with your basic instincts as a consumer can provide a remarkable amount of insight when you do some basic math. You may be humbled to find that your billion-dollar idea is only a million-dollar idea in the ideal case—and that’s precisely what you’re looking for.
Third, your SWAG is iterative guesswork. Your simple revenue model enables you to understand how the various financial dimensions of pricing, support costs, and marketing will impact your bottom line. Your model will provide a framework for you to rationalize decisions.
So keep your revenue model very simple, and don’t worry that it’s mainly guesswork. As you circulate the model, people will question your assumptions; as they do so, simply adjust your assumptions to meet their assumptions. At this point, your goal is to get funded and avoid wasting time building a product that has no legs at all, not to predict the future.
Here’s how to build a very simple revenue model. You can find a copy of the spreadsheet at http://www.shippinggreatness.com.
Figure out the total size of your market in terms of buyers.
For instance, when I worked on Google Talk, I looked at the total spend on video conferencing, audio conferencing, and long-distance IP telephony. That gave me a pretty attractive market size.
Many consumer products go to market with a “freemium” model, in which most users use the free product and some users pay to use additional storage, access extra features, or get an orc-killing sword. You’ll address your conversion rate later—focus on the total market. For example, if Facebook has more than 800 million users, the total market is approximately 800 million.
Analyst reports are useful for this data, and if you’re in a startup you can probably make do with the numbers that are published in the free summaries or your VC can get you reports.
Make an assumption about growth over time. This will be your baseline. As the market grows, so will your sales.
Reduce the market to your addressable market:
Reduce your market size to the segments you’ll target—for example, small businesses, the midmarket, and enterprises.
Reduce your market size to the countries you can reach. You might start reaching only the US, but there’s a big world out there—expanding to other countries will likely have a huge impact.
Make a guess at the number of users you can reach through your marketing efforts. A simple way to do this is to figure out how much marketing budget you have and look at keyword CPM (cost per mille/impression) cost.
Make a guess at how many of those contacts convert to users.
Identify any other channels of user discovery and add those inputs to the model. For example, if you have an “invite a friend” mechanism in your product, go ahead and assume that x% of users use this mechanism to generate a successful conversion.
Now you want to calculate your revenue. Multiply the price by the additional number of users each period. If your product is a subscription product, then you can assume a renewal rate and count that profit.
In the following simple model (Figure 2-1), I’ve assumed we’re selling a goat-tossing game (a classic fast-follower move). It’s not subscription based, but it does have in-game sales, which is where the real money comes from. In my first cut of the model, I’ve assumed we should give the app away and rely on in-app purchases.
I prefer to make the editable cells yellow (shown in grey in the printed version of this book) because doing so clearly identifies which aspects of the model are guesses. In this case, most of the model is guesses. This model shows that my goat-tossing game is not a particularly profitable enterprise, even with a situation in which I’ve reached 3.4% of the addressable market. Note that instead of working from a percentage of market share to users, I made a guess about users and cross-checked that with market share. I used this approach because I feel like I have better instincts and more data about specific download rates than I do about how broad groups of users adopt games.
Before we toss the goat-tossing idea, we should take a step back and look at our assumptions. The model I built is sensitive to many variables. For example, I’m spending $1 per user that I acquire through Internet advertising, and I’m getting a total of 1.15 users from that spend (assuming viral growth). I’m collecting $3 each on average from 20% of those users, or $0.60 per user overall. So my marketing plan generates a net loss of $0.40!
The nice part of models like this is that you can easily change them. It’s much easier to change this model now than it will be to deflate the budget allocated to the newly hired director of online marketing later. To get to profitability, I’ll guess that I want to spend less money on more cost-effective but lower-reach advertising. In other words, I’ll change our maximum ad bid so we trigger ads only when we can afford them. I’ll also change to a different pricing model, $0.99 to buy, which will result in lower adoption (50% lower, I guess) and greater margin. Let’s apply those changes. Figure 2-2 shows our revised model.
That’s more like it. We are now growing slowly. We haven’t taken into account our operational costs, but on a pure product standpoint, we’re profitable. We should work on driving our support numbers down, since they’re roughly 20% of our revenue, which is very high for a game. More important, we need to come up with some low-cost ways of growing usage. Maybe the viral features could be improved?
It’s clear that this model is highly sensitive to your assumptions. This is not a problem; it’s a benefit, because it exposes your assumptions and the importance of certain variables. If you build a reasonable and simple model, it will help you understand your business and sell your product upstairs. But you also must be careful of overspending your time on the model—real users and real data will inform your decisions far more than guesswork will.
I know there are MBAs out there groaning as they look at these overly simplistic spreadsheets. I hear you, and I agree: this spreadsheet is a very blunt tool. It lacks a wealth of detail that could be added. But the practical reality is that you are a software team lead and you do not need more detail to make smart decisions about the product at this stage. You just need a SWAG. This spreadsheet is a good SWAG.
If you’re in a startup, this is the one time you can probably take a break and go get a cup of coffee, because you can approve yourself. Unless you’re funded, that is, in which case you’re in the same boat as everyone else because the board wants to know what you’re doing with their easily earned money. They’re the 1%, and they feel entitled to know what you’re doing with it.
If you’ve followed the plan and sold the product to your development team, you already know what the objections and faults with the product are and you’ve addressed them. To make the final sales pitch easier, I suggest you spend time preselling your product. Most successful leads at Google have learned how to do this because it establishes some context for your managers before they have a public reaction to your product. In environments where everyone is overly busy, like Amazon and Google, preselling is a very powerful technique.
Preselling is a pretty straightforward process of pitching your way up the food chain until you hit the folks who are going to say yes. You simply need everyone between you and the exec to whom you are pitching to say yes, and then get the folks who report directly to that exec to preview the concept favorably. If you do a lousy job pitching, the lieutenants will take you out before you ever get to the decision maker. Or worse yet, they’ll misunderstand and your decision-making exec will have some potentially insane idea of what you’re going to build. Be careful, because the risk of the game-of-telephone effect is seldom greater than when you’re half-reading email messages while trying to pay attention to people desperate for your attention.
Another approach to preselling is to “do a drive-by” with the decision maker you’re trying to influence. In such a scenario, you’re not trying to get a decision, you’re just letting the person know that one is coming. You can do a drive-by in about a minute if you see the decision maker in the hall or getting coffee. The drive-by at least sets the tone for the conversation you’ll eventually have and will alert you to any violent allergic reactions that the individual may have.
Once you get to the point where you can pitch the execs directly, you’re in for a whole new kind of torture. Take Jeff Bezos, for example. When I was at Amazon, Jeff was a well-established details guy who was also very smart. If you wanted to build something substantially new, Jeff had to approve it. I only pitched a couple times to Jeff and I’m sure I was quickly forgotten, but I do know that I never got past my first couple of slides. Jeff jumped right ahead, trying to get to the meat of the conversation. Another executive I worked with at Google used to do the same thing, but over time he learned to turn his leap-ahead to a leap-backward and ask for context instead. I had no such luck with Jeff at the time.
This kind of leapfrogging around your pitch will happen to you at Google, at Amazon, in VC, and basically anywhere you get superintelligent billionaires hearing presentations all day long. VCs tend to have slightly better manners, though, and Eric Schmidt will just do email during every second slide so you don’t notice that he’s skipped ahead.
So first, before you pitch to the one-percenters, make sure you know everything on the periphery of your product. This is a lost cause, but it’s still a good idea. There will always be something you forgot or didn’t know about because it happened yesterday when you were massaging your slideware. Accept your failure now—it’ll be OK and it’s no reflection on your own superintelligent status. When you don’t know the answer, you are much better off saying, “I don’t know; I’ll find out and come back,” than pretending you know. Remember, these are hyperintelligent billionaires—they can smell a lie like a fart in a car. Trying to talk your way out of such a failure will just prove to them that you are a ding-dong who doesn’t understand how smart they are.
Second, go where they want to go. The billionaires are asking to be guided in a weird way, so do what they want and guide them. If they’re leaping ahead and they haven’t made a wildly incorrect assumption about your product, just leap ahead with them. Don’t be the jerk with the deck who insists on walking through the slide that has the background of every team member on it when all the investor wants is to know how much you can charge. Just go to the prices. If you absolutely must cover something, then you can try saying, “We cover that in just a moment in the deck, but I’d like to touch on a couple of important things first…” But really, just move forward. Your judgment on importance matters less than what these folks think is important.
Finally, read the section How to Build and Give a Great Presentation in Chapter 10 and embrace the deck-in-one-slide approach. The billionaires like this because it leaves out lots of details but gets to the heart of the matter and lets them interact with you.
At this point in your product life cycle, you’ve found a critical user need that many people share, and you’ve proposed a unique way to solve it. You can talk about the product in one page, a 10-minute presentation, or in rigorous detail through your functional spec—even to the point where you can answer all the frequently asked questions and expound on APIs and dependencies. You’ve also managed to set nearly outlandish revenue targets that are inspiring and remind you why you got into this business.
Before you start congratulating yourself, take a step back for some perspective. If this were a date, you’ve effectively buttoned your shirt, combed your hair, and planned your first coy response when you find a suitable match at the bar. That’s it. Now you actually have to go meet someone and close the deal. Execution is the hard part, isn’t it?
I’m serious about this point. Any reasonably smart team lead can invent a “strategy” or a reasonable product idea and declare success, because we measure success at this stage by the words you put on the page and the promises you made in meetings. It doesn’t matter how good you look if you keep coming home alone. What’s really hard is driving your team to build the right software in spite of real-world dilemmas. Did I mention that you have to do it in the right timeframe? And that you have to launch with a team that loves you and the product?
[1] You can see the whole post online at http://googleenterprise.blogspot.com/2009/06/use-microsoft-outlook-with-google-apps.html.
[2] The product is described in some detail in a 2004 New York Times editorial, “The Review of Reviews,” http://www.nytimes.com/2004/08/03/opinion/the-review-of-reviews.html.