4

VALIDATION

QUIZ

To set the stage for this chapter, try answering each of the following statements with Agree or Disagree. Answers appear at the end of the chapter.

Statement

Agree

Disagree

Validation is the act of making sure a project stays within scope, budget, and schedule.

Creating a simple online survey that collects data about your potential users is a valid release that provides value.

Everything is just a hypothesis until it is tested against the marketplace.

The more stakeholders provide feedback, the more they become accountable for the direction of the product.

Being compliant with all the internal governance rules provides all the validation needed to be successful.

Validation is all about closing the feedback loop to collect and measure data. By analyzing the data, you get information that turns into knowledge that can be leveraged for future benefit.

Since value is about delighting the customers who consume and buy your products, it raises this question: When do you actually deliver value?

Does a grand plan have value? Maybe indirectly to internal people who are creating the product. But does it have any value to customers? A plan is just an assumption that the prescribed path will lead to value.

Two famous military leaders and one boxer sum this idea up nicely:

Plans are useless but planning is indispensable.

—Dwight D. Eisenhower

No plan survives contact with the enemy.

—Helmuth von Moltke

Everyone has a plan until they get punched in the face.

—Mike Tyson

Eisenhower’s distinction between planning and plans is even more important in today’s globalized economy where everything moves faster and faster. To be successful, you need to validate your business assumption as quickly as possible. Eric Ries calls this the value hypothesis.1 All ideas are nothing but hypotheses until customers validate them by paying for and using the resulting products or services. This is called validated learning. When you think along these lines, every new product feature becomes an experiment.

“Eighty percent of the time you/we are wrong about what a customer wants” is the realization from Microsoft.2 The company that can fail and learn the fastest on an ongoing basis will be the most successful and sustainable. As an entrepreneur you need to develop the capability to deliver value quickly and learn from your mistakes (or false hypotheses) as quickly as possible.

Former Cisco CEO John Chambers described this in his last public speech as CEO: “Forty percent of businesses in this room, unfortunately, will not exist in a meaningful way in 10 years. 70% of companies will attempt to go digital but only 30% of those will succeed. If I’m not making you sweat, I should be.”3

STAKEHOLDER FEEDBACK

A Scrum product development effort succeeds or fails through the Product Owner. However, the Product Owner is rarely the one and only customer. Typically, the Product Owner represents other stakeholders or even whole market segments.

It is critical to identify and communicate with all stakeholders from the beginning to ensure that you build the best possible product.

Scrum provides an excellent opportunity for stakeholder feedback with the Sprint Review, which creates transparency around the Increment. Stakeholders are the key audience for this event, where you incorporate their feedback into the Product Backlog. Putting your product in front of your stakeholders at least once a Sprint provides these opportunities:

1. Engaging stakeholders. Implementing stakeholder feedback each Sprint helps stakeholders feel listened to and will have them coming back for more.

2. Correcting course. The more often stakeholders see the product, the less you risk going in the wrong direction for too long.

3. Creating accountability. The more transparency you create, the harder it is for stakeholders to hide behind ignorance.

A good example of stakeholder validation is a Subway sandwich shop. Subway creates transparency between the sandwich creation and the customer (see Figure 4-1):

1. Engaging stakeholders. At Subway, you build the sandwich together. The sandwich maker does not go for too long without direction from the customer.

2. Correcting course. Change your mind on sandwich ingredients along the way. “Oooh, those peppers look good, can I have more of those?”

3. Creating accountability. If a customer ends up with a sandwich that doesn’t have the mushrooms they want, whose fault is it? The customer should have said something along the way.

Figure 4-1 Transparent sneeze guard at a sandwich bar

MARKETPLACE FEEDBACK

You can run your product by all the smartest people in the world, but there is only one true way to ultimately validate your idea: get it into the marketplace.

As Product Owner, your focus must always be on getting the product to its next shippable state—a Minimum Viable Product (MVP).

MINIMUM VIABLE PRODUCT

MVPs are about the validation of your hypotheses. You need to validate two kinds of hypotheses: technical and market.

Technical

If you cannot do something from a technical point of view, then there is no product. Therefore, you need to address these risks as early as possible to validate that what you want to do is technically feasible.

Market

The other hypothesis is market acceptance: Do people want to buy the product? You need to get to market as soon as possible or at least as close to the actual marketplace as possible to validate your ideas.

If your hypotheses prove to be wrong, you have not lost too much money and can still investigate other options—or pivot as Eric Ries calls it. This also helps to avoid the “trap of sunk costs,”4 the tendency to irrationally follow through on an activity that is not meeting expectations just because of the time and/or money already spent on it. Poker players call this “pot committed.”

I worked on a year-long initiative to create a first-of-its-kind online document printing product. Five months in, we had the ability to upload a file and set some minimal printing options, without the ability to pay or add more complex finishing options (tabs, inserts, bindings, etc.) At the time, management did not want to hear about going to production early as we were not yet “finished.” Seven months later, after going to production, we realized that few customers actually used all the extra finishing options we painfully added to the product. Through a Special Instructions text area, we found out what customers really wanted: oddly sized posters and banners. Looking back, the ideal MVP would have been a simple upload with minimal printing options and an instructions text field. We could have then guided product development based on what our actual retail users were asking for—speeding up the feedback loop and generating true value more often.

Hypothesis validation should be a part of your Product Backlog and can be considered active risk management. The higher the impact, the higher the priority order should be and the earlier you get to inspect at a Sprint Review and Retrospective.

To properly validate the hypothesis, of course, you need to put the right measurements in place. As discussed in the preceding chapter, there are many metrics from which to choose. Avoid using the more circumstantial vanity metrics.5 Keep your metrics value neutral—objective/free from opinion—so that they tell you the real story. Face your challenges and work through them one after the other.

MINIMUM VIABLE PRODUCT THROUGH KANO

The model described by Kano6 is simple yet powerful. It plots two types of features on a two-dimensional graph: what a customer needs and their satisfaction level (see Figure 4-2).

Figure 4-2 Kano model

These features are grouped into three categories:

Basic: Essential features that customers assume are included with the product. A product without these basic features would come as an unpleasant surprise to customers and likely not sell, resulting in negative value. A car’s basic features, for example, include the engine, seat belts, steering wheel—everything you’d expect to see in a car.

Performance: Features that customers ask for as they are not always assumed to be included. They carry your customers over the satisfaction threshold from “good enough” toward “very satisfied.” The more performance features you can offer, the better. A car’s performance features are air conditioning, stereo system, navigation system, and some of the more gimmicky things that make driving fun.

Excitement: Features that your customers have likely not even thought about, but are thrilled once they discover them. Excitement features get people talking about your product. A car’s excitement features include seamless smartphone integration, massage seats, self-parking, or even a bird’s-eye view when parking (see Figure 4-3).

Figure 4-3 Jaguar’s bird’s-eye car view

Once you cover the basic features, enough performance features, and a few exciters, you have an MVP (see Figure 4-4).

Figure 4-4 Kano model overlaid with MVP

Because no product is ever “done,” you create a series of MVPs, one after the other. Once the MVP is on the market, you collect your key value metrics, create the next Product Backlog for the next MVP, and so on.

Over time, exciters turn into performance features and eventually into basic features (see Figure 4-5). Anti-lock braking systems (ABS) were once features found only in high-end cars. Now they are considered basic in all cars.

Figure 4-5 Excitement turning into performance and eventually into basic features

MVP PATTERNS

It takes a lot of courage to release something to production before it is “complete.” However, as was already described, market validation is crucial when building something unique and uncertain. Rethinking what could possibly constitute a releasable product can help. Below are some MVP patterns that can serve as early milestones for your product. MVPs are too often associated with externally facing products looking for potential markets. However, each of these patterns can and should be considered with internal products, which arguably make up the majority of software development efforts.

PROMOTIONAL MVP

Promotional MVPs include videos, UI mockups, viral campaigns, or anything that gets people talking about your product and to maybe even raise capital. Setting up a crowdfunding campaign (like Kickstarter) could be considered an MVP. For products internal to larger organizations, a promotional MVP could be about promoting a future set of features to stakeholders to build buy-in and a business case.

MINING MVP

Mining MVPs include surveys, proofs of concept, or anything else that collects data about your potential market and therefore validates your business model. A mining MVP is likely disposable after you mine the data.

For example, before investing months of time and lots of dollars into Bitcoin integration for an e-commerce application, in the very first Sprint you could add a Bitcoin payment option for your users that instead leads to a simple survey. Then you could count how many people actually click on the option and gauge their interest further by asking targeted questions like “Do you have a Bitcoin account?” and “Given the chance, would you pay for these purchases with Bitcoin?” Data like this provides valuable information to determine the validity of your product direction without investing in a complete development effort.

LANDING PAGE MVP

Release a single page that explains the value of your product (performers and exciters) and provides a foundation from which all other features can emerge. A landing page MVP provides stakeholders a place to go to see progress.

You can also leverage the landing page to gather data about traffic or create a call to action.

Landing pages are typically easy to do build in early Sprints and get the Development Team in the habit of releasing something right off the bat.

WIZARD OF OZ MVP

A “Wizard of Oz” MVP7 looks complete from the customer’s standpoint, but behind the scenes, humans are doing all the work. The idea here is to avoid investing in a complete automated backend when you are not even sure customers will buy your services.

The classic example of this is Zappos. The founder posted photographs of shoes found in local stores, putting them on a blog page where people could order them. As orders came in through e-mail, he ran down to the shoe stores to buy them and shipped them off himself. Only after he saw that there was demand did he invest in building out a backend system, inventory, fulfillment, and the other elements of his business.

SINGLE-FEATURE MVP

Once you have a product in the market, you can start to use this foundation as a basis for experimentation. Release a function, even as small as a single Product Backlog item, and then measure and compare with the expected outcome.

PIVOT OR PERSEVERE

Scientific method is a recognized research practice that builds continuously on past experience. Future experiments are based on empirical evidence from previous experiments (see Figure 4-6).

Figure 4-6 Steps of the scientific method

The step “Publish Results” is where fellow scientists are asked for feedback. That part could take a long time, making the whole feedback loop long.

In Lean Startup, Eric Ries introduces a simplified product-centric version of the scientific method steps with the intent of speeding up the feedback cycle. Ries describes this as the Build-Measure-Learn principle (see Figure 4-7).

Figure 4-7 Build-Measure-Learn feedback loop

As described in Chapter 6, the three pillars of the empirical process control theory fit nicely into this picture: Measure is transparency, learn is inspection, and build is adaptation. This also aligns nicely with the three Vs: ideas as vision, product as (potential) value, and data as validation.

With every product, initiative, or feature there should always be the question, “Have we created enough value?” Empirical process control allows you to inspect and adapt along the way.

At the core of Lean Startup thinking are two activities:

1. Persevere

2. Pivot

When the data validates assumptions or is inconclusive, then you continue to persevere, gathering more data along the way.

When the data reveals that you are not gaining the value originally hoped for, then it is time to cut bait, pivot, and avoid the trap of complacency. Even if the expected value is there, the data could unveil a new, more prosperous path that you must have the courage to pivot toward.

Figure 4-8 summarizes these concepts.

The problem is that we do not understand the problem.

—Paul MacCready

A pivot is about adjusting your path toward your vision or even establishing a whole new vision. Having data available—direct evidence—is helpful, but intuition also plays an important role in coming up with new ideas. Shorter feedback cycles give you courage to follow intuition more often as your ideas will eventually be substantiated through constant validation.

Customer Feedback is the basis for ideas, customer data is the basis for decisions.

—Roman Pichler

Pivoting is not an easy thing to do. It requires that you accept you have been wrong and that you must adapt. Too often the acceptance of failure is masked by vanity metrics or false loyalty to your product.

Once you accept that you are here to learn as quickly as you can and reduce the time spent between learnings, you will have more opportunities to pivot and hence improve your chances of success.

In 1959 Henry Kremer offered, in today’s value, a prize of $2 million USD for the first person to cross the English Channel by a human powered aircraft. This was achieved 20 years later in 1979 by Paul MacCready. Why so long? It certainly wasn’t a lack of interest. The many previous attempts by competitors spent months and years designing and building their crafts just to see them shatter after seconds into their flight. This fear of failure would weigh heavily on most and result in even longer, “better” plans.

For MacCready, he had no such fear. . . . He did, however, set himself up to fail time and time again as he knew that in order to succeed he needed to be able to iterate and test hundreds of times. Is this failure or is it the courage to fail?8

Paul MacCready realized that it was not about the plane itself but about the possibility to build a plane that could be tested and retested within hours. He achieved about four attempts per day, and with his 223rd try, he was successful. The speed of learning was essential to his success.

Ensure that the Vision, Value, Validation feedback loop is closed as often as possible. The higher the frequency, the more often you can pivot and the better your chances of success.

Figure 4-8 Three Vs leading to persevere or pivot decision

Your limited resources provide you with only so much runway (time and budget). By the time you reach the end of your runway, you must have enough speed to take flight. The length of the runway is set, so your ability to pivot is the determining factor for success, or you find yourself crashed at the end of that runway.

QUIZ REVIEW

Compare your answers from the beginning of the chapter to the ones below. Now that you have read the chapter, would you change any of your answers? Do you agree with the answers below?

Statement

Agree

Disagree

Validation is the act of making sure a project stays within scope, budget, and schedule.

Creating a simple online survey that collects data about your potential users is a valid release that provides value.

Everything is just a hypothesis until it is tested against the marketplace.

The more stakeholders provide feedback, the more they become accountable for the direction of the product.

Being compliant with all the internal governance rules provides all the validation needed to be successful.

____________________________

1. Eric Ries, The Lean Startup (New York: Random House, 2011), 61.

2. Ronny Kohavi et al., “Online Experimentation at Microsoft” (ThinkWeek paper, Microsoft Corp., Seattle, WA, 2009), http://ai.stanford.edu/~ronnyk/ExPThinkWeek2009Public.pdf.

3. Julie Bort, “Retiring Cisco CEO Delivers Dire Prediction: 40% of Companies Will Be Dead in 10 Years,” Business Insider, June 8, 2015, http://uk.businessinsider.com/chambers-40-of-companiesare-dying-2015-6.

4. “Definition of ‘Sunk Cost Trap,’” Investopedia, accessed February 22, 2018, http://www.investopedia.com/terms/s/sunk-cost-trap.asp.

5. Ibid.

6. Noriaki Kano, Nobuhiku Seraku, Fumio Takahashi, and Shinichi Tsuji, “Attractive Quality and Must-Be Quality,” Hinshitsu 14, no. 2 (1984): 147–56.

7. Ries, Lean Startup.

8. Anthony Morris, “A Willingness to Fail Solved the Problem of Human-Powered Flight,” Financial Review, October 16, 2015.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset