If you’re not failing every now and again, it’s a sign you’re not doing anything very innovative.
—Woody Allen
In the previous chapter, you looked at some of the shifts that organizations must make in order to begin working in a Lean UX style. In this chapter, we share with you some case studies that illustrate how companies across industries are making these shifts. As you read, you’ll see themes from Chapter 8 coming up. You’ll also see organizations making compromises, balancing old and new imperatives, and working to continuously adapt their processes, all the while using Lean UX principles and methods as their guides.
Regulations and Financial Services: Lean UX at PayPal
Industry regulation presents a challenge for Lean UX. Lean UX teams like to move quickly, to experiment with both process and product, to try new things, and learn from failure. In regulated industries, however, failure isn’t regarded casually. Regulation usually exists to prevent failures and/or limit the damage that failure can cause. Still, Lean approaches can work in regulated environments—it’s all about figuring out how to experiment safely. This case study shows how one team at PayPal began their Lean UX journey.
As one of the main forms of digital payment in the world, PayPal is ubiquitous on the Internet. But unlike many Internet companies, PayPal works with money. This means that all their work must run through a thorough legal, risk, and regulatory compliance process. Is it possible to build and run regular experiments in this context? Can it be done with a complicated organizational structure and multiple stakeholders? The Hermes project proved that it could.
Fixing Checkout
In 2012, PayPal’s checkout process was stuck in the late 1990s, and the company was feeling the pinch. Complaints from customers were growing. Fast-growing startups like Square and Stripe threatened to drive down the payment giant’s market share. PayPal’s new President, David Marcus, who came from the startup world, saw an opportunity. Marcus wanted to reinvent Checkout, PayPal’s core workflow. He appointed industry veteran Bill Scott to run the project that became known as the Hermes project. His mission: to modernize the primary way in which online customers bought products and services with PayPal.
At that point at PayPal, a project of this magnitude would require a massive team and a timeline counted in years, not months or weeks. Marcus gave Scott six weeks to get it done. To help move it along, Marcus also gave the team his Balsamiq-created sketches for what the experience should entail. For Scott, a PayPal newcomer, one thing was clear: he would need to find a new way for PayPal to work. Needless to say this was an intimidating proposition.
The Team
The first item on the agenda was staffing. Scott, working with the product and design organization, gathered a core cross-functional team of eight people. This team of product managers, designers, and engineers was astonishingly small by PayPal standards at that time. A typical PayPal project then would have involved dozens of individuals and multiple locations.
All eight team members were staffed out of the same office except one—the designer (who commuted regularly by plane to be with the team in San Jose, California). With only six weeks to get the work done, there was no time for emails, product requirements documents, conference calls, and the delays that come from working across multiple sites and multiple time zones.
To ensure that their ideas met with risk and compliance guidelines the team augmented their staff, when needed, with representatives from those practices. Working with risk and compliance people throughout the project was important: this practice alone saved the team weeks of effort. On a normal project, teams wouldn’t meet with compliance until the end of the project. This big-batch approach to compliance caused delays and typically forced a flurry of last-minute product changes. The Hermes team, facing a looming deadline, couldn’t risk that kind of late-stage delay.
Getting Started and Overcoming Obstacles
Setting up shop in the nicest conference room on campus, the team spent their first week throwing out their best ideas, sketching, and coming up with an initial set of hypotheses. They then decided on a rhythm for the rest of their activities. The team settled on a schedule of designing and building on Monday through Wednesday, testing with users on Thursdays, and reflecting and planning on Fridays.
The first couple of weeks felt productive. Engineers took advantage of existing APIs that moved real money, designers sketched on the whiteboards, and frontend developers connected the pieces by implementing the work directly from the whiteboard. There were regular check-ins on status and the team built momentum and shared understanding.
Then, about four weeks into the project the designers revolted. Sharing their work constantly on the whiteboard and receiving ongoing feedback and critique from their colleagues began to feel like design by committee. “When can we go and ideate on our own for a bit?” they wondered. The team realized that just throwing a cross-functional array of professionals together in one room, although a good start, was not enough to build productive collaboration. The team needed to rethink how it worked and not just what it was doing.
The designers wanted to avoid what the team came to call “pissing on the Picasso,” a situation in which engineers respond to pixel-perfect mockups by debating what they can and can’t implement. The team refined their process to ensure that designers got some time to refine their thinking. They learned when to create sketches collaboratively and when to create and present pixel-perfect mockups. They figured out how to get thinking time, and at the same time, avoid going too long without broader team feedback. Over time, the Hermes team found their rhythm and shipped the first iteration successfully.
The Results
The Hermes team redefined the process of building products at PayPal. They created a model that the company could use to move away from silos, geographically distributed staffing, and lengthy debates and approval processes. By including previously excluded colleagues in their process, they redefined the way teams work with regulatory and risk to continue to ensure that they didn’t expose the company in a negative manner. The tweaks the team made to their design process allowed high-quality work to ship with less pushback from engineers. Finally, the things the Hermes engineering team learned laid the foundation for redesigning their tech stack to allow the rest of the organization to begin working with similar speed from prototype to production.
PayPal’s new standard practice is based on the Hermes model: it builds full-stack teams that are self-sufficient, colocated (or at least in the same time zone), and focused on rapid delivery and learning cycles.
The product the team created with this Lean approach was also a tremendous business success for PayPal. It became the core of their new checkout experience, which drove measurably higher customer success rates for sellers and buyers.
Online to Offline: Lean UX at CarMax
For companies that offer a purely digital service, Lean UX practice can be fairly straightforward. They avoid many of the tangles created by real-world interactions and bricks-and-mortar commerce. But what happens when you’re creating a multichannel service? This is the case at CarMax, a retailer that uses their online channel to drive people to their physical stores. How do you know your online service is working? How do you optimize an experience that involves online shopping, onsite browsing, and the integration of a large in-store sales force? This is exactly the challenge that CarMax faces on a daily basis.
As the largest used car retailer in the United States and a Fortune 500 company, CarMax relies heavily on both its website to inform and educate its customers about its inventory as well as its retail stores (160 of them and growing every month) to complete the actual sale.
Seeking an Outcome
In car sales, financing the purchase of the car is a critical component of the process.
CarMax wanted to optimize the online channel to ensure that customers who arrive at the store are not only educated but qualified and ready to buy. Could they do it? Archie Miller, UX designer, and Beth Sutherland, product manager, told us how their team discovered the answer.
The product team hypothesized that if customers better understood the financing side of purchasing a car, those customers would have a more successful car-buying experience when they arrived at the CarMax store. This project framing, seeking outcomes rather than committing to outputs, is at the heart of the Lean UX approach.
Lean UX + Customer Experience + Service Design
The team began its research by creating a customer journey map. To validate their thinking, they asked existing customers to create their own journey maps. CarMax’s journey map had 80 steps. The customer-created journey map had eight. This was a big insight: the customers’ world view was far less complex than the team had hypothesized.
Proto Personas
The biggest “A-ha!” moment this exercise revealed, though, had to do with how customers began their car-buying process. The exercise revealed a clear set of categories of factors that drove consumers into the car-buying process. For example, one persona that emerged (they called her Tiffany) was thrown into car buying through an unexpected life event like her car breaking down. The team felt this was an underserved segment.
They created other personas, as well, and they further validated these personas with a series of conversations CarMax ran using Ethnio to intercept shoppers on their website and engage in dialog. (Overall, the team spoke with more than 100 car shoppers to confirm these new findings.)
In addition, they learned two more important consumer insights about this group of customers:
Most customers who fell into this category did not enjoy the car-buying process.
Many of these shoppers faced a confidence gap about whether they would be approved for financing to purchase a CarMax vehicle.
Many people knew their overall credit profile, but didn’t know what it would mean for their ability to be approved for a loan. In their words, “Do I even qualify for a car loan with CarMax?”
The team set out to help their customers feel comfortable during their car-buying experience and ensure that they made the best purchase for their needs and budget while at the same time helping the business achieve its goals. The team began by creating empathy maps during each contextual interview with a customer. This helped team members realize insights and recognize consumer behavior patterns.
Figure 9-1. Empathy maps that the entire team created during customer interviews
Testing a Hypothesis
One of the first things the team wanted to test was whether the shortest loan application form would increase completion rates. They mocked up an initial draft in Axure, which you can see in Figure 9-2.
Figure 9-2. The first form the team created—the shortest possible form—in Axure
Using Ethnio to get users to work through their prototype, the team was surprised to learn that a short form, albeit simple and quick to complete, didn’t give users a sense of confidence that they were actually approved for a loan. This led many of the participants to believe that when they got to the store they’d have to go through loan approval again. These customers actually wanted to provide more information so that it “felt more like a loan application” than the prototype led them to believe.
The Next Iteration
The team set out to design a longer form with the internal challenge to not make it “feel” like a long form. Having a good sense of “Tiffany” and her needs, the team decided that a budget calculator was a good place to begin. This would allow Tiffany to determine whether she could afford the monthly car payment. This was followed by asking for the remaining information necessary to complete a credit application. These two elements together would reveal whether customers qualified for a loan, making the rest of the form process relevant or, in some cases, not worth the extra effort. Again, the team went to Ethnio and Axure to create and test the form. The end result (see Figure 9-3) was a “chunked-up” long form—a higher number of fields broken down into categories—that didn’t feel as long. Figure 9-4 shows the actual finished form.
Figure 9-3. The “long and chunked-up” prototype form that the team created in Axure
Figure 9-4. The first “live” form that shipped to real customers
Testing Another Hypothesis
Sometimes, it makes sense to use your research process to test more than one question at a time. That’s what the CarMax team did here. As the team refined their form design, they began testing a second hypothesis: would customers even want to be prequalified for a car loan before arriving at the store?
To answer this question, the team used a method called the “404 test” or “button to nowhere.” In this case, the team placed call-to-action buttons to apply for a car loan in various places on their mobile website. Then, they watched traffic logs to see what the rate of response was for these calls to action.
Clicking the first implementation of the call to action displayed a “feature coming soon” message. It wasn’t a great experience for the user, but it gave the team a good sense of where to put these elements and how best to word them. They refined the test to drive away from this “404-style page” toward a lead-generation form. This worked to both the users’ benefit—making them feel like they were making progress in their application—and the team’s—informing them on the best entry points for the loan application process and providing real customer data to executives to validate their design decisions.
Integrating In-Store Sales Staff
Including sales consultants from the beginning of the project allowed the product team to ensure that the design of the information being delivered to the store sales staff met their needs and allowed them to have more meaningful conversations with customers either on the phone or when they arrived at the store. For sales consultants in the stores these new loan applications provided a way to understand customers’ needs more effectively.
The team observed sales consultants at two different stores to understand how they used this information. The team came to understand what information was useful at various points of the customer interaction, and this helped them design the way the information was presented to sales people. Initially, the team’s new loan application presented information on a “decisions page.” This assumed that the customer had already selected a car. In reality, neither the sales consultant nor the customer were at that phase yet. The feedback from the store staff allowed the team to iterate their information design to be more in sync with the actual sales process.
By iterating and testing different ways to present the information for their sales colleagues, the team found the right combination of information and design to shift these conversations from “What kind of car would you like?”—a question many buyers don’t really have an answer for—to, “Let’s discuss the kind of cars you qualify for”—which is a far more productive conversation for both parties.
Regular Cadence with the Team
This project was a textbook example of a balanced team sharing the Lean UX work. Research sessions were scheduled consistently on the same day (Thursday) every week and invitations extended to the entire product team. Between each session, the team would stay engaged by conducting sketching sessions to refine the prototype and the testing script.
In addition, the team proactively communicated to colleagues through a variety of channels. They posted all of their findings on wall space in the office where many of their colleagues could see it. The team took advantage of CarMax’s culture of storytelling by creating the concept of “open houses” conducted by various product teams, as demonstrated in Figure 9-5. These public space events showcased a team’s work and allowed executives as well as other teams to get a sense of how projects were progressing, what was working, and what the team was shifting based on its learning.
Figure 9-5. Miller (closest to the board on the left) and Sutherland (right) sharing their findings at an open-house event
Setting Client Expectations at a Digital Product Studio: Lean UX at ustwo
There are many challenges when it comes to selling product development consulting. By definition, you’re trying to create something from nothing, so ambiguity and uncertainty are high. In this context, getting aligned on project scope and working process is critical. Add a new, unfamiliar process like Lean UX and things become even more complicated. In this case study, we look at how ustwo, a consulting firm with studios in London, New York, Malmö, and Sydney, has addressed this challenge.
There are lots of challenges at the beginning of a consulting engagement. You need to build a shared understanding of the goals of the engagement with your client. You need to set the appropriate expectations about what the agency is committed to achieving. You must ensure that your client understands how you plan to work and how you expect to collaborate. Finally, if you’re working in a Lean UX approach, you need to help your client understand and commit to a more outcome-driven result (rather than the more typical approach in which you agree to build a specific predefined set of features). Design studio ustwo has created and refined an approach to tackling these challenges head on.
The Service Definition Workshop
ustwo has created a short preengagement phase they call a Service Definition Workshop. They sell this to every client before they commit to taking on the entire project. This workshop involves the team that will work on the full project and key stakeholders from the client. They spend one to two days together to lay out the assumptions of the project, the risks that are involved, what skills they’ll need, and what kind of support will be required from the client.
Using a series of facilitated brainstorming exercises, convergent and divergent thinking explorations, and affinity mapping techniques, ustwo introduces the concepts of Lean UX to the client, sets expectations about how much client participation will be necessary in the full project, and begins the trust-building process.
The Service Definition Workshop sets the scope of the work and expectations for how the team will address the work. It allows the newly formed client/agency team to create the following:
A shared understanding of the vision and goal of the project
A prioritized list of business goals
A clear sense of who the first users of the system will be
A proposed customer journey for those users
An initial set of desired features that the team feels are important
The team captures their learning on a single-page canvas, as shown in Figure 9-6.
Figure 9-6. The Service Definition Workshop canvas from ustwo
Following the Workshop, MVPs and Collaboration
After the initial two-day collaboration, ustwo spent an additional two weeks prototyping their early ideas and testing them with potential users. During this time the client team participates in daily stand-ups as much as they can. When the prototyping phase is over, both the client and ustwo have a clearer sense of what the project will entail, what the scope of that effort might be, and, perhaps most important, what it’s going to be like to work together.
Figure 9-7. ustwo team members and their client participating in a Service Definition Workshop
ustwo have found this fixed-price engagement (in many ways a variation of the Design Sprint technique) to be a far more effective way to share how they work with new clients than the traditional Keynote pitch deck. Because the investment in this workshop is small for all participants it’s a relatively easy service to sell, and the benefits—tighter scope, shared understanding, and team compatibility—far outweigh the costs. Amazingly, ustwo told us that more than 50 percent of these workshops end up in long-term engagements, a much better close rate than they see with their traditional pitches.
Lean UX in an Agency: Changing the Way We Sell Work
Lean UX grew up in the context of digital product development and the process changes required to thrive in the digital age. Companies that grew successful in predigital times built processes that made sense then but might have outlived their usefulness. Changing processes to become more digital can be difficult. This is especially true for advertising agencies that grew up around the production processes of the print and broadcast world. In this case study, we’ll look at how one successful marketing agency, Hello Group, is adapting to a digital world.
Hello Group, a growing digital agency with its roots in advertising, has been transitioning into broader design and strategy work for the past few years. As part of that, the agency had to rethink the way they partner with clients and third-party engineering vendors to deliver work. Lean UX has played a pivotal role in shifting the way they work. The company has drawn inspiration from Lean UX to create two important new tools to help them shape that conversation with their clients, vendors, and within the agency, with their own management team, as well.
Alignment, Coordination, and Flexibility
One big problem Hello Group faced was how to articulate the scope of their project work to the client and their engineering vendors in a way that created alignment and understanding but also provided enough flexibility to allow them to explore different features and alternative design implementations. Using the Lean UX hypothesis as a model, the agency created a tool called the Experience Story.
Experience Stories are mini-scenario statements that ensure designers stay focused on solving the problem at hand without becoming lost in the minutiae of feature details. It helps the design team stay focused on the vision, something that can so often become lost in the day-to-day micromanagement of tasks and progress. Experience Stories are based on customer research and observation insights. They describe the ideal experience a customer should go through when engaging with a service. They’re made up of three parts:
The current situation with which the customer is faced
The friction involved in that situation that the team is trying to address
The ideal experience the team wants to create
Here’s an example:
Two days on the cruise ship is a time filled with experiences. Guests come on board with the expectation of making the best of this time.
But we demand a lot from our guests; remember to print your boarding pass; remember which card was for breakfast and which was for dinner; remember whether you’ve paid for the dinner, etc.
A good experience would be when all of these details disappear and guests flow through the cruise ship from check in to check out.
These Experience Stories are shared both internally and with Hello’s clients and partners. This helps everyone orient around why the project is important and it serves as a consistent yardstick for all of the parties on a project.
Working with Third-Party Engineers
A second and perhaps even more daunting challenge Hello Group faced was their reliance on third-party engineering vendors to build the designs Hello Group created. If Hello wanted to work in a Lean UX way, how could they ensure that their partners—who are hired and incentivized differently—would conform to this way of working? They decided to use something they call the Project Working Agreement.
Inspired by David Bland’s Team Working Agreement, the Project Working Agreement lays out, in very clear terms, exactly how the different agencies will work together.1 The agreement covers things like the following:
What flavor of Agile the agencies will use
How long sprints are
Where the code will be kept
When the teams will meet
What tools they’ll use to meet and communicate
And much more.
It might seem like a lengthy, tedious exercise with which to kick off a project, but it’s proven to save Hello hours of negotiation later in the project. Figure 9-8 shows what it looks like.
Figure 9-8. The template that Hello Group uses to capture the Project Working Agreement
The Project Working Agreement is a tool for improving collaboration, something that is of paramount importance in Lean UX. And, like most things at the start of a project, the agreement is based on a series of assumptions. As a result, Hello Group treats the agreement as a living document. As the project progresses, the efficacy of the tactics listed in the agreement will vary. Teams can decide to update the agreement as needed in order to make the working process more productive.
A Last Word
Sometimes, it can feel impossible to change the entrenched habits of an organization. So, we were delighted to receive this email from our colleague Emily Holmes. As we read it, we knew we had to share it with you.
In the email, Emily, who is Director of UX at Hobsons, an educational-technology company in the Washington, DC, area, details the changes she’s made in her organization. Here are some excerpts that describe the journey her firm has taken:
I think a lot of enterprise companies struggle to figure out the best way to implement these techniques. We initially got a great deal of resistance that we couldn’t do Lean UX because we’re “not a startup,” but of course that’s really not true.
We brought in a coach to help reinforce with the team our goal of moving our development process toward a Lean UX methodology (it can help to have an outside voice to reinforce what’s being said internally), and since then we’ve made good progress. In less than a year, our team structure has moved from this:
Figure 9-9. Hobson’s original team structure
To this:
Figure 9-10. Hobson’s new lean team structure
I have introduced the following process/system for helping our teams internalize what needs to happen as we move through the discovery phase of a project, so we don’t skip any steps and so everyone can begin understanding why this thought process needs to happen.
It requires ongoing coaching on my part and we haven’t completely mastered it yet, but it is really helping to get the full team in sync and speaking the same language. That’s no small feat, since our team includes people who are accustomed to business analysis, technical specs and waterfall development. It’s a little bit fun, so people don’t feel too resentful about having to change old habits. And, it definitely helps us fight the “monsters” that have traditionally been problematic for our organization.
I believe a lot of the things that are working for us could be applied to other enterprise organizations quite successfully.
We believe that, too, and hope the stories we’ve presented in this chapter will help to inspire you on your Lean UX journey.
