6

Scrum

Scrum carefully avoids referring to itself as a process. Why?

In a court of law, you would likely lose the argument that Scrum isn’t a process. It has ordered events, roles, and artifacts. Sounds like a process.

The key element here is ownership. Who should truly “own” the process? Management? Your organization’s Project Management Office (PMO)? Some book or guide? Schwaber and Sutherland?

Figure 6-1 The Scrum Framework

Of course not. If you have been following along so far, the answer here should be obvious. The Scrum Team needs to own the process. Each team, each product, each company is different, and each process must emerge to support their unique needs. However, it would be helpful if teams could start with something to build upon, a minimal set of rules: a framework.

A metaphor that works here is renting versus owning a house. If you rent a house and something goes wrong, what do you do? You pick up the phone and complain to the landlord. Then you complain about how things are always breaking and you deserve better support from the landlord. You feel victimized.

However, if you own a house and something goes wrong, what do you do? Nowhere to turn, right? You need to suck it up and deal with the problem yourself. You make decisions based on return (value) and investment (cost, time). Sometimes you fix it on your own, sometimes you call an expert, and in some cases, you just live with it. In the end, it’s all about finding a workable solution for which you are accountable.

The same happens with processes. If the process is defined and installed from outside of the team and “forced” upon them, when something goes wrong they call their management/PMO. They assign blame to the process and complain about it over lunch with their colleagues and online with angry posts. It becomes all about finger pointing.

Now when a team sees Scrum as a starting point, a true framework with built-in opportunities to change and improve, they take ownership. They have nowhere to point to but themselves if (when) things do not work out. This is the essential ingredient for accountability and self-organization.

As the previous chapter explains, complexity requires emergent practices. Team ownership over a flexible (empirical) process is key to this.

If organizations are not careful, Scrum can also be viewed as a rented process.

You can see this reflected within almost any anti-Scrum rhetoric online.

Do a little digging and you will see the complaints are not about framework elements but instead about practices that were instituted on top of Scrum (often sizing mechanics, requirements capture, Sprint commitment, task management, etc.) by people outside the team (managers, PMOs, agile “coaches,” tools, etc.).

With help from the Scrum Guide itself, let’s look at the core elements of the Scrum framework.

Figure 6-2 Empirical pillars of Scrum: transparency, inspection, and adaption

Figure 6-2 symbolizes the three pillars of Scrum. Without all three in place, the whole construct collapses.

Imagine an organization that has all the data (inspection) and the ability to change (adaptation), but no transparency. The data may not reflect the true reality and may not be visible to the right people. Therefore, what they inspect will be incorrect and how they adapt will be wasteful. Having a culture that promotes openness and communication is essential for any empirical process to work effectively.

A lack of transparency is like putting a wet washcloth over a thermostat. The thermostat will inspect inaccurate data and adapt in the wrong way.

Imagine an organization that has nothing to hide and has solid communication (transparency) as well as the ability to change (adaptation). However, they have not taken the time to gather and analyze the data in any consistent way. You will end up with people feeling like they should adapt a certain way; but with no hard evidence to back it up, real change becomes more difficult. Having consistent and short feedback loops as well as clear information radiators is essential for any empirical process to work effectively.

Imagine an organization that has all the accurate data (transparency and inspection) necessary to change in a meaningful way. However, nobody is empowered or willing to do anything about it (adaptation). The ability to execute is essential for any empirical process to work effectively.

A single person who truly owns the product and has the last say is crucial for product success.

What good is it to have the best Development Team if the wrong product is being built? The Product Owner is the keystone on which the right product—the overall product quality—depends.

Figure 6-3 shows how Scrum uses a built-in flow of service. Ultimately, the Development Team is there to serve the Product Owner, and the Scrum Master serves the Development Team. You could also say that the Product Owner serves the customer and other stakeholders.

Figure 6-3 The service model of Scrum

Building on top of what is covered in “The Product Owner” section in the first chapter, let’s look at how the Product Owner works on a day-to-day basis.

Figure 6-4 Product Owner shown with crown

As the Product Owner, you have the absolute right to make all product decisions. This is in contrast to a traditional project manager who is responsible for making sure that the project stays within scope, schedule, and budget. Whether the product is the right product and whether the product creates value for the users and customers are likely not a project manager’s top concerns. As a Product Owner, this is your purpose. You are the value maximizer for the product.

It should be obvious that being appointed as Product Owner for a product in a domain you do not actually understand is a questionable decision. Most likely this Product Owner will become a proxy (as described in Chapter 1), which will result in delays and information loss every single day. A good Product Owner is a domain expert who knows the business from many angles. This allows the Product Owner to quickly navigate and adjust as new information emerges in synchronization with the stakeholders. A Product Owner who is a proxy only collects stakeholder requests and orders them in a way that pleases their loudest stakeholders (squeaky wheels). A Product Owner who steps up and owns the product plays an important role toward creating an agile business.

Realistically, demanding a Product Owner to know everything about the domain is an almost impossible request. While still maintaining the vision and ownership, an effective Product Owner should be able to leverage other Subject Matter Experts (SMEs) to help guide the product. The Product Owner could even create direct communication lines between Development Teams and the stakeholders. This is even more true in a scaled environment where the Product Owner’s capacity is strained.

Figure 6-5 Overlapping and distinct responsibilities of Product Owner and Development Team

What you really need to avoid is a requirements hand-off from the Product Owner to the Development Team. The more closely the Development Team is engaged with the features and their refinement, the more the Product Backlog items become theirs. This leads to more ownership, stronger engagement, and pride, which are key steps toward achieving excellence.

On a day-to-day basis, this means that you, as Product Owner, spend less time specifying requirements and more time educating the Development Team about the value of the requirements. Give them the support they need and then trust them to come up with a solution.

When refining the Product Backlog for upcoming Sprints, do not just hand over and explain. Use the refinement to provide understanding, answer initial questions, and then let the Development Team do their work. During the Sprint, support the Development Team wherever needed and provide feedback on Product Backlog items. The benefit here is not only that the Development Team gains better understanding and ownership of the Product Backlog, but this approach also frees you up to do the more strategic Product Owner work, like analyzing the market, observing the competition, talking to sales and marketing, aligning with other Product Owners, talking to stakeholders, and so on.

You will want to keep an eye on overall progress of the Product Backlog as you still have overall accountability. However, the plan for the Sprint belongs to the Development Team, so dictating what they work on and when is not your call.

Remember, the Development Team’s accountability is to produce a “Done” working Increment of the product every single Sprint. Accomplishing this requires the Development Team to have all the skills necessary to reach “no work remaining,” as in “yes, we can ship.”

Imagine you and your cooking friends are asked to cater a big event. All of you have spent quite some time in the kitchen cooking together. You know each other, you have established a routine for communication and moving around so as not to impede others. This, in combination with a stocked pantry, sets you up to amaze some palates.

But let’s assume not everything is perfect. Maybe two of your friends cannot make it, or the pantry is missing some key ingredients, or your team really does not have the skills needed to pull off the requested menu. You could certainly improvise, but that choice carries risks. The likelihood of having a successful outcome is greatly reduced.

Taking this analogy into the world of a software product development, it becomes obvious that what is in the Development Team can be in the product, but what is not most likely cannot. You can improvise, depend on external people, or come up with other creative workarounds. But in doing so you create significant risk for the feature quality, the technical quality, and progress.

As Product Owner, you never start with a perfect Development Team. It emerges over time. So give them the time, focus, and resources they need to become a professional team. Making use of Sprint Retrospectives and measuring key indicators like on-product index, innovation rate, and employee satisfaction (as described in Chapter 3) can help.

According to Sandy Mamoli and David Mole, 60 percent of success depends on the people doing the work.⁹ Companies that realize this do not fund projects. They fund teams that build and maintain a product, as shown in Figure 6-6.

Figure 6-6 Emergence of funding from project releases over products to product teams

Should the Scrum Master be knowledgeable about the business domain? There clearly is no harm; however, it is not a necessity. This is the beauty of the separation of roles within Scrum. The product quality, building the right product with value for the users, is with the Product Owner. The technical quality of the product lies with the Development Team. The Scrum Master facilitates all these responsibilities. In fact, a lack of knowledge of the domain could even be considered a benefit, freeing the Scrum Master to focus on facilitation, coaching, team dynamics, and to ensure that a process, based on the Scrum Framework, is emerging.

Too often, the terms “Development Team” and “Scrum Team” are used interchangeably. They are not the same: The Development Team is part of the Scrum Team. Often this subtle difference can have a big impact when generically talking about the “Team” and understanding where the responsibilities belong.

As Product Owner, it is your responsibility to identify the stakeholders who have a say in the direction of the Product and to ensure they are included.

Scrum provides many built-in opportunities to engage with your stakeholders. The obvious one is the Sprint Review, where stakeholders can provide feedback on the Increment that subsequently affects the Product Backlog.

However, including stakeholders in Sprint Planning and Product Backlog refinement can also be beneficial and should be done at the discretion of the Product Owner.

A good brainstorming activity to identify your stakeholders is to work through a mind map such as the one in Figure 6-7.

Figure 6-7 Mind map showing various stakeholders

For each category shown in Figure 6-7 (consider adding new categories that are specific to your product), name organizational roles and even the people in those roles. Doing so will likely result in an extensive list of stakeholders, each with unique needs and expectations about the direction of the product and how they would like to be included.

Another helpful technique is to place your stakeholders on an “influence 232” such as the one in Figure 6-8.¹²

Figure 6-8 Influence 2×2

This tool gives you a better idea of how to involve your stakeholders. Consider involving your interested parties more often, even during the Sprint where they can help with details. You may be able to accommodate the less interested parties in other ways that are less time consuming but still keeps them happy. Active stakeholder management builds trust, especially once they see the impact they can have on the product. It also shifts accountability to stakeholders: The more they are involved during development, the less they can complain once the product is released.

For stakeholders who have high interest yet low power (bottom right of Figure 6-8), consider putting them to work. As Product Owner, there is a lot to do. Maybe these stakeholders are willing to help with the more tactical items while you focus more on strategy.

The three artifacts are linked: The Increment is generated from the Sprint Backlog, and the Sprint Backlog is generated from the Product Backlog. The Product Backlog is then refined based on feedback from the Increment (see Figure 6-9).

Figure 6-9 Interconnected Scrum artifacts

In the next chapter, we will get into more details about how to create and maintain a Product Backlog.

A well-refined, ordered, and estimated Product Backlog provides you with direction and with all the information necessary for reporting and planning. However, for this to work, you need to make sure that the Product Backlog is your only—as in one—source of work for the Development Team. Assume that you have a Product Backlog for the features, a bug backlog for the issues, a technical backlog for infrastructure and architectural work, and you have unplanned work injected during the Sprint. In that suboptimal (yet common) situation, there is no true transparency regarding your direction. It is hard or even impossible to measure your bearings.

At the risk of repeating ourselves, the following rule always applies:

And this Product Backlog contains all the work for the Development Team.

This is an essential part in building a mutually respectful relationship between these two roles. A Development Team that truly owns the plan for the Sprint will demonstrate much more accountability and ownership over the work of the Sprint.

Together with the Development Team, you define the Sprint Goal in Sprint Planning and then trust them to create the Sprint Backlog and maintain it throughout the Sprint.

If the Product Backlog represents the direction of your current path, then the Increment is your GPS device that determines your current position. The Increment provides transparency for you and your stakeholders. In that sense, it is the most important element in the Sprint Review and fuels the most important feedback loop in an empirical process control. A working Increment allows you to inspect and adapt together and to determine the next steps, thereby updating your path—the Product Backlog.

For this to work, your Increment needs to be truly “Done.” How could you find your position with an inaccurate GPS reading? You can’t; and this lack of transparency is detrimental or even catastrophic. In the long run, you will lose your way.

At least once a Sprint, your Increment (your product) must be “Done”: “Done” as in no work remaining, as in it can be released. Whatever “Done” means in your context is something you as the Product Owner (representing the stakeholders) and the Development Team must work out. All needs regarding “Done” have to be addressed accordingly. “Done” reaches from the end user all the way to operations.

If you ignore “Done,” at best, you are just making progress. The question is toward what?

Read more about “Done” in the next chapter on Product Backlog management.

Figure 6-10 Example Sprint burn-down

Figure 6-10 shows a Sprint burn-down chart. The dashed line (with x’s) represents the hours remaining from the Sprint Backlog. The solid line represents the story points remaining from the forecast.

Although the Sprint burn-down chart is not meant for you, the Product Owner, you should be able to see it to get a gauge on how the Sprint is going. But it is not your responsibility to act on it.

A Product Owner and their stakeholders are likely more interested in progress across Sprints. For that, you have release burn-downs or burn-ups.

The important words here are “work remaining.” How much the team has worked is irrelevant; you should care only about how much work is remaining so that you can plan better. Your Development Team can work for 10 hours and discover 20 extra hours of work, or complete three Product Backlog items and discover five more, increasing the amount of work remaining. Regardless of what they did in the past, it does not change the fact that this new work still needs to get done. So you adapt the plan going forward.

Burn-down charts are a great way to visualize work remaining. But feel free to use other charts as long as they provide visibility to the work remaining.

Building release burn-downs/burn-ups (see Figures 6-11 and 6-12) as a way to represent release plans will be covered in more detail in Chapter 8.

Figure 6-11 Release burn-down

Figure 6-12 Release burn-up

In Scrum, each time-boxed event is designed to provide a specific inspect and adapt opportunity. In other words, there is always tangible input and always an explicit output for each event.

Keep in mind, although you have the flexibility to change direction from Sprint to Sprint, you must avoid changing the scope of a current Sprint as much as possible. If the need for changing the Sprint scope is a recurring problem, consider shorter Sprints.

A Sprint provides stability in a world of unpredictability. The Scrum Team makes hard choices about requirements, infrastructure, and people. It tries to stick with those choices for at least one Sprint and then inspects and adapts for the next Sprint. Sprints that are too long could set you too far off course, increasing the cost of getting back on track.

The length of the Sprint should be determined by how much of this stable work a Scrum Team can (should) take on. This is the Product Owner’s planning horizon.

Realistically, there will be exceptions to keeping a Sprint stable. Consider the following scenario:

As Product Owner, an exciting new feature comes across your desk. Arguably, it is more valuable than any of the Product Backlog items that the Development Team is working on in the current Sprint. What do you do? You have three options.

1. The best response is to place the new feature on the top of your Product Backlog and not disrupt the current Sprint. If what is already in the Sprint still has value, then let the Development Team finish. Too often, they are interrupted with new work and do not get to finish what they have started. There are few things more inefficient than constant starting and stopping. Help your team to “stop starting and start finishing.”

2. When the new feature request nullifies some of the Product Backlog items being worked on in the current Sprint, then the situation is different. There is no point in working on items that have no value. The obvious thing to do here is to go ahead and adjust the scope of the Sprint, without endangering the Sprint Goal. This will likely mean swapping out Product Backlog items.

3. The third option is that if you determine that the new feature replaces the need for everything in the current Sprint and voids the Sprint Goal, then this is where the Product Owner can consider canceling the entire Sprint. Since there is no point to continuing, stop the Sprint and immediately start planning a new Sprint. Given all the previous options, this should certainly not be a common activity.

Figure 6-13 outlines the flow of Sprint Planning.

Figure 6-13 Flow of Sprint Planning

Let’s look at two possible scenarios of Sprint Planning.

Which scenario has a higher likelihood of being successful?

The first scenario is a handoff, with all its associated risks.

The good news is that you are only handing over a smaller portion of the product. But it is still a handoff. And like any handoff, it can cause ambiguity as the written word is never the best way to communicate complexity.

Agile blends the business with engineering, which avoids the classic handoff problem in a phased linear development approach. As the first scenario above showed, even Scrum is not immune to this handoff problem. It is likely that some items turn out slightly different than anticipated, and often the answer from the Development Team is, “OK, but we did what was specified here, it is not our fault!” Back to finger pointing, blaming, and requesting more detailed requirements (CYA).²²

The second scenario engages the Development Team from the beginning. They understand the purpose of why they are doing this. They take ownership of the functionality; it becomes theirs. If something is yours, you watch out for it, you take care of it, you want to make it a success.

As described previously, you as the Product Owner are motivating the Development Team by leveraging what is described in Dan Pink’s book Drive:

Your Sprint Goal should be considered as the elevator pitch for the Sprint. Imagine a stakeholder passes you in the hallway one day and says, “Looking forward to the Sprint Review next week! What will I get to see?” How will you respond? Would you rattle off the nine Product Backlog items being worked on? Or would you give her a more concise summary of the most important element? If you find it hard to give an overall summary, it is likely a sign that there isn’t much cohesion in your Sprint. In other words, the harder it is to come up with a good concise Sprint Goal, the more you probably need one.

This does not mean that you cannot have Product Backlog items in a Sprint that are unrelated to the Sprint Goal. Certainly, they could also work on an extra feature, report, or bug that a stakeholder has asked for. However, they should be in the minority and would likely be the items dropped from the Sprint when the Sprint Goal is at risk of not being met.

Think about the Sprint Goals as a linked list of objectives that are guiding you toward your overall product vision one step at time. Having homed in on your vision, you identify upcoming goals toward the grand goal (see Figure 6-14). The Sprint goals are not afterthoughts, but honing beacons. Figure 6-15 shows some examples of Sprint Goals.

Your Sprint Goal should never be a comma list of user stories (Product Backlog items) to be completed by the end of the Sprint.

Figure 6-14 Sprint Goals lead to the Vision

Figure 6-15 Example Sprint Goals

Having regular communication with the Development Team is still a great idea. Just do not hijack the Daily Scrum for this. Set something else up that works best for your particular situation.

The stance you should take in the Sprint Review is more like a representative of the Scrum Team and host. You should kick it off by revisiting the product vision with everyone and sharing the Sprint Goal. Then let the audience know how the Sprint went with regards to the Sprint Goal. A common and useful next step is to invite members from the Development Team to demonstrate the “Done” work. This is a great opportunity to give stakeholders facetime with the Development Team, which builds trust, motivation, and ownership. While the Increment is being inspected, encourage feedback from stakeholders and make sure to capture it—hopefully directly in the Product Backlog. A good way to conclude the Sprint Review is to review the release plan with everyone. Given that a Sprint was just completed, this will provide new data from which to inspect and adapt the future path of the Product Backlog. Figure 6-16 shows an example agenda for a Sprint Review.

Figure 6-16 Sprint Review agenda²⁵

Product Owners often find themselves in a position of authority. This could shut a Development Team down during a Retrospective. In other words, they may not be as outspoken with you around, which could reduce transparency. As Product Owner, make it your mission to bridge that gap. Participate in team-building activities, invite team members to stop by your office, ask them to text or call you about anything. Be a team player.

Figure 6-17 is taken from “The New New Product Development Game,” a popular Harvard Business Review article from 1986 by Hirotaka Takeuchi and Ikujiro Nonaka.²⁷

Figure 6-17 Sequential versus overlapping phases of development

In Figure 6-17, Type B describes how the phases overlap. If you replace classical project management phases with Scrum Sprints, the figure describes refinement. Type B shows refinement for one Sprint ahead.

The overlap from Sprint 1 to Sprint 2 and so on is the time in the current Sprint you invest to make the subsequent Sprint “Ready.”

What about Type C? Assume you have some future Product Backlog items that need more forethought before they are ready. Or maybe you need deeper User Experience design that could take some time before it is ready to be consumed by a Sprint. In these cases, you must look further ahead than just one Sprint. Type C shows refinement for multiple Sprints ahead.

Done right, refinement provides you with a nice rolling Product Backlog (see Figure 6-18).

Figure 6-18 Conceptual refinement in a Sprint of two weeks in length

Figure 6-19 Emergent refinement of the Product Backlog

Velocity is the Development Team’s capability to turn “Ready” Product Backlog into a “Done” product Increment and simultaneously refine the upcoming Product Backlog to be “Ready” for the subsequent Sprint (see Figure 6-19).

Figure 6-20 Incremental product development

Often an Increment is understood as just the addition to the existing product.

Merriam Webster’s defines “increment” as “the action or process of increasing especially in quantity or value : enlargement.” The word’s origin is “from Latin, incrementum, from the stem of increscere ‘grow.’”²⁹

The Increment is the whole growing product, not just that little bit that was added since the last Sprint. The Scrum Guide also tells us that the Increment needs to be “Done” with no work remaining: that is, potentially releasable. That also means that the former Increment still needs to be “Done” even though it might have been changed in the current Sprint. Maybe the Development Team had to extend an API with an additional field, to swap a library with a thread-safe one, to change some aspects of the architecture to accommodate the new features. Those changes are not a bad thing or a sign of being a bad engineer, but rather acknowledge the emergent complex nature of product development.

In the agile community, it is understood that quality is not something you can test into the product after you have completed implementation; it must be built into the product from day one. This raises a question: How can you ensure that existing quality is not disappearing? This is one of the fundamental challenges in iterative and incremental development.

It boils down to good engineering practices and a sound setup of continuous integration/delivery with strong test automation.

This gives you a product that is capable of self-testing, providing you with an “I am OK and fully functional” feedback after every single build. That way, the Increment can grow (see Figure 6-21) and increase value without degrading over time.

Figure 6-21 Incremental versus iterative and incremental

Do not cut corners and sacrifice quality for speed. This only increases your technical debt, for which you have to pay dearly in the future. Your initial gain and more will be lost in the future when you struggle with existing implementation. Steve McConnell gets it right: “The problem with quick and dirty . . . is that dirty remains long after quick has been forgotten.”³⁰

AGILE MANIFESTO FOR SOFTWARE DEVELOPMENT

We are uncovering better ways of developing software

by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right [in italics],

we value the items on the left [in bold] more.³¹

The Agile Manifesto for Software Development was created in early 2001 and has been revisited multiple times since, without any resulting changes as it was still considered relevant. It was created by 17 leaders from the software industry, including the creators of Scrum, Ken Schwaber and Jeff Sutherland. It represents the underlying values and principles of good software development. Until the Agile Manifesto, agile software development was not called “agile.” Until then it was usually referred to as light-weight.

At Agile 2008 in Toronto, Uncle Bob (Robert Martin) suggested in his keynote the addition of a fifth value:

Craftsmanship over crap

However, he realized that it doesn’t fit the pattern of preferring the left-hand side over the right-hand side:

In the end, it is all about creating the right product with the right quality.

____________________________

1. Ken Schwaber and Jeff Sutherland, The Scrum Guide (November 2017), 3.

2. Schwaber and Sutherland, Scrum Guide.

3. Ibid., 4.

4. Ibid., 5.

5. Ibid.

6. Ibid.

7. Ibid., 6.

8. Ibid., 7.

9. Sandy Mamoli and David Mole, Creating Great Teams (Raleigh, NC: Pragmatic Bookshelf, 2015), 4–5.

10. Schwaber and Sutherland, Scrum Guide, 7–8.

11. Ibid., 6.

12. Rachel Thompson, “Winning Supportyou’re your Projects,” MindTools, accessed February 28, 2018, https://www.mindtools.com/pages/article/newPPM_07.htm.

13. Schwaber and Sutherland, Scrum Guide, 15.

14. Ibid., 16.

15. Ibid., 17.

16. Ibid., 18.

17. Ibid., 16.

18. Ibid., 9.

19. Ibid., 10.

20. Ibid.

21. Ibid., 10–11.

22. Cover Your @$$.

23. Schwaber and Sutherland, Scrum Guide, 12.

24. Ibid., 13.

25. Thanks to Ty Crockett.

26. Schwaber and Sutherland, Scrum Guide, 14.

27. Hirotaka Takeuchi and Ikujiro Nonaka, “The New New Product Development Game,” 1986, Harvard Business Review, January 1986, https://hbr.org/1986/01/the-new-new-product-development-game.

28. Schwaber and Sutherland, Scrum Guide, 15.

29. Meriam Webster’s, s.v. “increment,” accessed February 28, 2018, https://www.merriam-webster.com/dictionary/increment.

30. Steve C. McConnell, Software Project Survival Guide (Redmond, WA: Microsoft Press, 1998).

31. Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas, “Manifesto for Agile Software Development” (2001), http://agilemanifesto.org/.

32. Mike Bria, “Craftsmanship—The Fifth Agile Manifesto Value?,” InfoQ, August 20, 2008, https://www.infoq.com/news/2008/08/manifesto-fifth-craftsmanship.