When I first learned about XP, I thought, “What could be easier? Just follow a few simple disciplines and practices. Nothing to it.”
However, based upon the number of organizations that try—and fail—to become Agile, becoming Agile must be very, very difficult. Perhaps the reason for all that failure is that so many organizations think Agile is something that it’s not.
Kent Beck named the four values of Agile long ago. They are courage, communication, feedback, and simplicity.
The first value is courage—or, in other words, a reasonable degree of risk-taking. The members of an Agile team are not so focused upon political safety that they sacrifice quality and opportunity. They realize that the best way to manage a software project over the long term is with a certain degree of aggression.
There is a difference between courage and recklessness. It requires courage to deploy a minimum feature set. It also requires courage to maintain high code quality and maintain high-quality disciplines. However, it is reckless to deploy code that you are not highly confident in or that has an unsustainable design. It is reckless to conform to a schedule by sacrificing quality.
The belief that quality and discipline increase speed is a courageous belief because it will constantly be challenged by powerful but naive folks who are in a hurry.
We value direct and frequent communication that crosses channels. Agile team members want to talk with one another. The programmers, customers, testers, and managers want to sit near each other and interact with each other frequently, and not just in the context of meetings. Not just through emails, chat, and memos. Rather, they value face-to-face, informal, interpersonal conversation.
This is how teams gel. It is within the rapid, chaotic, informal firestorm of frequent interaction that light bulbs go off and insight is gained. A team that sits together and communicates frequently can work miracles.
The Agile disciplines we have studied are virtually all geared toward providing rapid feedback to the folks who make important decisions. Planning Game, Refactoring, Test Driven Development, Continuous Integration, Small Releases, Collective Ownership, Whole Team, etc. maximize the frequency and quantity of feedback. They allow us to determine when things are going wrong early enough to correct them. They provide massive education about the consequences of earlier decisions. Agile teams thrive on feedback. Feedback is what makes the team work so effectively and what drives a project to a beneficial outcome.
The next Agile value is simplicity—in other words,being direct. It has often been said that every problem in software can be solved by adding another layer of indirection. But the values of courage, communication, and feedback are there to ensure that the number of problems is reduced to a minimum. Therefore, indirection can be kept to a minimum. Solutions can be simple.
This applies to the software, but it also applies to the team. Passive aggression is indirection. When you recognize a problem but silently allow it to pass to someone else, you are engaging in indirection. When you accede to the demands of a manager or customer knowing that the consequences will be dire, you are being indirect.
Simplicity is directness—directness in the code, and directness in communication and behavior. In the code, a certain amount of indirection is necessary. Indirection is the mechanism by which we reduce the complexity of interdependence. On the team, far less indirection is necessary. Most of the time, you want to be as direct as possible.
Keep the code simple. Keep the team simpler.
It is easy to become confused by the vast numbers of Agile methods out there. My advice to you is to ignore the menagerie. In the end, regardless of which method you choose, you will wind up tuning and tweaking the process to fit your own needs. Thus, whether you begin with XP, Scrum, or any of the 5328 other Agile methods out there, you are going to wind up in the same place.
The strongest advice I can give you, however, is to adopt the full Circle of Life, including, and most especially, the technical practices. Far too many teams have adopted the outer business ring and have found themselves in the trap that Martin Fowler has labeled “Flaccid Scrum.”1 The symptoms of this disease are a slow decline from high productivity in the early days of a project to very low productivity as the project wears on. The reason for this loss of productivity is the corruption and degradation of the code itself.
It turns out that the business practices of Agile are a highly efficient way to make a very large mess. Furthermore, if you don’t take care of the cleanliness of the structure you are building, then the mess will slow you way down.
So, pick one of the methods or none of the methods. Ensure that you address all the disciplines in the Circle of Life. Get the team to agree. Then, begin. Remember courage, communication, feedback, and simplicity and regularly adjust the disciplines and behaviors. Don’t ask for permission. Don’t stress about “doing it right.” Just work through the issues as they arise and continue to drive your projects to the best possible outcome.
The transition from non-Agile to Agile is a transition in values. The values of Agile development include risk-taking, rapid-feedback, intense, high-bandwidth communication between people that ignores barriers and command structures. They also focus on moving in straight and direct lines rather than mapping out and negotiating the landscape. These values are diametrically opposed to the values of large organizations who have invested heavily in middle-management structures that value safety, consistency, command-and-control, and plan execution.
Is it possible to transform such an organization to Agile? Frankly, this is not something I have had a lot of success with, nor have I seen much success from others. I have seen plenty of effort and money expended, but I have not seen many organizations that truly make the transition. The value structures are just too different for the middle-management layer to accept.
What I have seen is the transition of teams and individuals, because teams and individuals are often directed by values that are aligned with Agile.
Ironically, executives are also often driven by the risk-taking, direct, communicative values of Agile. This is one of the reasons they try to transition their organizations.
The barrier is the management layer in the middle. These folks were hired to not take risks, to avoid directness, to follow and enforce the chain of command with a minimum of communication. This is the organizational dilemma. The tops and bottoms of the organization value the Agile mindset, but the middle layer opposes it. I don’t know that I have ever seen that middle layer make the change. Indeed, how could they? Their job is to oppose such a change.
To drive this point home, let me tell you a few stories.
In one Agile transformation in which I participated back in 2000, we had the support of the executives and the programmers. There was a lot of hopeful enthusiasm. The problem was the technical leads and architects. These folks incorrectly surmised that their roles were being diminished.
The roles of architects, technical leads, project managers, and many others are different in an Agile team, but they are not diminished. Unfortunately, these folks could not see this, and that may have been our fault. Perhaps we did not communicate how valuable their roles were to the team, or perhaps they were just uncomfortable learning the new skills they’d need.
In any event, they met in secret and devised plans to sabotage the Agile transition. I won’t go into the details of that plan. I’ll just say that they were caught at it, and they were immediately fired.
I’d like to report that the Agile Transition then proceeded apace and achieved great success. But I can’t.
We had great success transitioning one division of a much larger company. They adopted XP and did such great work over the years that they were written up in Computerworld. In fact, all that success led to the promotion of the VP of engineering who led that transition.
Then the new VP took over. And like a new male lion taking over the pride, he proceeded to kill all the offspring of the previous VP. And that included Agile. He stopped it cold and reverted the team to their old, far less successful process.
This led many folks on the team to seek employment elsewhere—which, I believe, was the new VP’s intent.
This last story is second hand. I wasn’t there to witness the critical moment. It was reported to me by my employees at the time.
In 2003, my company participated in an Agile Transition at a well-known stock brokerage firm. All was proceeding well. We trained the executives, the middle managers, and the developers. They were cooking along. Everything was going great.
Then came the final assessment. The executives, middle managers, and developers all met in a big auditorium. Their goal was to assess the progress and success of the Agile Transition. The executives asked the question, “How’s it going?”
The answer came from the various participants: “It’s going very well.”
Then there was a moment of silence in the room, which was suddenly punctuated by the sound of someone in the back of the room, weeping. And that’s when the emotional supports crumbled, and the positive mood in the room collapsed. “This is just too hard.” People were heard to say. “We just can’t keep this up.”
And so, the executives killed the transition.
I suppose the moral of all these stories is “Expect something weird to happen.”
Can Agile teams exist in an organization with a strong middle-management layer that opposes Agile? I have seen this happen upon occasion. Some software development teams quietly use Agile values to drive their development, while also conforming to the rigors imposed upon them by middle management. So long as middle management is satisfied that their procedures and standards are being followed, they may leave the development teams to their own devices.
This is what Booch and Parnas called “faking it.”2 The team does Agile under the covers, while providing everything that satisfies the middle-management layer. Rather than fighting a futile battle with the middle managers, these teams put a layer on top of Agile that makes Agile look safe and conformant to the middle managers.
2. Booch, G. 1994. Object-Oriented Analysis and Design with Applications, 2nd ed. Reading, MA: Addison-Wesley, p. 233–34.
For example, the middle managers may want an analysis document produced in the early phases of the project. The Agile team provides this document by writing much of the early code of the system using all the normal Agile disciplines, and then produces the analysis document by scheduling a sequence of documentation stories.
This makes sense because the first few coding iterations are heavily focused on requirements analysis. The fact that this analysis is being done by actually writing code is not something that the middle managers need to know. Nor should they care.
Unfortunately, I have seen organizations who are so dysfunctional that when middle managers smell that a team is “faking it,” they assume subterfuge and quickly move to expunge the Agile disciplines. This is a deep shame because those teams were actually giving the middle managers what they needed.
I have seen some medium-sized organizations adopt Agile. They had thin middle-management layers composed of folks who came up through the ranks and still retained the direct, risk-taking mindset.
It is not uncommon to see small organizations transition completely to Agile. They have no middle management, and the values of the executives and developers are strongly aligned.
Finally, sometimes only certain individuals within an organization adopt the Agile values. Individuals who make this transition do not fare well within organizations or teams who do not. The difference in values generally leads to a separation of some kind. In the best case, the individuals who make the transition will join to form new Agile teams that manage to hide from middle management. If that’s not possible, they will likely seek (and find) employment at a company that shares their values.
Indeed, over the last twenty years, we have seen this migration of values taking place in the industry. New companies that embrace the Agile values have been formed, and programmers who desire to work in an Agile manner flock to those companies.
Can large organizations be created that allow Agile teams to prosper within them? Certainly! However, note that the word is created not transformed.
When IBM decided to build the PC, the company’s executives realized that the values of the organization would not allow the kind of quick innovation and risk-taking that would be required. So, they created an organization with a different value structure.3
3. The birth of the IBM PC. Accessed at https://www.ibm.com/ibm/history/exhibits/pc25/pc25_birth.html.
Have we seen this happen in the software world? Have older, large organizations created smaller software organizations for the purpose of adopting Agile? I’ve seen hints of this, but I cannot bring to mind any overt examples. We have certainly seen many startups adopt Agile. We have also seen many Agile consulting companies being used by large non-Agile companies that want to get certain software projects done more quickly and reliably.
That’s my prediction. We will eventually see larger companies create new divisions that use Agile for their software development. We will also see Agile consulting companies used more and more by larger organizations that have been unable to transform their existing development staff.
Does an Agile team need a coach? The short answer is “No.” The longer answer is “Sometimes.”
First, we need to differentiate between Agile trainers and Agile coaches. An Agile trainer teaches a team how to conduct themselves in an Agile way. They are often hired from outside the company or are internal trainers from outside the team. Their goal is to inculcate the Agile values and teach the Agile disciplines. Their tenure should be short. Each team of a dozen or so developers should require no more than one or two weeks of training. Everything else they need to learn they’re going to learn by themselves no matter what the Agile trainer says or does.
Early in the transition of a team, a trainer may temporarily fill the role of the coach, but this is a temporary situation. That role should be filled from within the team as soon as possible.
In general, Agile coaches are not trainers. They are members of the team whose role is to defend the process within the team. In the heat of development, developers may be tempted to go off process. Perhaps they inadvertently stop pairing, stop refactoring, or ignore failures in the continuous build. It is the coach’s job to see this and point it out to the team. The coach acts as the team’s conscience, always reminding the team of the promises they made to themselves and the values they agreed to hold.
This role typically rotates from one team member to the next on an informal schedule and based on need. A mature team working steadily along does not require a coach. On the other hand, a team under some kind of stress—whether it be schedule, business, or interpersonal—may decide to ask someone to fill the role temporarily.
The coach is not a manager. The coach is not responsible for budget or schedule. The coach does not direct the team nor does she represent the team’s interests to management. The coach is not the liaison between the customers and the developers. The role of the coach is strictly internal to the team. Neither the managers nor the customers know who the coach is, nor even whether there currently is a coach.
In Scrum, the coach is called a Scrum Master. The invention of this term, and the sequence of events that followed it, were among the best and worst things that happened to the Agile community. The certification programs attracted large numbers of project managers. This influx increased the popularity of Agile in the early days but eventually led to the role of the coach to be conflated with the role of the project manager.
Nowadays, it is all too frequent that we see Scrum Masters who are not coaches at all but are simply project managers doing what project managers always did. Unfortunately, the title and certification tend to give them undue influence over the Agile team.
The Agile certifications that exist are a complete joke and an utter absurdity. Do not take the certifications seriously. The training that accompanies the certification programs is often worthwhile; however, that training should not be focused on a particular role, it should be for everyone on the team.
For example, the fact that someone is “certified” a Scrum Master is a worthless certification. The certifiers are not guaranteeing anything other than that the person paid the fee for, and possibly attended, a two-day course. The certifiers are specifically not guaranteeing that the newly minted Scrum Master will do a good job as a coach. The absurd implication of the certification is that there is something special about a “Certified Scrum Master,” which of course flies in the face of the notion of a coach. An Agile coach is not specially trained to be an Agile coach.
Again, there’s often nothing wrong with the training programs that go along with these certifications. However, it is foolish to train just one special person. Every member of an Agile team needs to understand the values and techniques of Agile. Therefore, if one member of the team is trained, all members of the team should be trained.
What would a real Agile certification program look like? It would be a semester-long course, complete with Agile training and the supervised development of a small Agile project. The course would be graded and the students held to a high bar. The certifier would guarantee that the students understood Agile values and showed proficiency in executing Agile disciplines.
The Agile movement began in the late ’80s. It was quickly recognized as a means for organizing a small team of between 4 and 12 software developers. Those numbers are soft and were seldom articulated, but everyone understood that Agile (or whatever we called it before 2001) was not appropriate for a massive team of thousands of developers. That was not the problem we were trying to solve.
Nevertheless, almost immediately, the question was raised. What about big teams? What about Agile in the large?
Over the years, many folks have tried to answer that question. Early on, the authors of Scrum proposed the so-called “Scrum-of-Scrums” technique. Later, we started to see certain branded approaches like SAFe4 and LeSS.5 Several books have since been written on the topic.
I’m sure there’s nothing wrong with these approaches. I’m sure the books are fine. But I haven’t tried the approaches, and I haven’t read the books. You might think that it is irresponsible of me to comment on a topic that I have not studied. You may be right. However, I have a certain perspective.
Agile is for small- to medium-sized teams, period. It works well for such teams. Agile was never intended for large teams.
Why weren’t we trying to solve the problem of large teams? Simply because the problem of large teams is a problem that many experts have worked on for well over five millennia. The problem of large teams is the problem of societies and civilizations. And, if our current civilization is any measure, it appears to be a problem that we have solved rather well.
How do you build the pyramids? You need to have solved the problem of large teams. How do you win World War II? You need to have solved the problem of large teams. How do you send men to the moon and bring them safely back to Earth? You need to have solved the problem of large teams.
But those big projects aren’t the only large-team accomplishments we see, are they? How do you build the telephone network, or the highway network, or the internet, or iPhones, or automobiles? All of this is about large teams. The infrastructure and defense of our vast, globe-spanning, civilization is proof that we have solved the problem of large teams.
Large teams are a solved problem.
The problem that was not solved back in the late ’80s when the Agile movement began was the problem of small software teams. We did not know how to effectively organize a relatively small group of programmers to be effective. And it was this problem that Agile solved.
It’s important to understand that this was a software problem, not a small team problem. The problem of small teams had been solved millennia before by military and industrial organizations around the world. The Romans could not have conquered Europe if they had not solved the problem of organizing small squads.
Agile is the set of disciplines by which we organize small software teams. Why do we need a special technique for this? Because software is unique. There are very few tasks like it. The cost/benefit and risk/reward trade-offs in software are different from just about every other kind of work. Software is like building, except nothing physical gets built. Software is like mathematics, except that nothing can be proven. Software is as empirical as science, but there are no physical laws to be discovered. Software is like accounting, except it describes time-ordered behavior instead of numerical facts.
Software is really like nothing else. So in order to organize a small team of software developers, we need a special set of disciplines that are tuned to the unique aspects of software.
Look back through the disciplines and practices that we’ve been talking about in this book. You’ll note that each one of them, almost without exception, is tweaked and tuned specifically for the unique aspects of software. This ranges from the obvious practices like Test Driven Development and Refactoring to the more subtle implications of The Planning Game.
The bottom line here is that Agile is about software. In particular, it is about small software teams. I am always frustrated when people ask me how to apply Agile to hardware, construction, or some other task. My answer has always been that I don’t know, because Agile is about software.
What about Agile in the large? I don’t think there is such a thing. Organizing large teams is a matter of organizing them into small teams. Agile solves the problem for small software teams; the problem of organizing small teams into larger teams is a solved problem. Therefore, my answer to the question of Agile in the large is simply to organize your developers into small Agile teams, then use standard management and operations research techniques to manage those teams. You don’t need any other special rules.
Now the question that could be asked is that, since software for small teams was so unique that it required us to invent Agile, why doesn’t that uniqueness hold for organizing small software teams into larger software teams? Isn’t there some unique aspect to software that carries beyond the small team and affects the way larger teams ought to be organized?
I doubt it, because the problem of large teams, which we solved more than 5000 years ago, is the problem of getting many diverse kinds of teams to cooperate. Agile teams are just one of myriad kinds of teams that need to be coordinated in a large project. The integration of diverse teams is a solved problem. I see no indication that the uniqueness of software teams unduly affects their integration into larger teams.
So again, the bottom line from my point of view is this: There is no such thing as Agile in the large. Agile was a necessary innovation for organizing small software teams. But once organized, those teams fit into the structure that large organizations have used for millennia.
Now, again, this is not a subject that I have diligently researched. What you just read is my opinion, and I could be very wrong. Perhaps I’m the old curmudgeon telling all those Agile-in-the-large kids to get off my lawn. Time will tell. But now you know which way I’m betting.
By Tim Ottinger and Jeff Langr, April 16, 2019*
* Used with permission.
Makers master their tools. Carpenters become adept with hammer, measure, saw, chisel, plane, and level—all inexpensive tools—during the first steps of their career. As their needs grow, carpenters learn and employ more powerful tools (which are usually more expensive): drill, nail gun, lathe, router, CAD, and CNC to name a few.
However, master carpenters don’t abandon hand tools; they choose the right tool for the job. Using only hand tools, skilled makers can craft wood projects with higher quality and sometimes faster than with power tools. As a result, wise carpenters master their hand tools before moving on to tools that are more sophisticated. They learn the limits of the hand tools so that they know when to reach for a power tool.
Regardless of whether they use hand or power tools, carpenters always seek to master each implement they choose for their toolbox. This mastery allows them to focus on the craft itself—the delicate shaping of a high-quality piece of furniture, for example—rather than the tool. Without mastery, a tool becomes an impediment to delivery, and a poorly used tool can even cause harm to the project and its operator.
Software developers must master a number of core tools:
At least one programming language, and often more
An integrated development environment or programmer’s editor (vim, Emacs, etc.)
Various data formats (JSON, XML, YAML, etc.) and markup languages (including HTML)
Command-line and script-based interaction with the operating system
Source repository tools (Git. Is there another choice?)
Continuous integration/build tools (Jenkins, TeamCity, GoCD, etc.)
Deployment/server management tools (Docker, Kubernetes, Ansible, Chef, Puppet, etc.)
Communication tools: email, Slack, the English language (!)
Testing tools (unit testing frameworks, Cucumber, Selenium, etc.)
These categories of tools are essential to building software. Without them, it is impossible to deliver anything in today’s world. In a sense, they represent the programmer’s toolbox of “hand tools.”
Many of these tools demand hard-earned expertise for effective use. The landscape meanwhile changes constantly, making tool mastery even more challenging. The savvy developer finds the path of least resistance and highest value through all the tools involved: what gives the biggest bang for the buck?
The tool landscape changes rapidly because we are always learning more effective ways to accomplish our goals. Witness the wide variety of source repository tools introduced in the past few decades: PVCS, ClearCase, Microsoft Visual SourceSafe, StarTeam, Perforce, CVS, Subversion, and Mercurial, to name a few. All suffered problems—too flaky, too proprietary or closed, too slow, too invasive, too scary, too complex. A winner eventually emerged, one that managed to overcome most objections: Git.
One of the most empowering aspects of Git is its ability to make you feel safe. If you were around long enough to have used one of the other tools, you likely felt a little nervous from time to time. You needed a live network connection to the server, or your work was at risk. The CVS repository would get corrupted occasionally, requiring you to muck around in its attic to hopefully uncorrupt things. The repository server crashed sometimes; even if it was backed up, you risked losing half a day’s work. Some proprietary tools suffered repository corruption, meaning you’d be on the phone for hours with support while soliciting donations for the king’s ransom needed to pay for uncorrupting it. With Subversion, you’d fear doing much branching because the more source files that were involved, the longer you’d have to wait when switching branches (we’re talking many minutes here).
A good tool should feel comfortable in your hands and not make you queasy with the dread of having to use it. Git is quick, it gives you the ability to commit locally instead of only to a server, it allows you to work from your local repository without a network connection, it handles multiple repositories and multiple branches well, and it deftly supports merges.
The interface to Git is fairly streamlined and direct. As a result, once you’ve learned Git well enough, you don’t think much about the tool itself. You instead focus on the real needs: safe storage, integration, and management of source code versions. The tool has become transparent.
Git is a powerful and complex tool, so just what does it mean to learn it “well enough”? Fortunately, the 80/20 rule applies: a small fraction of Git’s capabilities—20%, perhaps—gets you well over 80% of the day-to-day needs you’ll have for source management. You can learn most of what you need in minutes. The rest is available online.
The simplicity and effectiveness of using Git led to wholly unplanned, new ways of thinking about how to build software. Linus Torvalds probably would have thought it was insane to use Git as a tool to quickly throw away bits of code, but that’s exactly what proponents of the Mikado Method6 and TCR (Test && Commit || Revert)7 promote. And even though a key, powerful aspect of Git is its ability to handle branches very effectively, countless teams almost exclusively do trunk-based development while using Git. The tool is exapted (used effectively in ways the originators hadn’t intended).
6. Ellnestam, O., and D. Broland. 2014. The Mikado Method. Shelter Island, NY: Manning Publications.
7. Beck, K. 2018. test && commit || revert. Accessed at https://medium.com/@kentbeck_7670/test-commitrevert-870bbd756864.
Great tools do the following:
Help people accomplish their objectives
Can be learned “well enough” quickly
Become transparent to users
Allow adaptation and exaptation
We hold up Git here as an example of a great tool…as of 2019. You may be reading this in some future year, so remember that the landscape changes.
Agilists are known for using whiteboards, tape, index cards, markers, and various sizes of sticky notes (small and flip-chart-size) for visual management of their work. These simple “hand implements” have all the qualities of a great tool:
They help make work in progress visible and manageable.
They’re intuitive—no training required!
They require negligible cognitive overhead. You can use them easily while concentrating on other tasks.
They’re easily exapted. None of these tools was designed expressly for managing software development.
They’re adaptable. You can use tape or sticky putty with them, clip pictures or icons to them, tape additional indicators to them, and add nuance of meaning through novel use of custom colors and iconography.
They’re all inexpensive and easily acquired.
Co-located teams can easily manage huge and complex projects using only these simple and inexpensive physical tools. You can radiate key information using a flip chart sheet taped to a wall. Such information radiators summarize important trends and facts for team members and sponsors alike. You can use these information radiators to devise and present new kinds of information on the fly. The flexibility is nearly infinite.
Every tool has its limitations. One key limitation of physical tools is that they’re not very effective for distributed teams, only for people within visual range. The physical tools also don’t automatically maintain history—you have only the current state.
The original XP project (C3) was largely managed with physical tools. As Agile grew, so grew an interest in automated software tools. Some legitimate reasons for this are as follows:
Software tools provide a good way to help ensure data is captured in a consistent form.
With consistently captured data, you can readily derive professional-looking reports, charts, and graphs.
Providing history and safe storage is easy.
You can instantly share the information with everyone no matter where they reside.
With tools like online spreadsheets, you can even have a completely distributed team collaborate in real time.
Low-tech tools are a turn-off to some folks more accustomed to slick presentations and software. And since we are the industry that builds software, the natural inclination of many of us is to automate everything.
Bring on the software tools!
Eh…maybe not. Let’s pause and think here. Automated tools may not support your team’s specific process. Once you own a tool, the path of least resistance is to do whatever the tool provides regardless of whether it meets the team’s needs.
Your team should establish the pattern of work compatible with their specific context first, and then consider using tools that support their workflow.
Workers use and control tools; tools don’t control and use people.
You don’t want to get locked into other people’s process flows. Whatever you are doing, you want to get a handle on your process before you automate. The question, though, isn’t whether to use automated or physical tools. The question should be “Are we using great tools or not-great tools?”
Quickly after the onset of Agile, numerous software systems for managing Agile projects appeared. These Agile Lifecycle Management (ALM) systems, ranging from open source to polished and expensive “shrink-wrapped” products, allow you to collect Agile team data, manage long lists of features (backlogs), produce sophisticated graphs, provide cross-team summary views, and do some numerical processing.
It seems handy to have an automated system help us with this kind of work. In addition to their primary features, ALM tools have beneficial features: Most allow remote interaction, track history, handle some of the dirty-work bookkeeping, and are highly configurable. You can use a plotter to create professional, multicolor graphs on oversized paper that can be posted as information radiators in a team space.
Yet, despite being feature rich and commercially successful, ALM tools utterly fail at being great. This failure provides a good cautionary tale.
Great tools can be learned “well enough” quickly: ALMs tend to be complicated, usually demanding up-front training. (Hmm. We’re trying to remember the last index card training we attended.) Even with training, team members must often resort to searching the internet to figure out how to accomplish tasks that ought to be simple. Many acquiesce to the tool’s complexity, punt on digging any deeper to figure things out, and end up tolerating slow, clunky ways of working.
Great tools become transparent to users. We constantly see team members watching the designated driver trying to figure out the tool. They make seemingly drunken stabs at manipulating the story cards. They weave between web pages, slapping text about via copy-and-paste, trying to hook up stories with each other or to their parent “epics.” They stumble over stories, tasks, and assignments in an attempt to make them all get along. It’s a mess. These tools often require constant attention.
Great tools allow adaptation and exaptation. Want to add fields to an ALM (virtual) card? You might be able to find a local programmer expert dedicated (sacrificed) to supporting the tool. Or you might end up having to submit a change request to the vendor. A five-second turn around time for low-tech tools turns into a five-day or five-week delay with an ALM. Rapid-feedback experiments around managing the process become impossible. And of course, if you don’t need the extra fields after all, someone has to revert the changes and re-release the configuration. ALM tools aren’t always easily adapted.
Great tools are affordable. The ALM tool licensing fees, which can run into many thousands of dollars per year, are just the start. Installation and use of these tools can require additional considerable costs in the form of training, support, and sometimes customization. Ongoing maintenance and administration further add to the dear cost of ownership.
Great tools help people accomplish their objectives. ALM tools rarely work the way your team does, and often their default mode is at odds with Agile methods. For example, many ALM tools assume that team members have individual work assignments, which makes them nearly unusable for teams who work together in a cross-functional way.
Some ALM tools even provide a pillory board—a dashboard showing each individual’s workload, utilization, and progress (or lack thereof). Rather than highlighting the flow of work toward completion and promoting shared accountability—the truly Agile way—the tool becomes a weapon used to shame programmers into working harder and longer.
Where teams would gather for their morning standup (daily Scrum), they now gather to update their ALM. The tool has replaced interactions of individuals with automated status reporting.
Worst of all, ALM tools don’t often radiate information like physical tools can. You have to log in and hunt around to find what you seek. When you do find the information, it often comes accompanied by a pile of other information you don’t want. Sometimes the two or three graphs or displays you want are on different web pages.
There’s no reason ALMs can’t be great tools someday. But if you only need to manage a card wall and must use software, adopt a general-purpose tool like Trello.8 It’s easy, immediate, cheap, extensible, and won’t make you queasy.
8. Again, as of 2019. The landscape changes.
Our ways of working constantly change. We evolved from SCCS to RCS to CVS to Subversion to Git over the years, tracking a sea change in the ways we managed source code. We have similar evolution in testing tools, deployment tools, and the like (not listed here). We are likely to see a similar progression in automated ALM tools.
Given the current state of most ALM tools, it may be safer and smarter to begin with physical tools. Later, you can consider using an ALM tool. Make sure that it’s quick to learn, transparent in daily use, easy to adapt, and within your means to acquire and run. Most importantly, ensure that it supports the way your team works and provides a positive return on your investment.
By Damon Poole, May 14, 2019*
* Used with permission.
Damon Poole is a friend of mine who disagrees with me about many things. Agile coaching is just one of those things. So, I thought he’d make a good choice to provide you with a different viewpoint.
There are many paths to Agile. And in fact, many of us have walked the path unintentionally. One could argue that the Agile Manifesto was the result of the authors noticing that they were all on a similar journey and deciding to describe it such that others might choose to join them on their journey. My path to Agile started by walking into an appliance store in 1977 that happened to be selling TRS-80s. As a complete novice, I helped an experienced programmer debug a Star Trek game simply by asking questions. Today we call that Pair Programing. And, as it happens, asking questions is a big part of coaching.
From then until somewhere around 2001, I was unintentionally Agile. I only coded in small cross-functional teams, mostly with an in-house customer, focused on what are now called user stories, and we only produced small, frequent releases. But then, at AccuRev, our major releases started to get further and further apart, cresting at 18 months in 2005. For 4 years, I was unintentionally doing Waterfall development. It was horrible and I didn’t know why. Moreover, I was considered a “process expert.” Apart from the details, this is a familiar story for many people.
My introduction to Agile was painful. Back in 2005, before the Agile Alliance conference and others skyrocketed in popularity, there were the Software Development magazine conferences. At the speaker reception for Software Development East, after doing a talk on management practices for distributed development that was completely devoid of the word “Agile,” I found myself surrounded by software industry thought leaders such as Bob Martin, Joshua Kerievsky, Mike Cohn, and Scott Ambler. It seemed the only topics they had any passion for involved things like 3×5 cards, user stories, Test-Driven Development, and Pair Programming. I was horrified that all of these thought leaders had been taken in by what I saw as snake oil.
A few months later, while researching Agile in order to properly debunk it, I had an aha moment. As a programmer and business owner, this new insight came from understanding Agile as an algorithm for finding the highest-value-producing features in the market and then turning them into revenue faster.
After this moment of inspiration, I developed a passion for sharing Agile with everyone. I did free webinars, wrote blog posts, spoke at conferences, joined and then ran the Agile New England meetup in the Boston area, and did everything I could to spread the word. When people shared their difficulties in implementing Agile, I was brimming with enthusiasm to help. I launched into problem-solving mode and explained what I thought they should do.
I started to notice that my approach often led to objections and more questions. And it wasn’t just me. On the extreme end, I witnessed multiple Agilists in conference sessions becoming confrontational with those that had not yet seen the light. I began to realize that for people to really embrace and utilize Agile effectively, there needed to be another way to impart Agile knowledge and experience that took into account the learner’s unique circumstances.
The concept of Agile is simple. It is described in a mere 264 words in the Agile Manifesto. But becoming Agile is hard. If it was easy, everybody would already be doing it and there would be no need for Agile coaches. People have a hard time making changes in general, let alone the amount of change it takes to fully embrace Agile. Becoming Agile involves revisiting entrenched beliefs, culture, process, thinking, and ways of working. Getting one person to shift thinking and help them see “what’s in it for me” is challenging enough. Doing that for a whole team compounds the difficulty, and when it takes place within an environment that is purpose built for traditional ways of working, the difficulty is compounded again.
A truism of all change initiatives is that people do what they want to do. The key to lasting change is to find problems or opportunities that people are aware of and have a desire to invest in and then help them achieve their goals, offering expertise only as requested and needed. Everything else will fail. Coaching helps people discover the blind spots and surface the underlying beliefs that keep them from moving forward. It helps people to solve their own challenges and reach their own goals rather than just prescribing a solution.
In 2008, Lyssa Adkins came on the scene with a very different approach to Agile coaching. She put an emphasis on the pure coaching aspect of Agile coaching by introducing skills from professional coaching into the Agile coaching world.
As I learned more about professional coaching and Lyssa’s approach and started to incorporate it into my own way of working, I came to understand that people get a tremendous amount of value out of the process of coaching itself. That value is completely separate from any increase in Agile knowledge or expertise that a coach may also impart to them.
In 2010, Lyssa fully described her approach to Agile coaching in her book Coaching Agile Teams. 9 At the same time, she started offering an Agile coaching course. In 2011, those learning objectives formed the basis of ICAgile’s Certified Agile Coach (ICP-ACC) learning objectives and International Consortium for Agile then began accrediting other instructors with their own ICP-ACC offerings. ICP-ACC courses are currently the most comprehensive source of professional coaching instruction in the Agile industry.
9. Adkins, L. 2010. Coaching Agile Teams: A Companion for ScrumMasters, Agile Coaches, and Project Managers in Transition. Boston, MA: Addison-Wesley.
The ICP-ACC certification includes the coaching skills of active listening, emotional intelligence, presence, providing clear and direct feedback, asking open-ended and non-leading questions, and remaining neutral. The full set of professional coaching skills is even broader. For instance, the International Coach Federation (ICF), representing more than 35,000 certified professional coaches, defines 70 competencies organized into 11 categories. Becoming a certified professional coach involves a great deal of training and a rigorous certification process that requires demonstrating all 70 competencies and documenting hundreds of hours of paid coaching.
Many of the structures, practices, methods, and techniques used in the Agile community for teaching Agile and being Agile align with the intent of professional coaching. They are “coaching tools” that help individuals and groups discover for themselves what’s getting in their way and decide on their own how to move forward.
One coaching competency is powerful questioning, one aspect of which is “asking questions that evoke discovery, insight, commitment, or action.” Retrospectives, especially variants like “Team with the Best Results Ever” or “Six Hats” are a way to ask powerful questions that enable a team to discover opportunities for change on their own and decide independently how to go about pursuing those opportunities. An open space (aka unconference) is a way to ask a powerful question of a large group of people, even a whole organization.
If you have taken formal training on Agile or an Agile methodology, you likely played a number of games that illustrated Agile concepts. Games such as the penny game, Scrum simulations, Kanban Pizza, or LEGO city building. These games give participants an experiential connection to the power of self-organization, small batch sizes, cross-functional teams, TDD, Scrum, and Kanban. When the games are run with the intent of raising participants’ awareness and then letting participants decide what to do next, they capture the spirit of professional coaching.
There is a growing body of coaching tools. Many of them can be found at tastycupcakes.org, retromat.org, and liberatingstructures.com.
If we are working with a team that has never heard of Kanban but may benefit from it, no amount of powerful questions or other professional coaching techniques are going to result in someone spontaneously inventing Kanban. At that point, an Agile coach switches to a mode of offering potentially useful expertise. If the team is interested, the Agile coach then provides their expertise, teaching and mentoring the team with the intention of moving back to pure coaching as soon as the team has mastered the new knowledge.
There are six main areas of expertise that Agile coaches draw from: Agile frameworks, Agile transformation, Agile product management, Agile technical practices, facilitation, and coaching. Each coach will have her own mix of skills. Most organizations start by looking for an Agile coach that has Agile framework experience. As companies progress in their journey, they come to appreciate the value of each area of Agile expertise.
One area of expertise that organizations consistently underestimate is the need for everyone involved in coding and testing to get good at writing code and creating tests that are appropriate for an Agile environment as described elsewhere in this book. This is important to keep the focus on adding new functionality with new tests rather than constantly updating existing code and tests and/or increasing technical debt, both of which add an increasing drag on velocity.
Somewhere around 2012, as more organizations had success with individual teams, there was a huge uptick in interest in scaling Agile. That is, transforming the organization from being purpose built for traditional ways of working to one that is purpose built to support Agile ways of working.
These days, most Agile coaching happens within the context of multiple teams, sometimes tens or hundreds of teams. And it frequently starts with resources (people) siloed and allocated to three or more unrelated projects. Not all of those “teams” are working together toward a common purpose, but they are all working within a traditional environment that thinks in terms of multiyear funding, portfolio planning, and project-based thinking rather than team- and product-based thinking.
Agile in the large is a very similar problem to Agile at the team level. Part of the difficulty of getting the benefit of Agile is finding and removing all of the obstacles that get in the way of a team coordinating their efforts to go from request to ready for release in the span of a couple of weeks. It is even harder to get a team to the point of releasing on demand.
These difficulties are multiplied and magnified when trying to coordinate the efforts of multiple teams to produce a single deliverable. Unfortunately, a common pattern of Agile adoption in large organizations is to treat the adoption like a traditional project. That is, a top-down command and control rollout of a huge number of changes decided on as part of an up-front design. And when I say huge, I mean literally thousands of changes. It is in the thousands because when you ask hundreds of people to make dozens of changes to their day-to-day behavior, each of those have a chance to succeed or fail based on how each of those hundreds of people feels about the impact on them personally. Starting out by saying that a large Agile framework is your destination sounds very similar to saying, “Our plan is to implement this large stack of software requirements.”
In my experience working with many large Agile adoptions (many with hundreds of teams) and working alongside many experienced Agile coaches, the most important thing I’ve learned is this: the problem of successfully adopting Agile is exactly the same problem as creating successful software. Just as it is best to grow your software based on frequent customer interaction, the only process changes that will stick are those that are directly connected to what the people affected understand and desire based on their own unique circumstances. In other words, I believe that the most effective Agile transformation strategy is to treat the introduction of Agile as an Agile endeavor, applied using coaching skills.
The Agile Manifesto is a great template for coaching and coordinating the work of multiple teams: “Give them the environment and support they need, and trust them to get the job done.” In support of this, the Agile community has a whole host of scaling patterns that are compatible with the values and principles of the Agile Manifesto. I am not referring to frameworks here, but rather the individual practices from which all of the frameworks are built.
All of the frameworks are basically “ready-made” recipes composed of individual Agile practices. Rather than implementing one of these recipes, consider using Agile and coaching to discover and implement a custom recipe that is tailor-made to your specific circumstances. If, in the end, that recipe ends up being SAFe, Nexus, LeSS, or [email protected], that’s great!
Here is an overview of how the most successful enterprise Agile coaches combine Agile and coaching to discover and implement Agile in a way that is perfect for the organization. The essence of coaching at the individual level is helping people solve problems on their own. Coaching at the team and organizational levels means helping teams achieve their goals on their own. First, the coach thinks of everyone affected by the Agile adoption as “the customers.” Then, using retrospectives, open-space events, and similar techniques, they discover what the customers see as the challenges and opportunities. This becomes the organization’s Agile adoption backlog. Next, using group decision-making tools such as dot voting, the coach determines the top of the backlog. They then assist the organization to implement a handful of the top backlog items. Finally, they retrospect and repeat. Of course, for many of the people involved, this will be their first Agile adoption. Coaching alone is not enough; teaching and mentoring will also come into play so that people will be able to make informed decisions.
Here is a list of individual practices to consider for your Agile adoption backlog. This list was originally created and has been periodically updated by the Agile coaching trifecta of gathering sticky notes, de-duping, and dot voting with a group of a dozen or so enterprise coaches. This is just a high-level description of these practices for reference. There are many more Agile practices out there—consider this a starting place. For instance, rather than adopting Scrum, Kanban, XP, or one of the scaling frameworks, consider which single practice from the list below is most relevant to a current need for a defined group or team, and adopt that. Try it for a while, then repeat.
The practices of Kanban—Kanban practices include making the work visible (via a card wall), limiting work in progress, and pulling work through the system.
The practices of Scrum and XP—These two methodologies are grouped together because they are very similar apart from the technical practices in XP. In SAFe, for instance, they are referred to collectively as ScrumXP. Between the two of them, they include a wide variety of practices such as short daily team meetings, a product owner, a process facilitator (aka Scrum Master), retrospectives, a cross-functional team, user stories, small releases, refactoring, writing tests first, and pair programming.
Align team events—When the team events across multiple teams, such as standups and retrospectives, are aligned in time, it is possible to then roll up daily and systemic impediments via an escalation tree. This involves aligning iteration start and stop times as well as iteration length. Teams that don’t use iterations and are able to release on demand can align with any other cadence.
Escalation trees—If it makes sense to always work on items that produce the highest value, then it makes sense to escalate impediments immediately via a well-defined escalation path. This applies to the commonly used practice of “Scrum of Scrums” and the less well-known “retrospective of retrospectives.” One pattern for this is [email protected]’s fractal pattern of scaling via Scrum and Scrum of Scrums backstopped with an Executive Action Team.
Regular interteam interaction—This practice involves regular interaction between the Scrum Masters, Product Owners, and team members who are working together toward a common deliverable. One method for this is to run regular open-space events.
Portfolio Kanban—Traditional portfolio management practices tend to allocate people to multiple teams, which leads to rampant multitasking. Multitasking creates friction, increases complexity, and reduces throughput. Portfolio Kanban sets work in progress limits at the initiative level in order to ensure that the organization is focused on the highest-value work at all times. Having fewer projects in progress at a time also vastly simplifies (or even eliminates) multiteam coordination. Portfolio Kanban works best when paired with Minimum Viable Increments.
Minimum Viable Increments—There are many variants of this idea, but they all boil down to thinking about what is the shortest path to producing the highest value in the shortest time. A growing number of organizations are taking this to the extreme by implementing Continuous Delivery: releasing small updates on a frequent basis, sometimes as frequently as multiple times per day.
Most multiteam Agile adoptions run into problems when they focus on coping with complexity rather than solving for simplicity. In my experience, one of the cornerstones of Agility in the large is very high levels of Agility at the team level and very low levels of complexity everywhere. Having a flotilla of speedboats is nearly useless if they are all joined together. Here are some of the practices that are generally associated with team-level Agile that do double duty as enablers of multiteam coordination.
The SOLID principles—Although these principles are valuable at any scale, they are especially useful for simplifying multiteam coordination by dramatically reducing dependencies.
Small, valuable user stories—Small, individually releasable stories limit the scope of dependencies, which simplifies multiteam coordination.
Small, frequent releases—Whether these releases are delivered to the customer or not, the practice of having a releasable product across all of the teams involved helps to surface coordination and architectural issues so that the root cause can be found and addressed. Some Scrum teams forget this, but Scrum says, “The increment must be in useable condition regardless of whether the Product Owner decides to release it.” That means it must be integrated with the work of any other teams that it depends on.
Continuous Integration—XP takes an even stronger stand on integration, calling for integration across the entire product after every checkin.
Simple Design—This practice, also known as Emergent Design, is one of the hardest practices to learn and apply because it is one of the most counter-intuitive practices. Teams struggle with this even when they don’t need to coordinate with other teams. When coordinating the work of multiple teams, monolithic, centralized, preplanned architectures create massive dependencies between teams that tend to force them to work in lock step, thus defeating much of the promise of Agile. Simple Design, especially when used with practices such as a microservices architecture, enables Agility in the large.
In the last few years, professional coaching and professional facilitation have started to make their way into the Agile curriculum. The Scrum Alliance’s Advanced Certified Scrum Master (ACSM) course has a few learning objectives related to coaching and facilitation, and their Certified Team Coach (CTC) and Certified Enterprise Coach (CEC) programs require that you acquire even more facilitation and coaching skills than that. The Scrum guide now refers to Scrum Masters as coaching those they serve.
As more people are exposed to professional coaching through the courses mentioned above and through interaction with professional coaches working in the Agile community, the “coach” part of Agile coach is getting more attention. In the last couple of months, it seems that the interest in professional coaching has blossomed. People have started skipping the ICP-ACC path in favor of going directly to the ICF path. The first ICF-credentialed coaching school that caters to Agilists has been formed, and there’s at least one more on the way. The future of Agile coaching is bright!
In many ways, this chapter has been more about what not to do than what to do. Perhaps that’s because I’ve seen so many examples of how not to become Agile. After all, I still think, as I thought 20 years ago, “What could be easier? Just follow a few simple disciplines and practices. Nothing to it.”