Every Company Is a Tech Company

According to forecasts by Gartner, IT spending in the United States will grow to $920 billion in 2018, and software development makes up the largest part of that expenditure, at $275 billion. Yet studies such as the Standish Chaos study consistently show that failed and challenged projects make up about 70% of the total spend. In any other part of the business, this failure rate would be intolerable.

Yet many firms have beaten the odds of failure by adopting new management styles and software development practices. Just as manufacturers have transitioned to Lean and just-in-time production, firms that depend on technology to serve their markets (and today that means just about every enterprise) are transitioning to Agile.

Businesses born on the internet have long since proven that industrial-age project management and software development techniques don’t apply. Plans developed at the beginning of the project, when we know the least, don’t stand up to today’s velocity of market disruption. Traditional development life cycles—with their big up-front plans, fixed-price estimates, and rigorous gated disciplines—stifle innovation. They mislead sponsors, as dates and budgets are continuously set and exceeded. They deliver too late, or miss the target, as disruptive market forces change the game on the fly. Predictive techniques can’t keep up with the unpredictable, chaotic markets of the mobile, internet economy. Worse, these estimates, made on flawed plans, instantly become fixed deadlines and budgets in the sponsor’s mind. In short, the fallacies of predictability and control, especially in creative, emergent software development efforts, were exposed as flawed.

Over time we’ve learned that developing software and, in fact, designing most modern products, is a unique, creative endeavor that requires experimentation, exploration, and commitment from both the development team and their customers. Waterfall isn’t sustainable. Agile has grown to suit this new market reality and is proliferating beyond software and across the enterprise.

How Agile Delivers

Agile is both evolutionary and revolutionary. The basic principles of Agile development have been evolving since the 1980s, when experiments in new techniques such as Rapid Application Development and Radical Project Management were underway. Agile is a natural progression from lessons learned through the history of developing applications. It’s revolutionary because it’s more than just a new set of development practices; it requires significant changes to culture, roles, methods, and metrics. Like the Lean production methods that inspired it, Agile requires executives, managers, and team members to bring new skills and new attitudes to the development of software and, indeed, of all new products.

And Agile works. A recent InfoQ study clearly illustrates that Agile is more successful than Waterfall methods. Agile projects succeed 42% of the time; in comparison, Waterfall projects succeed only 14% of the time. Failed Waterfall projects made up 39% of the total surveyed, but Agile failures were only 9%.

Agility has measurable financial value, as well. According to a study by The Economist Intelligence Survey, Agile businesses have 29% higher earnings per share, 20% better net margins, and 8% higher revenue growth than their non-Agile competitors. And 88% of executives cited agility as a critical success factor, so much so that 50% called it a key competitive advantage.

What do Agile enterprises do that enable them to reap these benefits?

Unlike traditional Waterfall methodologies, Agile projects have the following attributes:

• Large projects are broken down into smaller increments and releases.

• Development teams are composed of members with cross-functional skills, so the team can take responsibility for the project from conception to delivery.

• Products are delivered incrementally, in short, regular iterations, with each iteration delivering a small number of features.

• The entire life cycle is performed within each iteration, from planning to design, development to testing, to get that module of functionality to done.

• Teams collaborate and self-manage, working together to make their own plans and figure out for themselves the best way to tackle their work.

• The entire ecosystem of participants, from the executive sponsor to the developers and testers, are actively involved in prioritizing, planning, and reviewing the project as it is built to ensure continuing alignment with strategic goals.

• Rather than measuring progress on a Gantt chart, Agile organizations have one simple measure of progress: working software.

• Changes, improvements, and additional features are incorporated throughout the project life cycle. Change, rather than being perceived as a failure of the process, is seen as an opportunity to improve the product and make it more fit for its use and business purpose.

Many firms considering the move to Agile worry about the familiar artifacts and practices to which they’ve grown accustomed in Waterfall. Where’s my project plan? How do I track and measure progress? The myth about Agile is that it’s an undisciplined, “make it up as you go along” practice, whereas the truth is that Agile incorporates more planning and rigor than most Waterfall programs. Although Agile has many variants, from Scrum to XP to Kanban, they all share the same working principles:

• Build plans just-in-time, in the form of prioritized backlogs of work items, that have the potential to be reorganized, changed, and reprioritized as business needs evolve.

• Make all work visible, through the use of taskboards, either physical (cards on a wall) or virtual (in an Agile project-management tool).

• Simplify metrics by relating all measurements to actual, valuable features delivered and accepted by customers.

Agile teams assume that clients can’t know what they want in detail until they see a prototype. Agilists assume that as we iterate toward a complete solution, small releases at a time, our stakeholders will guide us to a product that fits their needs and expectations. Each iteration delivers a working product or prototype, and the response to that product serves as crucial feedback into the succeeding iterations.

The strategic goal of Agile is not agility itself, it’s competitive advantage and enhanced business results. Respondents cite a number of compelling drivers of the migration to agility, including better ability to handle changing priorities, increased team productivity, and increased project visibility. Other reported advantages include improved morale and motivation, better predictability, faster time to market, and enhanced quality. Reduced risk, improved strategic alignment, enhanced maintainability; the benefits reported are so compelling that they seem almost magical.

Not only does Agile deliver, but its momentum has surged in the past few years. Agile has become the default approach used by a majority of IT shops. VersionOne’s 11th State of Agile survey illustrates the swell of adoption:

• Ninety-four percent of respondents said that their firm was using Agile, at least in some teams.

• Fifty-six percent have been using Agile for more than three years.

The broad adoption, the reported benefits, and the statistics all demonstrate that agility is not just the latest management fad, destined to go the way of Business Process Reengineering or Matrix Management. It’s the logical evolution of software development practices, inspired by proven lean ideas, and suited to the unique challenges of today’s dynamic, mobile, online economy.

Why Agile?

Years of experience, and piles of studies, highlight the problems with industrial-age management styles: rigid bureaucracy, disengaged workers, frustrated customers, failed projects, offshoring, and downsizing. The management styles that suited a hierarchical workplace in an industrial economy are no match for the emergence of rapid, dynamic, online markets, or the accelerating expectations of the consumer.

When Agile emerged, many IT shops had already spent years building predictive, proprietary project-planning models. Every company had its own branded method, often heavily gated and prescriptive. Project management offices grew, based on the hope that rigorous enforcement of gated methodologies would reduce risk. The early 1990s were the golden age of multibinder, bureaucratic, and paper-heavy project management disciplines.

At the same time, evolving management theories like Lean flipped the script on traditional management. The typical pyramid structure, with decision-making executives at the top and compliant workers at the bottom, was upended by Lean theories.

Lean organizations believe the following:

• The customer, not the firm, determines value.

• Workers, not executives, make the best decisions about how work gets done.

• Leaders serve workers, not the other way around.

• Teams don’t need foremen: they need alignment and autonomy to self-manage and self-correct.

• Responding rapidly to empirical data breeds success, not predictive, granular project plans.

The innovative mindset evolving from these Lean concepts was the jumping-off point for management theories from Six Sigma to Agile. Agile is based on radical ideas that have been percolating through the management community for years.

As the flaws in traditional models became apparent, the shoots of a “light methodologies” movement were budding. For example, Rob Thomsett wrote a book called Radical Project Management that became a keystone to this change. The IT project community was determined to devise a method that combined the discipline of a phase-gated approach with the low overhead of these new radical project ideas. Through experimentation with methods like Lean and Test-Driven Development, teams tried building smaller, more modular functions, delivering them quickly for feedback, and incrementally growing an application. Long before the founding Agile Manifesto was written (in 2001), experiments in software development proliferated. From Kent Beck’s XP trials at Chrysler, to the theories of Ziv Hadar and Watts Humphrey, the foundational ideas of Agile were long stewing under the surface. Agile was revolutionary to those shops that had invested their energy into developing and instituting rigorous project control methods. Early Agile ideas were vigorously contested, and the mythology of Agile as an undisciplined, improvisatory exercise were born in those debates.

Ziv Hadar and Watts Humphreys wrote about the uncertainty of developing requirements specifications, based on the belief that clients can’t know what they want up front, and that iterative demonstration of modular components is a core practices for delivering valuable software. They coined the phrase “The Software Uncertainty Principle.”

As we’ve seen, surveys show staggering claims for Agile benefits; can a simple software development methodology live up to the fanfare? Agility has many myths circulating around it, but the myth of Agile as simply a methodology, designed to replace heavy, bureaucratic software development life cycles (SDLCs), is particularly misleading. It’s true, though, that the Agile Manifesto initially focused on software developers iterating through an incremental process. Agile rookies learn Agile, at first, as a series of practices, like standups, demos, and retrospectives. Many firms begin their Agile journey by piloting these practices in small enclaves within IT. As Agile matures in an enterprise, these new software engineering practices end up touching the entire organization, as faster delivery cycles affect marketing and logistics, and enhanced quality and maintainability free developers from drudge work and enable even higher productivity.

However, if we take a look at the Agile Manifesto, it’s clear that these ideas can translate to any function in the enterprise (with some adjustments):

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

Even though the Agile Manifesto is brief, it contains a distillation of the ideas that had been circulating in the IT community, as developers searched for alternatives to Waterfall and its many pitfalls. The Agile Alliance, the organization formed to promote these ideas, subsequently published a set of 12 Agile Principles to flesh out the ideas surfaced in the manifesto. They stress the Lean concepts of simplicity, customer participation, just-in-time planning, and working products. Many Agilists believe the principles to be a clearer, more definitive guideline for the adoption of Agile practices than the manifesto itself.

These Agile principles, derived from Lean manufacturing theories, present a viable alternative to time-honored traditional models. The new team structures, roles, and interactions that Agile brings, however, imply deep and disruptive change, down to the organizational bone. When traditional meets Agile, friction is inevitable.

Understanding How Lean Fits In

Lean is a management practice based on the elimination of waste, and the optimization of the flow of work through the organization’s value chain.

The first popular account of this management approach came in the book The Machine that Changed the World, based on an examination of techniques implemented at Toyota. Due to the success of Toyota in producing quality vehicles inexpensively and competing in worldwide markets, many firms tried to adopt these Lean methods. Techniques such as just-in-time inventory, pull manufacturing, and waste reduction were introduced in many firms and then adapted for local conditions and needs. These ideas have influenced the development of Agile software development. In 1989, Professor James Womack and consultant Dan Jones published Lean Thinking, a survey of the Lean manufacturing techniques that helped create the “Japanese miracle” of the late 1980s and early 1990s. Lean manufacturing theories were highly influential in the creation of Lean development, one of the early experiments in Agile development techniques.

In recent years, these Lean ideas have influenced the world of startups, as well. In his bestseller The Lean Startup, Eric Ries has applied some of the principles of Lean, such as customer focus and simplicity, to the entrepreneurial world. The concept of Minimum Viable Product (MVP) was popularized by Ries and has since become a core element of the Agile philosophy. Delivering small, incremental features that are then evaluated by customers, it seems, is a good idea both in software delivery and in the establishment of a new business.

An example of a Lean principle that has transferred to Agile is the idea of work transparency. Agile advises teams to make their work items visible and easy to read, instead of hidden in project plans that require interpretation and commentary. Visual controls, such as product roadmaps, Scrum rooms with posted backlogs, Kanban boards, and burndown charts, are an integral element of Agile practice. Visibility drives the improvements we seek, by making both progress and impediments obvious to all. The visibility of the work the team has committed to, of work items as they move from “in progress” to “done,” and of the accelerating velocity of teams as they gain confidence and skill, enables any interested party to see with her own eyes the advancement of agility. Comparisons between projected and actual performance are no longer buried in a complex set of spreadsheets or obscure earned value calculations; the facts are right there in front of the team and its leaders.

The benefits of Agile at the team level, however, plateau quickly when team agility bumps into organizational inertia. As Agile productivity and quality benefits manifest, they often collide directly with the methods, practices, traditions, and culture of the enterprise. These frictions present the major challenges to the Agile transition. Agile practices, like Scrum, XP, or Kanban, are easy to learn and most teams can grasp the basics in a day or two. The art of transition is in the management of culture, tradition, personality, and expectation.

So, with all the just-described debate, disruption, and resistance, why are so many firms biting the bullet and taking on this difficult and potentially wrenching transformation? What are the business conditions that have driven firms, from the traditional banks and insurers to the web-born ecommerce and social media giants, to discard the traditional structures, roles, and practices, and embrace a subversive set of ideas that contradict everything we’ve learned in decades of industrial-age management?

The Imperative of Agile

Although short bursts of iterative work, customer feedback, and responding to market and customer demands might not be a panacea, they beat the alternative. We know that developing large, all-encompassing software products and releasing them in a single big bang causes problems. At the risk of sounding cliche, when it comes to IT projects, size matters. In 2015, Standish Group did a study seeking to answer a simple question; did the size of projects contribute to their success or failure? Commissioned by the Dutch government to determine why its IT projects kept failing, Standish analyzed failed and successful IT projects, controlling by size only, and made some simple comparisons. Their findings were illuminating:

• Large IT projects were successful only 13% of the time. Their results were considered highly valuable only 5% of the time and only 21% of their customers were satisfied.

• Small projects, on the other hand, seemed to fare much better: 57% were successful, and 26% were valuable. And satisfied customers grew to 49%.

Speaking of large projects, the Standish survey noted that “This segment has the highest rate of failure of all projects, with just a 6% chance the project will come in on time and within budget.”

It turns out, after decades of experience building IT projects, we’ve learned that incrementally delivering small, modular bits of capability, within a collaborative, corrective feedback loop, works better than multiyear omnibus projects that try to incorporate everything, and end up delivering little or nothing. We’ve learned that developing software, and in fact designing most modern products, is a creative, unique endeavor that requires experimentation, exploration, and commitment from both the development team and their customers. The traditional Waterfall process isn’t sustainable. Agile has grown to suit this new market reality.

Responding to Disruption and a New Marketplace

The recently published “Agility Is Within Reach” defined two goals that all enterprises seeking agility should focus on: strategic responsiveness and organizational flexibility. These two Agile objectives define the strategic thrust of agility, rather than its mechanics.

Disruption, enabled by innovative internet and mobile business models, stalks every business, from the global brand to the humble merchant. And disruption is just one of the risks emanating from our modern business environment. The exponential rate of change, with its accelerating product cycles, makes responsiveness essential. Consumers’ expectations are wildly divergent from those of a few years ago. Every transaction, from paying a bill to calling a cab, has been digitized. In the short life span of the commercial internet, the nature of our interactions, from commerce to romance, has irrevocably changed. The supercomputer in our pocket, connected to the global network in the airwaves, has enabled an online economy unrecognizable to a consumer from the year 2000.

“You can’t see big-bang disruption coming. You can’t stop it. And it will be keeping executives in every industry in a cold sweat for a long time to come…” So say Paul Nunes and Larry Downes in their Harvard Business Review article “Big Bang Disruption” (see also the book by the same name). Every business owner knows that there’s a young entrepreneur lurking somewhere, dissecting their business model and trying to figure out how to automate it, put it in the cloud, and take their market. The software market has evolved to an atomized world of apps, in which every niche has the attention of a startup.

At one of my recent Agile 101 classes with a team of project managers at a smartphone company, one participant told a definitive story of disruption. Her team was building prototypes of new phones, and near the end of the development cycle, Apple’s first iPhone was released. Prototypes just weeks from production were scrapped and the design team did the proverbial “back to the drawing board” pivot. The innovative technologies built into the iPhone went far beyond anything they’d been prototyping, and they realized that day that their market had changed massively and irrevocably.

Data Everywhere

The iPhone story is an example of digital disruption, but it also speaks to the exponential growth in the rate of change in technology. Product cycles compress tighter and tighter, queues form for the next version of a gadget immediately after its last release, and feature wars accelerate. Ray Kurzweil, well-known author and winner of the 1999 National Medal of Technology and Innovation, says of accelerating technical change:

An analysis of the history of technology shows that technological change is exponential, contrary to the common-sense “intuitive linear” view. So, we won’t experience 100 years of progress in the 21st century—it will be more like 20,000 years of progress (at today’s rate).

How’s that for disruption?

For consumers, one of the great benefits of the internet is price and product transparency, or the death of unequal information. Before the internet, the auto dealer had much more information about the price, lease structure, profit margin, and options than the buyer. The for-profit school had private information about its success rate, as did the hospital. That unequal information gave the seller a decisive advantage. Those days of information inequality are gone, thanks to the internet.

Car buyers can now go to TrueCar or a competitor to learn about the car they’re considering, to read reviews, and check invoice and local pricing. I can go to eBay and, in an instant, learn the going price for a guitar, a table, or a pair of vintage earrings. Amazon reviewers will tell me which books to stay away from. So much data is being generated that big data firms are springing up just to help businesses sort, categorize, and capitalize on these momentous data flows.

As commerce has moved online, customer relationships have evolved, as well. Before the internet, customer intimacy meant the extraordinary level of service, guidance, expertise, and hand-holding that companies provided. Customer intimacy was based on multiple personal relationships with clients through great service, outstanding support, and the human touch that cements relationships. In the time of big data, however, relationships are based on the streams of information that individuals and groups throw off, transaction by transaction. Amazon has no personal relationship with you, but it has intimate knowledge about your reading and shopping habits, demographics, and likelihood to respond to a product or promotion. From Facebook and Amazon to Google and Twitter, personal information is currency, driving ad revenue and product sales. So is the collective information, captured from millions of transactions and analyzed by sophisticated algorithms, which enables Netflix to make recommendations to individual subscribers. Internet-based utilities and merchants can trace every click, every purchase, and the demographics of every customer, even those just browsing. The web world of customer intimacy can be intrusive and invasive in some people’s eyes, but, in most cases, customers are volunteering this data gladly to gain the benefits of the technology.

Companies might have lost the advantage of unequal price and product information, but they’ve gained the advantage of aggregated transaction analysis. The examination of transactions and trends has evolved from a manual spreadsheet exercise to the new science of data analytics. By capturing and analyzing the streams of data that flow from every transaction, organizations can gain significant competitive advantage. Information flows can be analyzed immediately to inform decision making. Enterprises can use data analytics to understand the needs and desires of their marketplace and make small, incremental changes to products in order to be responsive. In fact, responsiveness to customer needs has overtaken sheer efficiency as a market differentiator.

The connection to agility is clear: the speed, change-readiness, and flexibility of Agile practices are the tools that enable quick pivots for businesses hoping to stay in sync with the customer. To benefit from these new capabilities, however, requires investment. Data must be reliably captured and retained, which requires investment in hardware and software. The use of cloud storage has made this less expensive, but the software and talent needed is still costly. Without Agile technical teams and data scientists to make sense of all this information, all we’re doing is adding bits and bytes.

The health care industry provides a perfect example of the benefits that accrue from the analysis of these data streams. The data in individual health records can be made anonymous and then analyzed to find trends, predict outbreaks and epidemics, and explore the results of treatments across various populations; and so potentially make diagnoses and save lives.

Retail is another example of a prime candidate for sophisticated data analysis, and in this industry, Zara is the exemplar. Zara is categorized as a “fast fashion” retailer, and it has upended the market with its responsive model. Women’s apparel, one of the most competitive niches in the world, has traditionally run on a well-known cadence, with high fashion houses previewing their latest trends in fall, lower price knockoffs of hot trends released a few months later, and mass market ready-to-wear fashions, based on successful trends, available 6 to 10 months later. This schedule had been traditional for decades. Then came Zara. Zara has the unheard-of turnaround of weeks, rather than months, to deliver fashions to stores, based on both haute couture and street trends. It uses a combination of technology, human interaction, and a sleek design chain to deliver to their worldwide network of stores. From store managers trained to initiate conversations with shoppers, to empowered teams of designers who can quickly change fabrics, design elements and colors, to an experimental “batch” model, in which small batches of new designs are tried out in various locations to test customer appeal; all of these elements make Zara the leader in fast fashion, and have made Amancio Ganoa, Zara’s owner and founder, one of the richest men in the world. Zara shows that agility doesn’t just relate to software, and is not limited to technology-focused enterprises.

The Samsungs and Apples of the world, whose entire value chain can be redirected for every new product, use their agility to keep other players off balance. Amazon can, and does, incrementally change both its business model and the customer experience. It even changes prices on the fly based on the purchaser’s history. These internet firms are responsive enterprises, using data analytics and customer intimacy to quickly give the market what it wants.

Becoming Customer Centric

In the entrepreneurial stage of business formation, customer-focused thinking is the norm. Every entrepreneur needs to understand their potential customers, their needs, and how they’ll respond to variables like pricing, convenience, and features. As the enterprise, grows, however, many firms become focused on the problems of production, scaling, hiring and managing, and lose some of their connection to the customer’s needs. A major element of Agile thinking is the reconnection with the customer, and the long-term maintenance of a customer-centric focus.

A lack of customer connection is a feature of Waterfall development approaches. Requirements were gathered in an initial phase and then the development team disappeared into their cubicles to write code and develop an entire solution, with neither the developers nor the customer very interested in maintaining daily interactions to ensure alignment. Development teams checked in with the customer, often perfunctorily, at the end of each phase, but they displayed little interest in modifying their design, and customers displayed little interest in becoming active participants in the development process. It’s not difficult to see how this combination of interests (or lack of them) developed into a flawed model for system development.

Customer centricity is built in to typical Agile frameworks, like Scrum. In Scrum, as in many Agile practices, one of the central principles is the involvement of the customer throughout the development process. Scrum includes the role of the product owner (PO), specifically designed to ensure that the development team stays in close contact with the customer’s expectations. The PO is responsible for developing the backlog, our Agile version of a project plan, in direct coordination with the ultimate customers of the product being built. The PO prioritizes the work, and ensures that the increments of features emerging from the team meet customer’s expectations.

Also built in to the Scrum process is the customer demo, or iteration review. In this practice, at the conclusion of each iteration, teams demonstrate to the customer what they’ve been building to ensure that the customer community has the chance to view their solution and validate that the team is on the right track. The ideal goal for this exercise is for the team to show actual working software, even if it’s only a small feature or function of the product. This ideal is not always achieved, especially in early iterations, as teams might be building prototypes or frameworks, but Agile teams always keep the ideal in mind. The important thing to remember about the demo is that customers have a regular opportunity for feedback so that they can evaluate both the functionality and the look and feel of the new product. This is one of the core values of Agile delivery; unlike Waterfall, in which the customer usually sees only the finished product (when it’s too late to adapt), Agile enables the customer and the development team to iteratively collaborate and respond to changes in the market or the customer’s expectations. Change-friendliness, versus change-resistance, is one of the hallmarks of the Agile mindset.

The Evolution of the Workplace

The atmosphere of the workplace has evolved significantly from the military-style, functional-line organization structure of the 20th century, and that evolution both drives and is driven by Lean and Agile ideas. Lean principles assume that front-line workers understand the company’s inner workings, can continuously improve their own work, and are self-motivated to achieve success and enhance their own mastery and craftsmanship. Lean teams believe that when the flow of work is visible and obvious to everyone, less time is spent in unproductive reporting, briefing, and providing status updates. Many office environments are now specifically engineered to ensure that teams work together; that their artifacts, like technical designs and task boards, are visible; and that progress is tracked publicly, in an easily-parsed format that casual viewers can grasp. New office environments are designed for this collaborative work style, in which teams sit (or stand) at their desks and design around a roving whiteboard, and then create visible artifacts documenting their team vision, goals, and iterative outcomes.

Agile has created its own momentum. Ranks of developers exert pressure from below. Those who have experienced agility bridle at reverting to traditional techniques, and those who haven’t are eager. The cadence of technology, the expectations of markets, the fickleness of customers, the evolving workplace; all increase the pace of change, putting more businesses at risk of disruption. It’s no wonder executives are now looking at agility as the next holy grail of management theories.

The Agile Decision Process as Strategy

In the ideal Agile world, transitioning organizations will have thought through their guiding vision of agility, phrased it in pithy and compelling language, and evangelized its benefits. In reality, most haven’t. They might see a version of an Agile future, but every participant sees it through a different lens. Executives might desire the benefits of agility without recognizing its inherent disruptiveness. Even if they have crafted a common vision, it’s often poorly composed and unpersuasive. And then, when they have a compelling message that they’ve agreed upon, they probably haven’t planned a promotion campaign to communicate and evangelize it.

Is moving to an Agile approach a decision, or are the drivers so compelling that it should be seen as an imperative? When we consider the strategic drivers we discussed earlier, it seems that all organizations must consider how to make their enterprise more responsive and flexible. Responsiveness means that every enterprise must be externally aware, always on the lookout for disruptive new business models and competitors. Faster product cycles mean that we must be constantly innovative, or we’ll lose customers to the next shiny object. In our dynamic market, roles, organizations, and even company missions need to evolve as the marketplace evolves. Rigid organizational cultures, with hierarchical structures and bureaucratic rules, must transition to more flexible, changeable models. The enterprise must stay engaged with its customers at all times, as their needs, expectations, and buying habits change more quickly than ever before.

Still, even if we accept Agile as an imperative, there are decisions that must be made. Consider the following:

• How Agile does the organization aspire to become?

• Are we driving toward agility at the strategic level, changing our strategic planning, budgeting and forecasting models from the top, or are we thinking of experimenting at the team level and seeing where that leads?

• Are we viewing Agile as simply a software process improvement, or do we visualize it as changing our entire mode of operations?

• Are innovation and creativity a key expectation of our transition, or are we focused more on financial and productivity metrics?

• If we intend to move to Agile incrementally through experimentation and exploration, which groups will we target for these trials, and what metrics are we using to determine their success?

From this list of questions, it should be clear that migration to Agile is a strategic decision, not a tactical one. Even when our organization intends to experiment and explore incrementally, selecting a few teams as our laboratory, there are complex decisions that we must make. If we accept the notion that, ultimately, the Agile transition will affect the entire organization, it’s incumbent on leaders to carefully consider the implications of the aforementioned decisions.

In the debate about “big bang” versus incremental Agile evolution, most experts are in the incrementalist camp. Agile evolution is too complex, even at the team level, to expect the enterprise to move from a predictive mindset, adopt a set of new practices, work through the initial bumps, and identify roadblocks and impediments in one fell swoop. Many teams and organizations have already tried Agile and failed, or have been through serial “change of the month” efforts, so, on top of the challenges, Agile champions often face a skeptical and weary atmosphere. Most organizations want to see improvement on a short time scale, and so part of transition work is to instill patience in sponsors and inform them of the realities of wading through the swamps of misperceptions and broken practices that can hamper the road to agility.

It’s imperative also that we are honest about the risks and challenges of an Agile transformation. Organizations that have never undertaken a disruptive change program like Agile evolution can believe that it’s merely a matter of training and coaching a couple of scrum teams. If that’s the ultimate goal, that’s fine, but if the firm believes that doing so equals enterprise agility, we’re setting the transition up for disappointment. On the other hand, organizations that have already experienced an attempted Agile migration that’s fallen on its face or “snapped back” to the prevailing culture will bring skepticism and doubt into the migration.

The strategic planning process for Agile process requires leadership. As Jim Highsmith notes in Adaptive Leadership, “This is one area where effective leaders lead—they help cut through the ambiguity and confusion of creating an effective vision.”

Whether we’re dealing with a single product owner trying to build a single scrum team or a leadership committee that is striving for full enterprise agility, we must bring the strategic clarity required to avoid spiraling into an endless semantic debate. Leadership is also required to ensure that the teams participating in visualization do so in a productive manner, with a strategic, enterprise-wide viewpoint, and avoid any parochial, siloed instincts that might have developed in their culture.

To make these strategic decisions wisely, let’s look at an Agile transition framework that will enable leaders and decision-makers to confidently proceed through the transition process.

The Agile Transition Roadmap

The transition to agility is itself an iterative project. Every enterprise discovers unique truths about its cultural, historical, and technical readiness to absorb Agile ideas. To prepare for a firm’s Agile journey, it’s necessary to understand the unique problems we’re trying to solve in that particular enterprise, the benefits we hope to gain, and the challenges and barriers that we might encounter. In my work as a consultant, I recommend to aspirants to agility that they do a thorough internal discovery process first, so that their transition is based on their particular reality and the goals and objectives are clearly articulated. As with any journey, after we’re underway, we’re sure to find challenges that were not apparent at the beginning. Successful transitions require focus, tenacity, and, most important, honesty. The transitioning enterprise must have the cultural openness to call out challenges when they arise, and work in good faith to resolve them for the sake of the Agile benefits to come.

A roadmap to agility is one of the transiting organization’s most important tools. Not only does it position us to set expectations and begin to outline time boxes, it also sets big, visible goals for our teams. When the outcome expected is team adoption, the roadmap reminds the team of its commitments, time boxes, and enterprise expectations. A definitive roadmap will remind the sponsors that guiding teams to successful adoption includes moving the enterprise, at least at the critical junctions, to understand, accept, and encourage Agile methods.

Roadmaps that don’t include any mention of enterprise obstacles create a false narrative, set the wrong expectations, and ignore the focus areas that will have the most impact on Agile success. Of course, we can’t know every obstacle upfront, and so roadmaps will necessarily evolve as we observe and experience. That’s a core concept of agility. We can, however, anticipate some of the more obvious impediments, such as reluctant project management offices (PMOs) and siloed functional teams, and ensure that we’re thoughtfully observing all the intersections so that we can foresee some of the traffic jams we’re likely to encounter.

Both Schwaber and Cohn advocate an Enterprise Transition Committee approach. An executive committee of Agile proponents willing to craft a vision and evangelize it is obviously an enormous boost. Building a small, cross-enterprise transition team positions the organization to have the honest, enterprise-level conversations that are necessary to craft a realistic strategy for this transition. This transition team owns the Agile migration, and must come to decisions and take actions depending on the particular needs and goals of the enterprise.

Mike Cohn in Succeeding with Agile recommends an adoption framework based on the acronym ADAPT:

• Awareness that the current process is not delivering acceptable results

• Desire to adopt Agile as a way to address current problems

• Ability to succeed with Agile

• Promotion of agility through sharing experiences so that we remember and others see our successes

• Transfer of the implications of using Agile throughout the company

Although there are plenty of other Agile adoption frameworks, including the Version One Roadmap and the Embracing Agile, the general path is quite similar. Each framework recommends the recognition that Waterfall is not serving the needs of the enterprise, strategic decisions regarding the organization’s goals and transition approach, the education, training, and coaching of participants, the promotion of successes to build momentum, and the scaling of Agile ideas and practices across the enterprise.

Successful Agile champions invest the energy in educating their sponsors and teams in the motivating factors behind the drive to Agile. As with any enterprise transition, a simple command from leadership can drive teams to adopt new techniques, but to sustain change, the organization must accept the guiding principles. Strong Agile champions will devise a robust strategy for demonstrating that traditional Waterfall methods have not served the enterprise well. Looking across the enterprise, the industry, and the entire corporate landscape for compelling case studies of failures and waste generated by adherence to Waterfall methods is a worthwhile exercise. So is creating a strategy to ensure that these examples are communicated and accepted by all layers of the organization.

Because organizational inertia is the major factor causing Agile adoption obstacles, if the management team isn’t ready for servant leadership, change will be very difficult. Other obstacles include a change-weary culture that’s skeptical about yet another new program, especially when the “way we always do things here” mindset has encouraged and rewarded bad habits. The recognition of existing problems is essential. Moving just one team to a new concept of work is a challenge. Moving the enterprise to a new mindset without the universal acceptance that change is required is Herculean.

Agile transitions can be sabotaged by unconvinced Waterfall adherents who passively refuse to adopt Agile methods, or who aggressively push back against them. Developing a convincing case for change is always the first hurdle in evolving any organization, and building this “burning platform” case is especially important with Agile, due to the entrenched interests, from PMOs to executives, who are comfortable with familiar artifacts and techniques.

How do organizations migrate from a top-down, hierarchical structure to an inclusive, servant leadership culture? With all the strategic and technological variables, all the personal and cultural sensitivities, and all the individual agendas that arise, getting to a consensus, putting it into persuasive language, and communicating it effectively can be a marathon. The challenge is to craft a concise, logical, positive, and persuasive vision that is aligned with the enterprise’s strategy. At enterprise scale, this is obviously an iterative, incremental exercise because the chance that an Agile vision with all of these attributes included the first time through is nil. If we do it right, we’re using the Agile practices we apply in product development to craft the strategic Agile vision.

Organizational inertia is weighty and won’t be moved by the issuance of an email. If our Agile vision is to motivate the enterprise, it must stand out from the reams of corporate proclamations, and inspire teams to action. Agile promotion will generate a range of responses, from cynicism of “another fine program” to enthusiasm from Agile advocates. The feedback is part of the point; it makes visible the fears, concerns, and hopes of the community, whether team or enterprise. In true Agile form, it exposes the weak spots, so we know what to prioritize. The community transitioning to agility will tell you how to succeed through your interactive campaign of communication and persuasion.

There are many decisions required to promote agility. Through what venues will we be communicating with our enterprise? Should we go beyond emails and surveys and create a Facebook page, a series of training videos, an executive presentation, or a “big room” work session? Are we asking for understanding, compliance, or feedback (or a bit of each)? Persuasive communication of an Agile vision is our chance to demonstrate executive support, evangelize the benefits, demonstrate the successes of our pilot programs, and build some momentum.

Transferring Responsiveness to the Enterprise

As we’ve noted earlier, customer responsiveness is a central goal of agility. It seems obvious that this objective reaches beyond the IT function to the entire enterprise. According to a report by Tata Consulting Services, the responsive enterprise can “shift rapidly to where customers want it to go next—the next buying experience they want, the new innovations they desire, or the new way they want to do business with your firm altogether.”

My favorite definition, however, comes from Responsive.org, a member-run organization that believes the following:

Responsive Organizations are built to learn and respond rapidly through the open flow of information; encouraging experimentation and learning on rapid cycles; and organizing as a network of employees, customers, and partners motivated by shared purpose.

The consistency with Agile and Lean principles is clear. We exchange information quickly and openly, without regard to rank or title; we fail fast in order to learn what works and what the marketplace values; we work as a collaborative network, which includes our customers, teammates, and partners, to deliver the most value to the ultimate consumer as quickly as is feasible.

Agile evolution within the enterprise is often visualized as ripples in a pond. The agile champion, coach, or consultant throws a stone into the pond by assembling the initial Agile teams and using Agile methods to deliver a pilot project. The pilot teams begins to experience improved results and make them visible. The successes and challenges of that initial pilot expose obstacles and broken processes. These discoveries cascade through the enterprise and focus improvement efforts on identified roadblocks. Incrementally, as each new challenge is uncovered, the ripples of change and agility reach into further corners of the organization, eventually washing across the entire enterprise and up to the executive level. Executives eventually acknowledge the power of agility, and change their mindsets and their management techniques to employ Agile practices across the enterprise. Lean, agile thinking permeates the organization. Productivity, innovation, and happiness multiply. The enterprise achieves the nirvana of agility (for more on this, see Chapter 9 of The Agile Consultant).

There are now many Agile scaling theories, from SAFe to LeSS and DAD to scrum-of-scrums. However, regardless of which enterprise scaling philosophy is applied, it won’t just touch some isolated groups of technicians in a small Agile pilot team. It’s imperative to evolve way beyond the “Agile 101” activities of guiding and coaching individual teams, and instead tackle the entire edifice of management style, philosophy, and organization. The enterprise’s entire structure is involved and managers and executives are being asked to migrate from the ingrained styles they’ve applied to a completely new set of beliefs and practices.

Every experienced Agile coach or consultant has seen the challenges many teams go through to adapt to agility. Project managers; client representatives; sales teams; operations teams; and developers, testers, and solution designers all initially struggle with their changing roles as agility evolves and they encounter unfamiliar territory. I’ve said before that Agile practices, like Scrum, can be taught in a day and begin showing results in a few sprints, but that doesn’t mean that Agile is easy. The deeper you go, the more difficult it becomes, and the evolution to enterprise agility must go broad and deep to generate the results the enterprise expects.

The most important work on Agile leadership comes, unsurprisingly, from Jim Highsmith, and here is an interpretation of some of his key ideas regarding the migration from Agile software development to the Agile enterprise.

As Highsmith reminds us, adaptive leadership is the work of energizing, empowering, and enabling teams to rapidly and reliably deliver business value by engaging customers and continuously learning and adapting to a continually changing environment. Focusing on value, rather than on tasks or activities, is a core Lean and Agile philosophy. Executives and managers who aspire to agility must also make the transition from managing and monitoring activity, to measuring the value delivered. The Triple Constraint, or Iron Triangle, taught in traditional project management programs, consisting of scope, time, and budget, is still important in Agile work, but it is viewed as a set of constraints and boundaries, not as the marker of success. All experienced project managers know that projects can deliver on scope, cost, and schedule and still deliver zero value to the customer.

It’s a central element of the enterprise Agile transition to help leaders understand this concept, and persuade them that it’s value, not activity, that requires measurement and oversight. Agile champions often encounter managers who continually refer back to the traditional management style, as in, “Agile doesn’t allow me to estimate like I’m used to,” or, “Agile doesn’t fit into my traditional strategic plan process like traditional project management.” They are presenting current conditions as if they are ideal, and it’s the Agile champion’s job to remind them that these techniques are not working. Otherwise we’re contending with a comparison between a perceived ideal and an unsure and unfamiliar future.

Even traditional strategic planning processes must evolve in the Agile enterprise. The assumptions behind traditional strategic planning, that the past was predictive, that markets were stable, that competitive differentiation was lasting, that enterprises could conceal their “secret sauce” and profit on it indefinitely—all of these core strategic ideas are now obsolete. Static, exclusive, and long-term strategic planning, like predictive project planning, is no longer relevant. Instead, firms must evolve to lean, agile strategies in the executive suite, and migrate to an adaptive, change-friendly, and participative planning model. As in project planning, strategy must stop attempting to predict from the past, and instead adapt to the reality of the ever-changing now.

It’s one thing to identify the challenges of the new, unstable marketplace, and quite another to devise a planning process that can give managers a window into what might be coming, without committing to speculative and unrealistic predictions. As agility is increasingly viewed as an enterprise-level initiative rather than simply a software process improvement, it behooves agilists to consider the ways in which strategic thinking must change to accommodate Lean, Agile practices and ideas. The application of Agile methods and practices, even for teams that have evolved to Agile strategic planning, is still “doing Agile.” Our teams and leaders might have learned and internalized the techniques and tools of agility, and they might be seeing results from their efforts. The enterprise is not truly Agile, however, until both the leaders and the ultimate customers have made the transition from “doing Agile” to “being Agile.” Agile theorists have addressed this issue of migrating to an Agile mindset across the organization, and thus enabling the enterprise to truly be Agile.

As with any transition that is about more than just following a set of rules, and requires a change in attitude, behavior, and perspective, it’s easier to learn the moves than it is to become adept. We’ve heard, from second grade on, that we need to learn the rules before we know enough to break them properly. Learning the rules, however, is necessary but not sufficient. To go beyond blindly following rules to the adaptive mindset required for enterprise agility, the enterprise must abandon traditional ideas and embody a different approach to leadership. Leaders and customers, and the teams that deliver value to them, need to go beyond the “know how” and internalize the “know why.”

Conclusion

The Agile enterprise needs to elevate its focus from just software development practices, and evolve leadership, marketing, and sales teams to understand that their customers are not only at the end of their value streams but also at the beginning, driving their product direction with their needs, desires, and expectations. They need to understand the difference between a push system, with its focus on ever-new methods of marketing and promotion, to a pull system, in which the focus is on responding to the market and partnering with the customer to understand their expectations and challenges. We must help sales teams learn to throttle commitments to fit capacity, a difficult chore when their compensation is based on selling more stuff. Finally, we need to convince leaders that the traditional metrics, like sales quotas and advertising impressions, must be combined with customer participation and feedback loops. These systemic organizational adaptations are as important as the Agile behaviors of delivery teams, and far more difficult to adopt. From product to business model innovation, the responsive enterprise is monitoring the signals from the entire environment, from internal teams all the way to the ultimate customer, and all the social channels in between. Responsive companies have the ability, from their IT systems to their supply chain, to respond to those indicators, innovate and develop new products, target unique niches, and stymie the competition.

Agile is cascading from information technology into sales, marketing, analysis, and leadership. It’s influenced everything from the design of office space to our relationships with customers and partners. The widespread adoption of Agile principles has the potential to help enterprises evolve into the responsive, flexible, and adaptive organizations necessary to compete in our turbulent markets. And Agile leadership has begun to replace the notion of an all-knowing leader, or a supreme project manager, with servant leadership, participative decision making, and self-organization. As Agile pushes against the constraints and boundaries of traditional management, strategy, and budgeting, the extension of agility to these realms seems inevitable. The mindset of agility, sustainability, collaboration, and value focus will transform our enterprises and interactions.