The hybrid continuum
While many organizations develop project management practices that mix waterfall and agile, it’s important for businesses to understand where they are on the continuum and why.
If you are developing an innovative new product, the more agility your organization can embrace, the more competitive it will be. But for organizations that build power plants or ships, or develop medical technology that lives depend on, there are factors that make a plan-driven approach more appropriate for their projects.
Service life – When the outcomes of an organization’s projects have long service lives, plan-driven approaches tend to be the norm. Innovations in endeavors such as shipbuilding, bridge construction, office buildings, and the like do not evolve as rapidly as electronics or computer software.
Economies of scale – A large project that requires significant resources be procured and committed upfront points to a waterfall approach. The construction of a power plant, for example, requires large quantities of concrete and steel be readily available along with a large and reliable workforce. On the other hand, acquiring many developers for a software development project whose requirements are still evolving would be a potentially wasteful decision.
Level of risk – Organizations that need to minimize risks will find themselves operating on the waterfall end of the continuum. For example, the organization that is developing a medical device that impacts life safety must drive risk from the product design and project execution to the extent possible. This factor drives this organization to perform a great deal of planning and research early in the project life cycle to identify and manage risk.
Need for innovation –In contrast to scenarios aimed at minimizing risks, the more the organization can embrace risk and early failures in order to be competitive, the more agile it can be. This flexible approach to project management helps innovative businesses leapfrog competition by getting new versions of product out quickly (whether it’s software, hardware, or another physical product), which in turn allows them to implement customer feedback quickly in future iterations.
Organizational culture – Perhaps the most significant influence on an organization’s place on the project management continuum is organizational culture. Organizations that are large and bureaucratic tend to find it harder to use agile methods, as this requires quick decisions, direct engagement and feedback, and immediate pivots in response to inputs. More bureaucratic organizations tend to land closer to waterfall on the continuum. With these organizations, it can be difficult to complete quick releases or pivots in strategy when management layers and longer approval processes are part of the culture.
The point of discussing the continuum in agile hybrids is to recognize where your organization sits so that this can be considered in various project management decisions, whether project selection, methodology approach, or likelihood of successful adoption and transition to more agile methods. Organizational placement on the continuum does not necessarily dictate or preclude an approach or predict success or failure of agile implementation. Rather, the continuum helps the practitioner understand the practical realities in play when considering options and making project management decisions at a strategic or tactical level.
Organizations mix their project management strategies based on their endeavors, which is exactly the practical point. I’ve worked in large enterprise resource planning (ERP) programs in which the overall program was run from a highly plan-driven perspective with a project management office (PMO) in overall control. However, underneath the program level, waterfall and agile projects coexisted with their activities and metrics rolling up to the program layer for overall reporting. In these examples, the ecommerce projects I was responsible for used a highly agile approach underneath the plan-driven program structure.
Hybrid Examples
Hybrid models of project management combine elements of waterfall and agile or combine elements of two agile frameworks. Examples include “AgileFall,” “ScrumBan,” LeanBan,” and “FrAgile” (that’s friendly agile). For now, it’s important to note that many if not most organizations delivering projects work somewhere in the middle of the continuum noted earlier. It’s this middle ground where organizations leverage elements of plan-driven and agile practices that work for their particular culture, business, and scenarios for success on their projects.
Rather than attempt to follow rigid and prescriptive applications of project management methodologies that may not be wholly suited to their cultures or the nature of their business, these organizations are pragmatic and practical. They leverage experienced and well-trained practitioners to combine the most practical and applicable elements of methodologies and frameworks to just get stuff done. In these organizations, the methodology does not matter. As Jesse Fewell states, the methodology is far less important than adapting methodologies to suit the needs and realities of the project and the organization (Fewell, 2010).
The best way to begin learning and thinking about success with agile hybrids is to describe some examples of agile hybrids. One day, I went to do a search on the term “AgileFall.” I thought I had created a new term. When I got back thousands of search results for “AgileFall,” I realize that many practitioners had long been thinking what I’d been thinking: so many organizations take elements of traditional waterfall practices and combine them with elements of agile. As noted earlier, these hybrids have a lot to do with where the organization is on the continuum.
There are many hybrid examples combining elements of waterfall with agile, or various agile approaches with each other. We will look at a few examples to illustrate how this is done, why it works, and start thinking about how you can evaluate these for use in your organizations and on your projects.
AgileFall
Some agile methodology purists shudder in horror at the idea of AgileFall. Much is written about how to recognize AgileFall as a “bad” thing and what to do about it. The fact is, many organizations find this to be a sweet spot or a comfort zone. AgileFall (or as a former colleague of mine liked to call it “Wagile”) is all about balancing elements of agile’s flexibility, adaptability, and learning with some degree of predictive planning. There are benefits to both methodologies. Thoughtful and intentional use of elements from both approaches can result in outcomes that might not be as successful if one or the other approach was used in a rigid way.
Organizations working in verticals like consulting, custom product development, software development, and other verticals where some degree of scope definition is necessary can benefit from elements of waterfall in their overall agile approach. As a professional services leader working for the implementation arm of an ecommerce cloud software company, I needed to have some degree of sizing, scope, and requirements discussion with customers upfront. However, the project was defined with latitude built in knowing that as the customer saw deliverables after each sprint, their feedback and experience with the product would lead them to better understand what they really wanted.
This would lead to changes in configuration, deliverables, and completely new ideas for features and functionality. Therefore, an agile approach to the execution and delivery of the project was critical. The AgileFall approach can work well in scenarios like this. A key is forming a partnership with customers early and being very clear about the degree of certainty but also emphasizing how critical the agile and flexible approach with iterative learning and execution is to meeting and satisfying their ultimate requirement.
The approach I took to this AgileFall project and customer partnership was to do the requirements gathering and scoping with the customer to size the approximate duration of the engagement and project. This helped to determine the budget estimate and give the customer some certitude of what they should expect to spend. However, I was very clear in setting expectations with customers. For example, within eight 2-week sprints representing 16 weeks of work, each sprint would reveal changes and inspire customers to think of different ways they would want to adapt or customize the software. The AgileFall approach offered the flexibility to reprioritize features or to add sprints should the customer be interested in increasing their budget and project duration.
ScrumBan
ScrumBan is a hybrid of Scrum and Kanban. Scrum, as we will discuss later in this book, is one of the most widely known and used agile frameworks. In short, it uses predefined and recurring rules, roles, and processes to expedite the release of higher-quality products. One Scrum element is the use of time-boxed iterations or sprints during which teams commit and focus to complete a specified increment of work. There are prescribed meetings – the daily stand-up, the sprint review, and sprint planning meeting (Alexander, 2017). Scrum teams determine the work they will commit to in sprint planning meetings and focus exclusively on completing the work within the sprint, rarely allowing change within the sprint.
Kanban takes its name from the use of a card which is a component in just-in-time manufacturing and delivery. Kanban uses a visual framework to encourage continuous improvement and uses visual workflows to limit work in progress and match desired requirements to a team’s ability to deliver. Like Scrum, Kanban relies on self-organizing teams. There are few “formal” roles, and teams meet and change approach or process as needed. Unlike Scrum, Kanban does not generally use time-boxed iterations or sprints as part of the process (Alexander, 2017). Work-in-progress capability is the only limiting factor – there is no prescribed start and end to a period of work.
ScrumBan uses Scrum as an approach to perform the work itself and uses elements of Kanban to seek and realize continuous improvements. ScrumBan can help to maintain focus on managing work in progress, which is a core element of Kanban. While Scrum is often ideal for new product development and Kanban for manufacturing, ScrumBan is useful for maintenance projects and in service verticals where both new development and maintenance work are present (Pahuja, 2017).
ScrumBan combines the Scrum framework and processes with Kanban’s process improvement elements and pull process. While Scrum relies on the backlog to manage work and on burndown charts to visualize work completion, ScrumBan focuses on process optimization and smoothing the work-in-process queue. Teams use a board to track work, limit the backlog to a fixed size, and perform planning at regular intervals to prioritize and add items to refill a depleted backlog. Demos and retrospectives may also be performed at regular intervals but are not done at the end of prescribed sprints (Pahuja, 2017).
Estimation in ScrumBan involves planning units of work that fit into the team’s work-in-progress queue at approximately the same size, rather than the mix of user stories of varying sizes found in Scrum (more on this later in the book). Estimation is also done as/when needed as opposed to specific points (sprint planning). ScrumBan teams may be specialized (Scrum prefers cross-functional) and may meet daily. ScrumBan allows change to planned work in progress – this differs from Scrum, which generally locks down the sprint commitment for the duration of the sprint.
Savita Pahuja (2017) cites quality, just-in-time fact-finding and decisions, short lead times, continuous improvement, and process improvements as advantages of ScrumBan. Pahuja notes that ScrumBan looks like Scrum at the practice level but like Kanban at a cultural level which can make adoption less jarring and more of an evolution.
Waterfall Plan – Agile Execution
Many organizations seek the security of waterfall planning and a more deliberate approach to project initiation while also seeking the agility and opportunities for incremental and rapid value realization that exist with agile execution. These organizations recognize that there is value in planning, but that the plans themselves are less valuable and, in most projects, will change. They therefore embrace change by executing their planned projects using agile tactics.
The Project Management Institute (PMI) prescribes initiating and planning as the first two phases of what many call a waterfall approach to project management. Organizations that merge waterfall planning with agile execution may perform many project initiation and planning steps including team formation, project infrastructure, communication and stakeholder management planning, and some risk management work.
Organizations that combine these approaches well do not spend too much time trying to define requirements upfront. This is where recognition that projects change and embracing that change helps them realize value from agile execution. These organizations will go into sprints upon launch of their execution phase. They will continuously groom the backlog of original requirements and add or drop features and requirements as the evolving project and input from stakeholders dictate. Even though these organizations did significant planning upfront and may feel they have a good sense of the amount of work and duration of the project, they also recognize that as the project proceeds and they learn more about the output and how the team works, the project duration and overall amount of work required to deliver the desired scope will change.
Organizations using agile execution with waterfall planning can realize rapid value delivery. As the project proceeds through each sprint, the organization can evaluate team accomplishments and determine if completed work is ready to be released or deployed. Rather than waiting until the end to deliver value as with a waterfall execution approach, these organizations use the waterfall planning approach with agile execution to realize incremental value and potentially early project completion.
This approach can also be useful in organizations that are adopting agile but whose leadership may not yet be comfortable with a complete end-to-end agile approach. Retaining elements of waterfall planning helps to provide a degree of comfort to these senior leaders – they see the estimating and risk management elements as particularly useful during project initiation and planning, and also recognize the opportunities for rapid and incremental value realization available through agile execution.
Mixed Environments
Perhaps the most complex situation involving hybrids is an overall hybrid environment mixing waterfall, agile, and hybrid projects in the same project delivery environment within the organization. These organizations will run waterfall projects alongside of agile or hybrid agile projects. This high-level hybrid mixed environment requires significant maturity in project management practices and complete understanding and mastery of each of the methodologies and frameworks that are used.
One of the most challenging parts of the mixed environment is merging the metrics and in some cases dependencies across waterfall and agile projects. Project governance structures must be designed so as to process metrics from all types of projects and synthesize them into actionable information for project governance teams or governance boards.
Projects using different methodologies with cross-project dependencies must have visibility to each other’s plans and deliverables. The agile project may negotiate with the waterfall project to ensure that dependent deliverables will be available for sprints based on the respective project plans and release plans for each project.
The challenge in this environment is the likelihood that the waterfall project may have planned completion of a dependent deliverable for the agile team, and then fall behind schedule. The prescriptive plan-driven approach will not likely lend itself to agile replanning so as to deliver the dependent item to the agile team who may have included it in a particular sprint in their release planning.
The benefit here is that the agile team can indeed be agile and update their release plan to pull work from their backlog or restructure their release so as to accommodate the delayed dependent deliverable from the waterfall project.
Case Studies
Let’s examine some case studies that illustrate how hybrids and combinations of waterfall and agile methods helped organizations adapt and then adopt new ways of performing projects. In each case, the organization found that elements of agile combined with elements of waterfall were appropriate in some way to achieve their objectives.
Practical Desperation – Trying Agile
I was the project manager on a large, failed, plan-driven ecommerce system replatforming project at a biopharmaceutical company in the mid-2000s. It failed for a few key reasons, the main one being that the vendor underestimated the project and was focused on recovering to profitability through change requests, but also because our leadership had forced the vendor into lowballing their estimate. Fortunately, we had project budget management in place that preserved the majority of our budget when we ultimately parted ways with the vendor and brought the project back in house.
This scenario is not typically where a team would choose to start using some agile practices – the failure of the first edition of this project was highly visible and some of us were surprised to have kept our jobs. However, the project team knew that a plan-driven approach had failed on the first attempt, and that there was too much we did not know about the new technology we were implementing and how the complexity of some of our ecommerce processes would work on this new platform to attempt a plan-driven approach again. We dubbed the project “.NEXT” because we were implementing a Microsoft ecommerce system with a lot of custom development using newest elements of the .NET framework at the time.
The team had minimal formal training or experience with any agile practices, and yet we determined to use Feature-Driven Development (FDD) and to start doing daily stand-ups. We broke the project into its major component features and replanned based on the logical sequence of these feature sets. The iteration lengths varied based on the size of the feature set – some were as short as three weeks, others as long as eight. We determined to demo and release each feature set to end users as soon as it was completed, and we instilled a rigorous and disciplined build and test framework for the development team.
In short, we adopted what was realistic and practical for our team and project scenario at that time. Not everything about this project worked as planned, and we were still several months later than our initial projected completion, but our FDD approach enabled us to tell the business that we were learning as we worked through each feature set and to update our projected completion date after each release. Of equal importance to the team and the organization’s further adoption of agile was the learning that took place.
Once this project was completed, the team leveraged its learnings and transitioned into a two-week sprint and release cadence of feature enhancements that lasted for sixteen months. This also set the stage for the organization’s continued adoption of agile in incremental and practical ways, examples of which will appear throughout this book.
The Health Insurance Company
I recently spent a year consulting for a large health insurance company. This organization was highly plan-driven and had a large project and program management office (PPMO) with several leaders and many project managers. Their IT department was organized by IT function and was separate from the PPMO. The overall organization was very hierarchical and bureaucratic, and as such they were naturally toward the plan-driven end of the continuum. This organization struggled to get projects done – they approved and launched large technology-driven projects with a great deal of upfront planning on the assumption that massive requirements gathering endeavors would enable them to execute the software development work according to a plan determined by those requirements. These projects were then chronically late and underresourced in critical areas while overstaffed in others.
This organization had failed to recognize that their evolving reliance on software and technology required a move toward the agile end of the continuum. They had, practically speaking, become as much a technology company as a health insurance company. I worked with them as they considered and launched their first agile project and teams, and since I had also worked within their PPMO on some large waterfall projects, I had the perspective to see and recommend changes they needed to make in order to become more efficient and to recognize their evolution into more of a technology-reliant organization.
My recommendation to them after a year was to change their overall operating model and their project delivery model to become a more agile organization. I recommended they adopt some practical agile changes. I recommended that they leverage their experienced functional leaders as product owners and pair them with technology teams to identify opportunities to transform and eliminate operational bottlenecks through technology. I recommended that they change their PPMO model and instead embed their project managers with their business functional areas so that they became real business partners instead of project management as a service. Most importantly, I recommended rigor and discipline in considering and launching projects and ruthless prioritization in approving and executing them.
Shortly after I completed my assignment and made my recommendations, this health insurance company reorganized both its PPMO and IT department and hired a firm specializing in Agile transformations. They began the move that I had recommended and continue their move toward the agile side of the continuum.
SAP and Ecommerce in Biopharma and Cement
SAP is one of the largest and best-known ERP systems and is typically implemented using a prescribed, plan-driven approach. Organizations implementing SAP will typically hire a consulting partner with experience in SAP to plan and lead their implementation program. ERP implementation is a significant endeavor for any organization and certainly benefits from a plan-driven approach. However, even with a plan-driven program, organizations will find it practical and necessary to leverage agile practices for projects within the program. I experienced this on two SAP implementations – one for a midwestern US biopharma and the other for a large cement company based in Bangkok, Thailand.
In both instances, the organizations lived on the plan-driven end of the continuum, and their SAP implementation projects were certainly plan-driven endeavors and rightly so. Within these scenarios, the ecommerce programs that I led used, out of practical necessity, agile practices. Aside from the learnings that came from coordinating agile delivery within plan-driven programs, there were two big learning events that came out of each SAP program.
In the biopharma SAP implementation, the SAP program team began to gradually adopt various agile practices until by the final “playback” (the term given to the iterative attempts to test-drive the ERP system), the SAP project team was referring to elements of the playback as “sprints” and doing daily stand-ups in order to discuss the day’s plan and identify impediments. This shows how a heavily plan-driven program benefited from practical application of practices typically associated with agile projects.
In the cement company’s SAP implementation, their ecommerce system had (for various reasons) been an afterthought, so my software company’s system and implementation team were late additions to the overall program. The SAP program was being run by a PMO from one of the largest global consulting firms in the world, and they were using a program pattern that was familiar to me from my previous experience. Upon arrival in Bangkok, the jet-lagged project director and I were summoned to meet with the “CIA,” which caused us much consternation until we learned that “CIA” was a Thai acronym for their PMO.
The PMO insisted that we follow their program management approach and begin providing them with project artifacts that aligned with their overall program. My project director and I explained that due to the late addition of our product and team to their program and given that agile was our normal approach to rapid implementation, we would use our approach while aligning with their needs for reporting as best as possible. The success of this approach proved for me beyond all doubt that agile projects can function well within plan-driven programs.
Salesforce
Salesforce tends toward a “waterfall-ish” approach early in their release cycles before accelerating and transitioning to their agile practices as the release gets going (Ayers, 2016). This works because it is practical – the hybrid approach allows organizations like Salesforce to plan to a point where the level of risk and certainty present in the release is acceptable while still acknowledging the certainty of change while using an approach that can respond to change and provide the opportunity for rapid value realization – the hallmark of agile approaches to project management.
Summary
You’ve learned about the continuum and how most organizations find themselves and their project management methodologies somewhere on that continuum. You’ve learned a bit about what often influences where an organization and its projects land on that continuum. You’ve seen some examples of agile hybrids – AgileFall, ScrumBan, waterfall plan/agile execution, and environments where waterfall and agile methods are used concurrently.
The key takeaway from this opening chapter is that rather than using some “pure” version of agile, most organizations are using and succeeding with forms of agile hybrids – taking the best of waterfall and merging it with elements of agile to create methodologies that work for them and their organizations. In direct contradiction to some experts and practitioners who believe that projects and organizations will fail if they do not follow a pure approach, I contend that a practical approach and embracing a hybrid method that works for you and your organization is the best and most realistic way to go.
In the next chapter, we’ll look at ways to assess your own organization and your projects by revisiting the continuum. We’ll look more closely at the aspects of your organization, its structure and culture, and the type of projects that you do. This will help you understand where your organization and its projects are on the continuum.