1 Why should managers adopt an Agile approach?

“When a man is alive, he is soft and supple.

When he dies, the body becomes hard and stiff.

When a plant is alive, it is soft and flexible.

When it is dead, it becomes dry and brittle.

Therefore, hard and rigid shall lead to death.

Soft and gentle shall lead to life. Lao Tzu”

As a leader and manager, I feel that I need to understand a situation, concept or new tool before deciding whether it is right for my organization. This also helps me to decide on the approach to take to introduce it. With this in mind, I will spend this introductory chapter defining Agile.

There are a lot of misconceptions about Agile. For example, I have heard people say, ‘We are Agile – we use Kanban boards,’ but everything else is done as it has always been. You may also come across those I call ‘Agilistas’. They are dogmatic and claim that ‘If you’re not doing Agile our way, you’re not doing Agile.’ Given the values and mindset behind Agile, this is quite ironic.

Later, we will look at what Agile actually is, but let’s start by looking at the potential benefits.

1.1 The benefits of Agile

The 9th Annual State of Agile Report from VersionOne claims 94% of companies surveyed now use Agile in some form or another. This is an impressive figure, given that Agile as we know it started only in 2001. It means that Agile is now an established way of working and has gone beyond the ‘passing fad’ stage. This is further evidenced by the increasing number of Agile conferences. These are well attended events, and the audience has changed over the past few years from software developers to senior members of companies wanting to understand how to implement Agile in a sustainable way in their organizations.

So what has made Agile so popular? The following are some of the reasons senior managers want to adopt Agile.

1.1.1 Responsiveness to change

There is an increasing pace of change in all walks of life, but particularly in business. Businesses can no longer afford to ignore the changes happening or to react too slowly to them. The successful businesses of the future will be those that are highly responsive to change.

Small, empowered teams with defined goals are central to all Agile thinking. These teams can react to and embrace change quickly and redirect their efforts to the current business priorities.

The Agile approach also emphasizes delivery of value early and often. This means that the organization can realize benefits sooner, benefiting from otherwise lost opportunities and potentially beating competitors to the marketplace.

1.1.2 Better value for money

We all need to do more for less. A very competitive world market squeezes prices and demands that we keep costs down.

The Agile approach is to concentrate on the business need and on what will add real value. We focus on delivering the outcomes that add significant value to the organization. We reduce waste by, for example, not implementing business changes that were never required or add little value. It also ensures efficient use of all resources, such as money, people’s time and equipment.

The Agile approach can be applied at many levels of an organization. The prioritization techniques, for example, can be used at the portfolio level to determine exactly which high-level initiatives will add most value to the organization. Introducing a big initiative in small, frequent steps allows your organization to get most benefit as early as possible. You can also react quickly to change, moving resource and budgets to other initiatives if they become more urgent or represent better value for money.

1.1.3 Pleasing the customer

To remain competitive, all businesses need to please their customers and build relationships that can last well into the future. This is obviously true for our external customers, but a culture where internal parts of an organization can have a similar relationship with its internal customers can also result in a highly motivated, effective organization.

Agile has the customer at the heart of every initiative. It is customers who make the business decisions and constantly have the chance to provide feedback and redirect Agile initiatives within governance guidelines. They also benefit from high-quality outcomes delivered on time, enabling them to plan the inevitable business change activities to get the most value from what is delivered.

It is therefore no surprise that there is a high level of customer satisfaction with respect to both the outcomes and the initiatives that created them. Customers often feel real ownership of the solutions they are given and will promote them and their creators within their spheres of influence.

1.1.4 A collaborative culture

Traditionally, initiatives in organizations are undertaken with much trepidation. There is contention between the customer and the supplier. This often leads to a total breakdown of communication and trust, and the probability of being successful is greatly reduced.

Agile approaches demand the creation of cross-functional teams focused on the same goals. The customer is a central and key member, and everyone involved soon works in a collaborative way as they have the same goal and feel part of the same team. This is equally true of the other stakeholders represented in the team.

This collaborative atmosphere does not disappear when the initiative is over. Those who have worked in such teams develop relationships that go further than just that initiative; collaboration and mutual respect grow and start to permeate the organization. This can lead to a working environment that promotes innovation and removes conflict.

1.1.5 Higher-quality outcomes

Although cost and competitiveness are important, organizations are unlikely to succeed for long if they are not seen as producing quality products and offering a quality service.

In the past, products and solutions were created with little customer feedback until well into the process. In an Agile approach, customers are asked for their contributions and feedback throughout. This means that the solution delivered will really meet their requirements. Technical testing of a solution is also included from the start, test personnel being an integral part of the team and contributing to design discussions. In this way, both technical and functional product quality are built in throughout the Agile lifecycle. Potential problems are discovered early on and can be corrected before they become major.

As long as these principles are followed, the result is a high-quality outcome that really meets customer needs and adds value to the organization.

1.1.6 Better-motivated people

We all want our organizations to be full of motivated individuals. As well as making the organization a fun place to work and attracting the best people, motivation can lead to high performance and give the business a competitive edge.

Central to Agile are empowered, cross-functional teams working together to achieve a goal. Having such teams actually improves motivation. Once the team becomes a unit, it normally enjoys what it is doing. There may be disagreements and heated discussions, but generally the team is focused on the outcome and collaborates to achieve it. The final outcome is normally very positive, and therefore the team’s motivation remains high even at the end of the initiative, when it looks forward to the next one.

1.1.7 Better perception by customers

As Agile becomes embedded in an organization, the customer starts to get frequent, punctual deliveries of high-quality solutions that meet their requirements and add value. Inevitably, this changes the customer’s perception of the supplier. A relationship that was once strained and mistrustful becomes collaborative and trusting. In fact, the customer will start to promote the supplier (whether internal or external) throughout the organization and beyond.

As the customer is closely and continually involved in the initiatives, it will start to understand the stresses and strains of the supplier organization, and vice versa. This builds an environment of respect and trust and enables both sides to create future plans together.

1.1.8 Summary

Of course, there are other advantages to the Agile approach, including, for product-based companies, shorter time to market. A great advantage of Agile is the environment of transparency, trust and respect that it creates. People working in such an environment want to do the best job they can. As they focus on the goals, not on the politics, innovation becomes the norm, and delivering the right solution is guaranteed.

1.2 The Agile approach

Having looked at some of the benefits of adopting Agile, let us now understand what it means for us. Is it a tool, a method or a technique?

Although Agile implementations may include Kanban, Scrum, Lean or other methods and tools, these do not in themselves define an Agile approach. Agile is, foremost, a way of thinking and acting. The Agile Manifesto explains this to some extent; however, in my opinion, it does not give a complete answer, being limited to software development. In recent years, the Agile way of thinking has gone a lot further and now whole businesses are being run on Agile principles.

So what is the approach? In essence it is centred on people and their interactions, and delivering benefits to an organization early and often.

Central to the approach are a number of fundamental ways of thinking, working and interacting. It may come as no surprise to you that, as good managers and leaders, you have already introduced some of these ways into your organization. Whether this is the case or not, you have to take the following ideas seriously if you are to benefit from the Agile approach.

1.2.1 The future is uncertain

We have talked about businesses being responsive to change. The way we approach planning at all levels in our organization is a key part of enabling this.

Traditionally, throughout our organization, we have tried to plan well into the future, get plans agreed and then stick to them. We have also tried to predict what the organization might need in future. Budgeting activities, where we have to lay out what we will need for the year ahead and possibly longer, are an example of this. If we adopt an Agile mindset, we accept that this is not generally possible as business imperatives change quickly. The experience of talking about and using the capabilities we introduce into our organizations, or introducing new products and services into the marketplace, clarifies our understanding of what they actually need to do.

So does this mean we do not need to plan and we should just let everything evolve? No – taking this approach, we would not fulfil our responsibilities as leaders and managers.

Although we have to accept that the future is full of uncertainty and change, we still need to have a clear idea of what we want to achieve and why. We need to ask ourselves whether the benefits outweigh the potential costs, risks and disruption of making the journey. We need to have a clear vision, and then plan how to achieve it. Planning the whole journey as an outline is sufficient; however, we need to recognize that this is likely to change. As we embark on the each step, we can plan in more detail as requirements become more obvious.

1.2.2 Change is normal and inevitable

The pace of change in today’s business is phenomenal. Unless we can make our organizations responsive to change, we put them in danger of lagging behind, losing opportunities and business. Generally, we understand that some events that occur will change our status quo. These can be changes for the better or for worse. As human beings we are equipped to cope with and embrace change, learn from it and move forward. Charles Darwin taught us that those living things most responsive to change are more likely to survive.

Ironically, we often create complex bureaucratic procedures for change in our organizations. We see change as a disruption to be avoided or assessed in detail and minimized. We think we will fail to achieve our goals if we let change happen. In fact, we are more likely to fail if change doesn’t happen. Our organization will be out of date and will not meet today’s requirements. But if change is inevitable, why not embrace and plan for it? This is exactly what Agile asks us to do.

Budget and time constraints are still important. So if we want to use Agile to accommodate change, we must find a way to protect budgets and time, but still get what we need. In section 3.4, I will explore how this is possible within the Agile approach.

1.2.3 Provide capabilities early and often

Traditionally, it can take a long time for the benefits of change initiatives to become a reality and to start to add real value to our organizations. Could we have benefited if some of the change was delivered earlier? Could we have used our experience and feedback to influence future outcomes? We may also gain confidence that the initiative is working and delivering, instead of seeing it as a potential failure that is late, and over budget.

The Agile approach is to design our initiatives to provide the organization with capabilities incrementally, as early and often as possible. The organization then reaps the benefits of early use and becomes more and more confident of the value of the change and of the team making it happen. Of course, this implies we must continually prioritize what we are doing to ensure we optimize the value delivered into the organization. Prioritization is discussed in more detail in section 6.1.3.

1.2.4 Iterative feedback

How often have you been frustrated that something you have bought or commissioned does not do what you want it to? Unless we are people of exceptional vision, we find it difficult to explain what we need. We often don’t know until we see it, or until we have seen some examples. This can lead to real problems if we are trying to define our exact requirements years into the future. We confuse what we would like with what is needed to do the job, and we tend to include bells and whistles that seem like a good idea, but in reality are hardly ever of any use. The result is often something that doesn’t really meet our requirements, or in fact is almost unusable. Figure 1.1 shows an example of the usage of IT systems as presented in Chaos: A Recipe for Success (The Standish Group, 1999).

The Agile approach recognizes that it is impossible to define completely and in detail exactly what is required upfront. It is necessary to set the landscape – i.e. what the overall requirement is – but then to understand that the detail will emerge and change as more is known and as inevitable changes in the business environment affect the requirements.

With the best form of Agile, enough investigation is done initially to understand the problem and engage potential suppliers in discussions. Then iterative feedback techniques, such as Plan-Do-Check-Act (see Figure 1.2) are used to identify a solution that will really meet requirements.

The benefit of this approach is that each requirement is assured to meet customer needs and no time or money is lost trying to define something that is not known.

1.2.5 Small, motivated and empowered teams

Process-driven techniques are often applied to what is essentially innovation. Although this approach may work well for a production line or other repeatable tasks, it tends to stifle innovation and delay the process.

In process-driven techniques, individuals are responsible only for their part of the process. There is often little or no communication between the individuals involved in the different processes. If we apply this process to complex innovative situations, it can lead to ambiguities in understanding, and quite often the end product will not meet requirements.

The Agile approach is different. Small, multidisciplinary and multifunctional teams are created (see Figure 1.3), comprising representatives from those that will benefit from the solution as well as those that will create it. The team is given a specific goal and this sense of common purpose helps to unite it. The team builds a collaborative environment in which all team members understand that the goal is the priority and that they will contribute in any way they can to achieve it. The team is responsible for all aspects of achieving its goal; there are no points where the process moves to a different team.

So, to get the most from adopting an Agile approach, we will need to examine the structure of our organizations. In creating an Agile business, we may need to remove separate departmental ‘silos’, and consider creating multidisciplinary teams focused on specific products or goals.

1.2.6 Empowerment or decision-making close to the impact

The Agile belief is that teams work best when left alone to deliver their goals, making their own decisions and finding the solution. Teams are empowered to fulfil the goals they have been given. This will create high-performing, motivated teams. Of course, there may be times when specialists are required to perform some functions that the team cannot.

Clearly, the empowerment is not limitless; it is restricted to the fulfilment of the goal. It does not allow decisions that may impact others outside the team. The team must also consider the impact on other teams and determine when it will need to interact with them.

A small team (that could be in more than one location) communicates more easily with less confusion or misunderstanding. The fact that all disciplines are represented in the team means that any potential miscommunication between the customer and those developing the solution can be avoided.

You may be thinking, quite rightly, that small teams are all very well, but what about when the project or programme demands larger, more complex structures? I will discuss this in more detail in later chapters.

1.2.7 Continual and close involvement of the customer

The previous sections imply that those who will benefit from the initiative – the customers – are an integral part of the team and are making decisions on a daily basis towards the final outcomes. This places the onus of making business decisions on the customer, and helps to ensure that the right outcome will be delivered. It also means that the customer has to be available as required to participate in the process and give business knowledge and feedback. In section 2.3.2, I will discuss this in more detail.

1.2.8 Learning from experience

Throughout our lives we are learning, and this often comes from trying something new. The result of this is that we review and modify our future behaviour based our experience. Often, this natural learning cycle is forgotten when we are placed in a work environment. We carry out tasks, complete them, and start the next one without taking time to understand what went well or not so well.

The Agile approach embraces the learning process and incorporates the concept of continual improvement. Many Agile methods have specific techniques to facilitate the process. One example is the ‘retrospective’. A retrospective is a meeting that takes place after most significant events or deliveries and often at regular intervals. The team reviews what has been done, with the objective to learn and to change future behaviour, interactions and processes.

1.2.8.1 Fail fast

Uncertainty is one of the drivers within the Agile approach. In order to understand exactly what is required, Agile teaches us to try something early and get feedback. If it is wrong, we can modify it. This concept is known as ‘fail fast’.

A word of caution – fail fast means failing early so that you can change in time for it not to be a problem. This doesn’t mean, as one developer once told me, putting live something that is not fit for purpose: ‘We put the change live and the system failed – no-one could use it. But we had identified the problem and corrected it within an hour. We failed fast!’

In fact, Agile should never mean compromising quality.

1.3 Agile myths

Now that we have looked at the benefits and the Agile approach, you may be wondering what the catch is. As leaders and managers, we always need to approach change initiatives with our eyes open. Agile is obviously a good approach, but it is not a panacea. The following sections describe six of the myths I have come across in my Agile journey.

Myth 1 – Agile is faster

Some organizations have adopted Agile because they have heard it can be faster. Does this mean work can be done more quickly merely by adopting Agile? What about the quality of the work? Why should an individual suddenly work faster just because they are using Agile techniques? Does this imply they are taking more risks?

In fact, Agile demands very high quality and often reduces risk. The savings in time come from concentrating on things that add value and are really required. The same initiative can appear to go faster because we have removed a lot of the unnecessary ‘padding’.

Potential delays caused by communication and decision-making can be eliminated using Agile approaches. This is because the teams consist of stakeholders from all areas affected by, or benefiting from, the initiative. They can make decisions quickly and communicate well as they are often together, possibly even co-located.

The use of techniques such as Lean and Kanban can also identify and eliminate process delays in the way we do things, hence speeding up the Agile processes.

So Agile really can be faster, but only because waste in functionality, communication and process is eliminated.

Myth 2 – Agile is easy

On the surface, Agile can seem very simple and easy. A cross-functional team is assembled, it is given a goal and is empowered to do what is required to meet that goal. Short steps are then executed to produce an incremental set of quality outcomes.

In fact, good Agile requires discipline and control. It is important to get the following right when implementing Agile in your organizations:

• Each person must accept the personal accountability and responsibility that comes with empowerment.
• Agile initiatives are normally fast moving, and teams can get caught up in the enthusiasm of it – they need to be disciplined to keep their focus on the goals.
• The iterative techniques can lead some to believe that an initiative is closer to completion than it actually is, and this can cause a problem with customer expectations.
• Agile requires fast, efficient decision-making and commitment from all involved to reach the goals in the time defined.

Empowerment is powerful, but the limits of it need to be clear – otherwise teams may make decisions that are out of their control or not appropriate.

Myth 3 – Agile is a documentation- and governance-free zone

Having once been a software developer, I can see how lack of documentation could be seen as a benefit. Having also been a member of my organization’s leadership team, I see the opposite problem with this. How can we support something without any knowledge of it? How do we satisfy our auditors? How do we decide what to do, and who should do it? How do we communicate with senior stakeholders?

The use of Agile approaches does not imply that documentation and governance are not necessary; rather that they should add value to the process. Chapter 4 describes the various Agile methods. Those in the ‘full project/programme’ category provide good support for Agile governance, suggestions for documentation and its use.

Myth 4 – Agile is a planning-free zone

I have heard anecdotes of customers being told statements such as, ‘We can’t tell you how long it will take. The requirements will emerge as we do it.’ Or, ‘In Agile you just dive in and do it.’ Perhaps you have been told the same. Whenever I have done either of these things the result has been disastrous, costly, or both.

Although it is true that we have to let the detail emerge, we still need to understand the problem sufficiently to be able to plan. We must determine that the projected benefits outweigh the costs. Otherwise, Agile or not, we should not do it. Also, if we don’t plan, how can we know how to deliver incrementally?

Planning is vital in Agile, but only to a level of detail that is appropriate for the period we are considering. In general, the further into the future we plan, the less detailed our plans are and we accept that they may change. As we embark on a specific, shorter timeframe, we plan in more detail. There is often more planning in Agile than in more traditional environments – the difference is that planning is integrated throughout.

Myth 5 – Agile does not need managers

By implementing Agile, do you make management redundant? Of course not. In many early Agile implementations, the team was seen as all that was needed to deliver a solution. Anything else was seen as unnecessary interference – it would either slow the team down or undermine its empowerment.

This may be true for a very small piece of work in which there are no complex interfaces or politics. As we know, however, this is rarely true. Normally, the small piece of work is part of a larger initiative (see Figure 1.4).

There are many areas that may be affected, or need amendment to be able to reap the real benefits of what we are doing. These can include business processes, organizational structures and company infrastructure. Senior stakeholders also need to be kept informed. Having multiple suppliers, locations and customers can complicate the picture.

Good project management has always been about dealing with all of these different aspects. So please don’t sack all your project managers, or yourselves!

Project and programme managers need to change their approach – command-and-control or micro-management will not work.

Myth 6 – Agile never fails

In theory, Agile initiatives deliver new capabilities early and often. These capabilities are tangible and can either be used in the live business, or form part of something that will. We can class them as ‘done’, meaning that no more work is required on them. Since they are real and not merely documents, models, architects’ drawings or other specifications, we can get a good impression of progress. We are also provided with early signs of problems. If we also take into account the constant involvement of the customer, ensuring that the solution will really meet their requirements, it becomes obvious that taking an Agile approach can be less risky than more traditional methods.

However, Agile initiatives can and do fail and there are some spectacular examples. Some reasons are:

• Company philosophy or culture in conflict with an Agile approach
• Unwillingness to change, leading to pressure to follow more traditional processes
• Organizational or communication problems
• Lack of sufficient training or coaching
• Lack of experience with Agile methods
• Unwillingness of the team to follow Agile
• Lack of management support.

Failures are generally caused by people. Starting with yourself, ensure that everyone understands the risks associated with Agile and put in adequate risk management processes; this way the probability of failure can be minimized. We will examine the risks in more detail in section 5.3.

1.4 Agile approaches – what’s different?

I haven’t yet compared Agile with other ways of working. Ten years ago there was a lot of scepticism about Agile as opposed to more traditional techniques, such as the Waterfall process in software development. Now, however, the distinctions between Agile and Waterfall approaches are very clear (see Appendix A).