3 Agile governance and organization

“When the government is simple and clear, people are simple and sincere.

When the government is complex and stringent, people are cunning and shall cause trouble. Lao Tzu”

I often use the analogy of Jekyll and Hyde to compare corporate governance with Agile practices. In Robert Louis Stevenson’s novella Dr Jekyll and Mr Hyde, Dr Jekyll is an upstanding individual who conforms to moral and social codes. Mr Hyde, his alter ego, has no such code and does what he wants. In the end, the conflict destroys the person.

Governance is often interpreted as providing a structured environment in which rigid procedures and gates are applied to give us the illusion of control. Agile, on the other hand, is often perceived as an unstructured, chaotic environment where there is no control. Neither of these perceptions needs to be true. When applied properly, the destructive ‘Jekyll and Hyde’ outcome can be avoided, and we can build an environment that takes the best from both worlds and consequently gives a powerful governance model.

3.1 What is governance?

Let us start with a clear definition of governance. Although there are many ways of describing it, I think it is something very simple. Governance helps us to ensure that:

• The right things are done:
  • Portfolio, programme and project planning and execution ensure that we do the most valuable things for the organization.
• Things are done right:
  • The capabilities delivered to the organization are fit for purpose.
  • Due care and attention are paid towards budgets.
  • Time constraints are taken into account.
  • Resource is utilized efficiently and effectively.
  • The required process and quality procedures are followed, and consequently statutory requirements are met.
• All stakeholders are satisfied:
  • Senior stakeholders understand the status and impact of the initiatives.
  • The delivered capabilities are fit for purpose and add value for the customer.

The reputation for corporate initiatives to overrun, or not deliver, has caused many governance processes to concentrate on the time and budget components. This has led to some complex procedures for budget approvals and controlling change.

Agile can help you to control budget overruns and delivery delays. Value – something real and tangible – is delivered incrementally. This assures us that initiatives are progressing and also provides some early indications of real timescales and budgets. Any potential overruns can be highlighted very early. Typically within Agile, however, time is fixed and the capability delivered is flexible. So normally we can ensure that something is delivered within the required timescale and budget.

The collaborative and iterative nature of Agile helps to ensure stakeholder satisfaction. All stakeholders are involved throughout, constantly reviewing features and keeping up to date with progress. The result should be a high-quality outcome. If it all works as expected, governance procedures can be minimal in this area as it is already integrated into the process. However, the theory is not always applied, and it will be necessary to have health check procedures to ensure the right level of stakeholder involvement.

Let us now examine some of the governance areas in more detail, and see how they are applied in Agile.

3.2 Meeting goals and remaining aligned with business strategy

Taking an Agile approach to initiatives requires that the teams assigned to make the idea a reality are empowered to do so. The teams will, however, be internal-facing, focusing on delivery of the outcomes expected from the initiative. Sometimes, because Agile initiatives create a large amount of enthusiasm and sense of purpose, the teams may get distracted from their goals.

Your Agile governance should include procedures to ensure that the initiative remains on track to achieve its ultimate goals and delivers early and frequently. Governance procedures also need to check the initiative’s alignment with current business strategy, which may differ from what was in place when the initiative started. Governance procedures can, for example, help to determine whether an initiative has already delivered sufficient value or no longer aligns with business strategy, and therefore should be stopped. These are hard decisions for those closely involved in an initiative, so the external governance influence is very valuable.

This is equally important for work sometimes classed as ‘business as usual’, as defined in section 2.5.

External governance procedures need to provide checks and balances, ensuring that such teams are still adding value in the wider business context.

3.3 Portfolio-level governance

In your organization, you will probably want to do more than current resources can accommodate. You will therefore have to decide which initiatives will provide most value to the organization, given the point in time when they are being evaluated. Linking the evaluation to the current business strategy makes obvious sense, but is often ignored; too often an initiative is undertaken because of who is asking for it. Initiatives should always be assessed in terms of the value they will add based on the current business strategy – this is at the heart of Agile thinking. Often, the only people with the required holistic view of the organization and the intended way forward are at a senior level – you are probably one of them. You have to be involved in deciding which initiatives go forward.

In many organizations, these portfolio decisions tend to be made once or twice a year, typically in line with the budgeting cycle. Often, approval is given for specific programmes or projects, and the budget is not available for anything else. Changing the priorities or adding new ones to the portfolio then becomes difficult, even when some of the initiatives once considered important are no longer so. A more flexible approach to budgeting and portfolio prioritization is required if we are to support Agile initiatives.

3.4 Budgeting

3.4.1 Overall budgeting

If you are a leader in your organization you will probably have to spend a lot of time at certain points in the year creating budget forecasts and justifying potential spend. These used to be frustrating activities for me as we were trying to predict and plan well into the future, often knowing that the year would turn out quite differently!

This is because budgets are often fixed at the start of the financial year. If we want to change them we have to work our way through complex procedures. Of course, you need to understand your commitments for the budgeting period, and equally due diligence needs to be paid with regard to proposed major expenditure. Frequently, this type of process results in lost opportunities as initiatives are decided on the basis of their relevance at budgeting time. You can only commit to budgets that are realistic. Some initiatives are approved; others are rejected based on arbitrary arguments and the need to make the budget fit.

In many organizations, previous budget performance is used to set new budgets. So areas of an organization may have their budgets cut simply because they did not spend everything the previous year. This often leads to inappropriate spending towards the end of the financial year just to use up budgets. This is not good for the organization, as it does not ensure that it is getting best value for money.

So how can we make this more Agile and flexible? Let us look at a model that provides adequate budgetary control while supporting a fast-moving, flexible culture.

3.4.1.1 An Agile model

Using the Agile principles of trust and empowerment, we can assign overall budgets to leaders of business areas, who can then decide which initiatives will be implemented using the budget, within the constraints of overall business strategy. We create the initial plans and prioritize the initiatives using methods such as the MoSCoW technique.

Using this approach, we include all initiatives in the plan, rather than eliminating them too early. We do not formally ‘approve’ any initiative; approval only comes just before we want to start the work – when the business case is sufficiently well formed to make real decisions about it.

The plan is reviewed and updated frequently throughout the year, taking into account the changing business imperatives. Some initiatives that were originally high priority may turn out to be of limited value and therefore can removed from, or reprioritized within, the portfolio. A review is also an opportunity to ensure that the initiatives are still worth doing in the current business climate. Overall budgets can be reviewed more often, reallocating money across operating units or increasing budgets if necessary or possible.

There is a potential problem with this approach, however. Changes in the business environment may mean that other business areas need initiatives earlier, and business areas can be very protective of their budgets. To ensure real flexibility, it should be possible to reassign budgets across the whole organization.

Some initiatives may be delivered incrementally; this means that they can be removed from the portfolio if, following an incremental delivery, sufficient benefit has been realized and there is no longer a good business case for expending more time, money or effort.

Table 3.1 shows some of the differences between the two approaches (reproduced courtesy of the DSDM Consortium).

3.4.2 Budgeting control within programmes and projects

Many organizations have so-called ‘gates’ defined within their projects, programmes, and other initiatives. The main reasons for the gates are to control budgets or resource, or as checkpoints in which the current status of the initiative can be assessed, and corrective action taken if necessary.

These processes are designed mainly for large, complex initiatives that may have large budgets, and are a tool for managing the risks within them. With Agile initiatives, the driving philosophy is to split complex problems into small, manageable chunks and to organize delivery and realization of benefits early and often. In many ways, this approach removes the risks of budget overspends or project overruns. In addition, traditional gates have developed for traditional approaches – for example, in the IT Waterfall method there are often gates after analysis, after design, after development, and after testing. This does not make sense for Agile initiatives, as all of the focus is on regular incremental delivery of features – in fact all of the Waterfall stages are happening all of the time.

3.4.2.1 Gates within the Agile process

Gates can still be useful in Agile projects and programmes. They can be used to check whether initiatives should be done at all, whether enough has already been done, or whether the next set of budgets should be assigned. In Figure 3.1, I use the DSDM Agile Project Framework to demonstrate some potential gates. This is explained in more detail in the Agile PMO Pocketbook. Other methods at the programme/project level described in Chapter 4 have similar concepts.

Risk assessment Based on the known state at the start of the financial year

This framework has a pre-project phase which determines whether the initiative is in line with business strategy. If not, it can be stopped at that point. ‘Feasibility’ and ‘Foundations’ examine the problem sufficiently to understand the benefits that will ensue compared with the costs and risks. These phases also break the problem down into increments, each of which will potentially deliver features that will lead to benefits. The end of these increments can include a gated review, if required, in which we can decide whether or not to proceed with the next increment.

This form of Agile fits well in organizations where gated procedures are required, but also provides protection from the ‘perpetual projects’ situation that could result from permanently assigned teams. Similar approaches exist at the Agile programme level.

3.5 Developing objectives and vision

In the early 1960s, John F Kennedy set a clear objective for going to the moon. In part of his speech he said:

“We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard; because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one we intend to win.”

The speech inspired a large number of people to make the vision a reality, and indeed the first moon landing took place in 1969, sadly after Kennedy’s death. The speech set a clear objective and the reasons behind it. Everyone involved in the initiative could refer back to it to ensure they were doing the right thing. They could also use it to communicate the vision to all stakeholders.

You may be concerned that such a vision will not exist if an Agile approach is taken. You may have been told that in Agile the team dives straight into the detail and everything emerges. Although this may have been the case in some early Agile approaches, the importance of defining the overall objectives and vision is now clearly understood.

The Agile approaches defined as project/programme management in Chapter 4 will have these steps built in. The difference is the detail and the length of time spent at this stage. Enough planning is done to understand the purpose and objectives; the detail will emerge as more becomes known. Enough is also done to be able to justify the initiatives based on potential benefit compared with risks and costs.

Once the teams have defined the overall roadmap, they operate within the Agile governance model. This empowers them sufficiently to steer their course to their destination, but the necessary controls are in place for them to understand when they are going off-course and need to be steered back.

3.5.1 Business cases

It is our duty as leaders and managers to ensure that we are doing what is important for our business. We always want to understand the costs, risks and value. We may decide to go ahead even if the initial business case looks a little shaky, but we know we have made an informed decision. So why should we expect anything less of our Agile initiatives?

Business cases are alive and well in Agile. The vision and high-level plans help to define an evolving business case. Following the Agile mindset, business cases are living entities that are refined and reviewed as often as possible. Initial business cases will be high level, but give enough information to understand that the initiative is worthwhile; detail is added as it emerges. At any point, the business case may no longer be viable and the initiative will be a candidate for termination.

3.6 Governance levels

As with any governance structure, it is important in Agile that all involved understand their roles – that is, their responsibilities, accountabilities and details of empowerment. Agile approaches, however, demand a faster pace of decision-making and people work best in an environment of collaboration and trust, where decisions are made at the point of impact. If you want Agile to work in your organization, you will need to address this.

The Agile decision-making model shown in Figure 3.2 may help. It also includes project and programme organizational models that provide the right level of governance while allowing the teams the freedom they need to be effective and innovative.

3.6.1 Agile decision-making

The Agile approach demands that the team carrying out the work has the power to make decisions about it. The team always includes both those creating the capability and those who will benefit from it. Decisions range from how the solution operates to what is included in it. This can be a culture shift for organizations with more of a command-and-control approach; they may perceive that decision-making is out of control in Agile, as Agile means trusting people to be accountable and responsible.

If the decision-making process is implemented well, however, all stakeholders will be happy with the outcome. It is not a free-for-all but a layered approach; the intent is to move decisions to the point of impact. This implies that some decisions will still require higher-level approval as those impacted include more senior stakeholders. In Agile, decisions should be made:

• At the lowest possible level
• As efficiently as possible
• At the latest responsible moment.

This advice is now examined in more detail.

3.6.1.1 Decisions made at the lowest possible level

In Figure 3.2 we see some of the categories of roles that we need for Agile governance to work effectively. The traditional hierarchical model is deliberately inverted to emphasize that the lower levels are empowered to make most of the decisions as long as they do not break the constraints described. The flows show how decisions are escalated. This applies equally to ongoing Agile initiatives (such as business as usual) and to projects and programmes.

The model assumes that decisions can be made at the lowest possible level unless some specific rules and constraints would be breached by doing so. The rules and constraints that apply must be suitable for the organization and facilitate the Agile process. For example, someone feeling they should make a decision is not a rule or a constraint. The decision should always be made at the point of impact. So decisions that may affect business strategy probably still need to be made by senior executives.

3.6.1.2 Decisions made as efficiently as possible

Decision-making should be as efficient as possible, particularly where decisions have to be made at a more senior level. This may mean changing existing processes to facilitate this. For example, decision-making where the decision-makers meet regularly but infrequently will not work well. Create an environment of ad hoc decision-making where you can assemble the relevant people quickly should the need arise. Consider using electronic decision-making tools to achieve this.

3.6.1.3 Decisions made at the latest responsible moment

Agile incorporates the concept of ‘just in time’, and this also applies to decision-making. To an extent it is true that the more information is available for making a decision, the better the decision. It therefore makes sense to delay decisions until the risks and consequences of not making them become too high. Conversely, decisions have to be made when the initiative cannot proceed without them – there is no excuse not to.

3.6.2 Portfolio decisions

In many Agile implementations, a senior business person, often called the product owner, makes decisions about priorities for products or capabilities within their business area. They often have dedicated Agile teams that develop new products or capabilities or enhance existing ones. Although this can be very effective, it is a limited view and does not take into account the wider business. Often, capabilities span several business areas so it is more complex to identify a single product owner.

At the portfolio level, it is important that a group exists with the role of examining products and capabilities throughout the organization and determining overall priorities. The group can consist of the product owners from the various business areas, and it can probably make most of the decisions. If we follow the ‘point of impact’ model, some decisions may still need to go to business strategy level, although the senior product owners will often operate at that level anyway.

The roles involved in the management of Agile initiatives are described in section 5.2.2.

3.7 Dealing with change

It is important to have the correct approach to change; otherwise Agile initiatives will be stifled or may spiral out of control. If you are a senior leader in the organization, you may not get involved in the fine detail of the processes but you should ensure that you have the required flexibility with the required control. It may therefore be relevant to understand what that means; it will depend on the formality applied to controlling change.

3.7.1 Formality of change

Many organizations have comprehensive and often bureaucratic procedures for handling change in initiatives. Let us first consider the high level. This can be equated to overall scope and budget. Sometimes it is referred to as the ‘breadth’. Since this level defines the overall rationale for an initiative, it is important to take time to reflect and reassess the business case if these are to change. There may also be other portfolio-level initiatives that are reliant on the agreed results of it. Consequently, senior stakeholders will probably need to be involved in deciding whether such changes are appropriate, and therefore some formal change control is advised. However, the process should not be unduly bureaucratic nor add unnecessary delays.

Most other changes, commonly known as changes to the ‘depth’, can be accommodated without formal procedures as long as the changes are being adequately documented. In fact, as the detail emerges it is normal for initial perceptions to change. The whole concept of iterative development relies on feedback requiring changes to initial designs.