Appendix F: Assessing the impact of portfolio management

Portfolio management can be a source of real value to the organization in terms of:

  Doing the ‘right’ projects – optimizing the contribution to strategic objectives subject to considerations of risk/achievability.

  Doing projects ‘right’ – delivery (on time and to budget) and realization of the anticipated benefits.

Achieving this should be more than a leap of faith; a commitment to assessing the impact of portfolio management has several benefits itself:

  It can help to demonstrate a compelling case for investment in portfolio management.

  By helping to identify what’s working and what’s not working, it helps in the ongoing development of more effective portfolio management practices.

  The process of measurement can help to ensure success – reflecting the management adage, ‘what gets measured gets done’.

Measuring success is, however, far from straightforward and consequently it is recommended that a suite of metrics is used to assess progress from more than one perspective, encompassing both quantitative and qualitative data and the dimensions of project and programme delivery on time and to budget; efficient and effective use of constrained resources; strategic coverage and balance; and strategic and operational impact achieved. These can be augmented by a relevant maturity framework such as P3M3. Although the actual metrics used will need to be tailored to the specific circumstances, the following list provides some sample metrics:

  Trend in balance of spend by portfolio category/segment.

  Savings in business case production – cost and time.

  Stage/phase gate and OGC Gateway red, amber, green (RAG) ratings on delivery confidence (see Figure 6.4).

  Post-implementation review ratings for benefits realization against forecast and achievement of value for money (actual benefits compared to actual cost).

  Proportion of initiatives rejected/stopped at each stage/phase gate and portfolio-level review.

  Speed with which initiatives progress through the development pipeline and the shape of this pipeline (it should be a ‘funnel’ not a ‘tunnel’).

  Proportion of portfolio in value terms (and by cost) invested in modular programmes and projects.

  Length of projects from inception to closure, e.g. % less than 6 months, 6 months to 1 year, 1 year to 18 months, >18 months.

  Scale (and trend) of reliance on external resources (contractors and consultants).

  Scale of unplanned delays due to resource constraints.

  Scale of unplanned delays due to ineffective dependency management.

  Key resource utilization rates.

  Amount of investment written off.

  Improved reputation for effective change management – assessment by auditors etc.

  Quantitative data on:

  Percentage of initiatives delivered on time compared with initial forecast.

  Percentage of initiatives delivered on budget compared with initial forecast.

  Percentage of benefits realized compared with initial forecast.

  Process maturity assessment – using, for example, the P3M3 model or the health check assessment at Appendix A of this guide.

  Number of portfolio process improvements recommended under the champion–challenger model and from post-implementation reviews.

  Stakeholder survey of views on the efficiency and effectiveness of the portfolio definition practices.

  Stakeholder survey of views on the efficiency and effectiveness of the portfolio delivery practices.