In 2003, the UK government instigated the National Programme for IT, a scheme designed to link over 30,000 GPs and 300 hospitals in the country. The project was budgeted at £6 billion and was to include, among other things, an online booking system, a centralized medical records system for 50 million patients, e-prescriptions and fast computer network links between NHS organizations. By 2006, two years into the project’s 10-year lifespan, a group of academics was calling for an independent review. A journalist at the time wrote:

“Technology is still seen as just another form of engineering. But IT systems are not like bridges – they are a tool, not an entity. ... Technology is a process, with no clear start, no clear end and ever-shifting goalposts. And as fast as IT itself evolves, the potential uses of it morph and multiply. There is no end date. The bridge is never built. But that does not mean it is a disaster. That is simply how it should be.”

computing.co.uk, 27 April 2006

By 2011, the project budget had ballooned to over £11 billion and a new government announced that it would be dismantled. The announcement, however, was more a PR exercise than an accurate description of what was happening. The programme’s architecture had changed; what initially had been conceived as an enormous top-down project was now a patchwork of different programmes, generated locally and then plugged together.

When a presented problem becomes ill structured, it ceases to be a puzzle. The vagueness or complexity of the problem causes stress. Hence the name I give to such problems: headaches.

A problem may be a headache for various reasons. We might lack the expertise to solve it: I can mend a puncture but I can’t change the clutch on my car. We might have been presented with a problem but not given clear goal conditions: how can your friendly IT consultant rebuild your network if you’re not sure how you want it to operate? We may not be able to judge a solution’s effectiveness without implementing it – by which time it may be too late (‘I think this improvised rope bridge will bear our weight; shall we try it out?’). And we may be overwhelmed by the sheer complexity of the problem: how do we manage climate change, or mediate between warring communities in a disputed territory?

Solving headaches: what styles work best?

Headaches lend themselves, at least to begin with, to the Explorer style. Headaches, by definition, are unsolvable; in order to tackle them, we need to transform them into another kind of problem. Explorer helps us find different ways of seeing the problem, giving us new possibilities of dealing with it. We shall probably need the other styles at some point, but Explorer is vital at the early stages.

In this chapter, we’ll look at the treatment options for headaches, in increasing order of effectiveness:

• Treat the presenting symptom.
• Deny that the headache exists.
• Transform the headache into a puzzle.
• Model the problem system.
• Recognize the problem as wicked.

Treat the presenting symptom

The first option is to act immediately to stop the pain. Take a pill; put a sticking plaster on the wound; shore up the collapsing wall; throw money into the gaping financial hole.

Obviously, treating the symptom may stave off disaster for a while, and it may relax the stress. No bad thing. But none of these solutions is likely to be lasting or satisfactory. Taking emergency action may give you thinking time; but you need to use that time wisely to find a better course of action.

In particular, don’t take emergency action and then ...

Deny that the headache exists

... which is, of course, what we do, all too often.

The obvious sign of denial is that we give up looking for a solution. We might distract ourselves with other activities. At work, we might just follow orders, do our job, and try not to get in trouble. Maybe the organization will fix the problem at a higher level, in a new version, or at the latest restructuring.

Another common symptom of denial is to assert that the problem is already solved. It’s much easier to do this if you fail to specify the problem clearly. An addict may claim that he doesn’t have a drink problem, because it might be hard to define the point at which alcohol addiction kicks in. A politician may claim that her policy has reduced the number of patients waiting for serious operations, by carefully failing to define the word ‘serious’.

Denial is an abnegation of responsibility. We are moving the problem out of our circle of influence into our circle of concern. We may then find ourselves engaging in the two behaviours typical of problems that sit in our circle of concern: blame or resistance (look back at Chapter 4 for more on these responses).

Sometimes it may be a very good idea to give up responsibility for a headache – or to give up some responsibility. This isn’t denial – it’s managing our responsibility productively.

If you decide you want to retain responsibility for a headache, you need to transform it because, as a headache, it’s almost certainly unsolvable. We can use the problem matrix to find ways of transforming the problem.

The alternative to treating the presenting symptom, or denying that the problem exists, is to transform the headache into a different type of problem.

Transform the headache into a puzzle

The aim here is to tame the problem: to reduce and simplify it to manageable proportions. And to do that, we need to improve the problem’s structure.

The aim of the transformation is to increase the clarity of the problem’s:

• initial conditions,
• goal conditions,
• operators,
• constraints.

One of the easiest ways to transform a headache into a puzzle is to look for an expert to help you. Solving puzzles, as we’ve seen, requires expertise and experience. Indeed, we employ experts – plumbers, electricians, financial advisers, IT consultants – precisely in order to turn our headaches into neat puzzles.

We can learn from the experts:

• Experts codify the problem-solving process. They use manuals, tried-and-tested procedures or regulated protocols. They pass examinations to show that they understand them.
• Experts concentrate on clarifying initial and goal conditions. They’ll work hard to define the problem state and the goal state as clearly as possible.
• Experts establish measures of success. They know that the solution to a puzzle must be testable. Often, they’ll test by measuring, and use a mechanical device to ensure that the measurement is objective and accurate.
• Experts can test a solution without risking catastrophe. Whether it’s checking that the aircraft is airworthy or testing a disaster-recovery system, experts design procedures that will test whether their solution is robust without endangering the system itself.

If we can do the same with some of our own headaches, we can successfully transform them into puzzles. We can become our own experts.

How to transform a headache into a puzzle

If you want to try transforming a headache into a puzzle yourself, try the following.

1. Tighten the problem definition. Limit the problem to a local, measurable set of variables. For example, if you’re trying to clean up a database, limit the number of variables by which you can cross-link items. If you want to reduce violence in a school, focus on measurable sub-problems: stop students carrying knives by installing metal detectors; monitor the number of physical assaults on teachers and look for ways to reduce them; record lessons and count the number of verbal assaults. Resist at all costs any tendency to let the problem definition expand or become vague.
2. Define a clear ‘stopping point’. How will you know that the problem is solved? Can you run a test that will demonstrate that your database is now complete, with no errors or repetitions? Will you stop your anti-violence campaign when assaults of teachers have been reduced to zero?
3. Limit the number of solutions. One way to limit the number of solutions is to set yourself a clear limit: for example, that you will consider only three or four. One of the most dramatic limits is to look for two opposing solutions: an ‘either/or’ choice that clarifies your thinking and can spur you to action. For example, you could tell yourself that either you will manage your diet more effectively, or you will put yourself at risk of contracting type 2 diabetes within five years.
4. Specify how you intend to measure success. With a limited number of variables to cross-link items in a database, you can check whether the database is performing to spec. If you count the number of knives brought into the school, and it goes down to zero, you can say that you have solved the problem.
5. Redefine the problem so that it resembles a previous, solved problem. Filter out, or ignore, information that complicates the picture. ‘It’s just like that problem. This solution worked then; it should work now.’ The success of the previous solution may convince you that it will work in this instance.

Turning a headache into a puzzle has many advantages. It brings the problem within your circle of influence, giving you a sense of control and the opportunity to use puzzle-solving tools and techniques. And that reduces stress. The satisfaction of seeing success materialize before your eyes will increase your sense of competence, giving you the courage to tackle further problems. Use this strategy to break down complex problems into manageable parts, or to extract puzzle elements from larger, intractable problems.

Figure 7.1 An Ishikawa diagram

Some huge problems are puzzles that may have headaches embodied in them. In 1961, John F. Kennedy set his nation a very well-structured problem: to put a man on the surface of the Moon and return him safely to Earth. Solving the problem involved large numbers of headaches, some with tragic consequences; but the main problem was clearly defined, with a limited number of solutions and clear measures of success.

The risks of taming a headache

Taming a headache carries some risks. Here are the most important.

A puzzle can still be hard to solve. Taming a headache may bring the problem under our control, but we shouldn’t assume that the resulting problem will be simple. Calculating the square root of 5734, finding the shortest route to a given destination, decommissioning a nuclear reactor – all of these are puzzles, but they have varying degrees of difficulty. The point about all three, however, is that, with the appropriate expertise and experience, almost anybody could solve them.

Limiting the problem definition may simply shift the problem. We may think we’re simplifying the problem, but we may be doing no more than allowing the real problem to scurry away and raise its head somewhere else. Trying to make our organization more efficient by, for example, reducing the number of staff, might simply transfer inefficiencies to new external suppliers, temporary staff or contractors.

Measurement can be a double-edged sword. ‘What gets measured gets done’: we often use that quotation, both to justify measuring and monitoring, and to criticize them. Metrics may give us objective indications that a solution is working; and they may tell us simply that something is being measured. We measure academic performance in schools principally by monitoring pass rates in examinations, but what do such measures tell us about the quality of teaching or learning? We measure many aspects of performance in our hospitals, but how much do such measures indicate that healthcare is improving – or that we are becoming healthier?

Albert Einstein reportedly had a sign on his office wall: ‘Not everything that counts can be counted, and not everything that can be counted counts.’ The aim of problem-solving is to change a situation for the better. And measurement alone can never do that.

The headache may not be like previous problems. Viewing a problem as a repeat of a past problem may be helpful, but if the problem is truly complex it will probably be more unlike previous problems than like them. Using past, successful solutions might then become counter-productive, or even damaging.

Model the problem system

Problem-modelling takes us a stage further in the task of transforming headaches into puzzles. The aim of modelling is to represent the system within which a headache arises. We can then discover points of intervention: places in the system where we can act to solve the problem.

The most important point about systems is that they are well structured. Identify the rules governing the system and you can predict its workings precisely.

For example, if I flick the switch in my living room, the light comes on. The switch is the input, my flicking it is the operation, and the light illuminating is the output.

Identify the variables in the system

To analyse a system, we first identify the variables in the system. We then observe what happens when each of those variables shifts. By repeated and close observation, we can work out the rules that govern the behaviour of the system: from input, through operation, to output.

For example, a central heating system will include variables such as the pressure of the fluid in the system, the amount of fuel heating the fluid, the number of radiators built into the system, and so on.

Modelling the system

To understand a system more easily, it helps to model it. The simplest models of problems are graphic – pictures or diagrams drawn on paper or a whiteboard:

• A flowchart models a process, from input through operation to output.
• A map models a piece of terrain.
• A tree chart might model an organizational structure or a set of decisions and their consequences.

Models can be three-dimensional: we can represent the problem of reducing fuel consumption in a vehicle, for example, by making a model of the vehicle and placing it in a wind tunnel, to study its aerodynamic performance. Models can also be mathematical or coded in computer language.

All models simplify complex situations; good models simplify usefully.

A good model adheres to the following rules:

• The model must be transparent. We must be able to inspect and understand the rules and principles the model is using.
• The model must be testable. It must work to inputs that we can define and determine, and it must yield outputs we can observe and measure.

The critical question when modelling a problem is to decide what kind of problem we want to model. The model of the problem will derive from the type of problem we choose: how we label it. If we label the problem ‘a manufacturing problem’, we will generate a model showing the flow of materials and information, the processes and controls used, evaluation of output quality, and so on. A financial problem might involve variables from the income statement, balance sheet or chart of accounts (sales, expenses, costs, assets, long-term and short-term debt, and so on).

The model we use depends on how we classify the problem.

Identifying the desired outputs

Once we’ve modelled the system, we can begin to ask what outputs we’re looking for. For example, if we turn up the central heating, the increase in heat emitted is the output.

The key with outputs is to identify how the existing outputs differ from our desired outputs. (A presented problem, you’ll recall, involves a gap between what is and what should be.) Existing outputs can differ from desired outputs in four ways:

1. What do you want to achieve? (What do you want that you don’t have?)
2. What do you want to preserve? (What do you have that you want to keep?)
3. What do you want to avoid? (What don’t you want to have?)
4. What do you want to eliminate? (What do you have that you don’t want?)

And, this being a system, we’re looking for outputs that are measurable. Only by measuring them can we see whether the output has changed in the way we want. (The model shown in Figure 7.2 derives from the work of Fred Nickols.)

Figure 7.2 Measurable outputs

Identifying the points of intervention and evaluation

The only way to change existing outputs into desired outputs is to intervene in the system. And because we want to know whether our intervention has been successful, we also need to identify the point where we can evaluate our intervention. Because we’re working with a system, change will be indirect: we’ll almost certainly have to intervene at one point in the system in order to change the outputs at a different point.

Think back to our central heating system. We want to make the house warmer. The points of intervention to achieve that increase are the thermostatic control, the taps on the radiators and the central controls on the boiler. Of course, if we think of the house as a system, other points of intervention become important: the doors and windows, insulation layers in the roof, and so on. The point of evaluation is a thermometer – often incorporated into the thermostat.

If we’ve modelled the system well, we’ll be able to create linkages between the point of intervention and the point of evaluation:

• With a simple system, we may be able to rely on a mental model: changing the fuse in the plug may result in the Christmas tree lights coming on, without the need to draw the system on a piece of paper.
• With more complex systems, a visual model becomes necessary. We need a set of drawings to help us assemble the flat-pack furniture, a circuit diagram to repair the hi-fi, or a map of a transport system to help us reach our destination.

Recognize the problem as wicked

Some headaches resist simplifying.

Consider these presented problems – all expressed as statements of what’s wrong:

• The team doesn’t work well together.
• Our road becomes dangerously congested when parents deliver or pick up their children from the local school.
• Our company has no clear strategy.
• We can’t integrate customer preferences with product profiles in our current data management system.
• Our city is plagued by sectarianism that occasionally erupts into violence.
• The charity we work for is about to be merged with another charity with a very different management culture.
• The town centre fills with young people indulging in anti-social behaviour after the shops close.

There’s a name for headaches of this kind. We call them wicked.

Wicked problems are the most ill structured of presented problems. They may have innumerable causes – or no identifiable cause. They’re tough to define; and they can’t be solved with a single course of action.

Simplifying a wicked problem into a puzzle may be counterproductive. You may find that your solution actually makes the problem worse by generating undesirable consequences. For example, managers may seek to improve a healthcare system by concentrating on a few measurable variables – waiting times in surgeries or numbers of patients awaiting operations – both of which could have serious repercussions for quality of care.

Wicked problems resist modelling. Managers may find it hard to model the increasingly complex environment in which their organization operates. (How can we model customer preferences, for example?) As a result, they may be confronted with problems that they can’t solve simply by gathering more data, analysing information or breaking a problem down into smaller, manageable problems.

Wicked problems have another defining characteristic: they involve people. Any problem involving a group of ‘stakeholders’ is likely to be wicked. The word ‘stakeholder’ is well chosen: when different people have a stake in a problem, they are likely to see the problem differently:

• Initial conditions: different people see the problem differently; they use different terms to define and describe the problem, even though they are all affected by it. Reaching agreement simply on what the problem is can be hugely difficult.
• Goal conditions: the people seeking to solve the problem may be different from the people who will be affected by the solution. Different people will demand different goals. Some may see the solution as another problem.
• Operators: different people will tackle the problem in different ways, especially if they are in different departments, different parts of a network, different professions or different cultures. Implementing a solution can become fraught with political peril.
• Constraints: different people will have different restrictions on their actions, different aims, priorities or values. They may also have conflicting ideas about what can or can’t be done, and what must or must not be done.

The term ‘wicked problem’ was coined in the 1970s by Horst Rittel and Melvin M. Webber, professors of design and urban planning at the University of California in Berkeley. Since then, the term has been used principally for large-scale problems faced by corporations, nation states or the international community. But many small-scale problems are wicked: tensions within a family or a work team; conflict in a neighbourhood; growing a small business. It’s not the size of the problem that makes it wicked, but its irreducible complexity.

The 10 characteristics of a wicked problem

As Rittel defined it, a wicked problem has 10 key characteristics:

1. There is no single, definitive formulation of a wicked problem. It’s not possible to reduce the problem to a single statement of what is wrong.
2. Every wicked problem can be considered a symptom of another problem. A wicked problem is inextricably entwined with other problems – an evolving set of interlocking demands and constraints. Rittel said: ‘One cannot understand the problem without knowing about its context.’
3. Every discrepancy in a wicked problem can be explained in different ways. If we were to ask all the stakeholders to do a ‘five whys’ analysis on a wicked problem, we would have completely different sequences of cause and effect.
4. You don’t understand the problem until you have developed a solution. The structured two-stage process of problem-solving – understand the problem and generate a solution – will not work with a wicked problem. Rittel wrote: ‘One cannot first understand, then solve.’ Instead, we have to combine the two stages: try a solution; test it against the problem; adjust the solution and reapply; and so on. Solving the problem becomes part of the process of understanding the problem.
5. Wicked problems have no stopping rule. Since there’s no single problem, there can be no single solution – only a solution that is provisionally good enough. Problem-solving will only end when we reach a solution that’s good enough for the moment, or when we run out of resources: time, money or energy.
6. Wicked problems do not have an exhaustive set of potential solutions. There may be no solutions; there may be a host of potential solutions. We may be able to generate a range of solutions, and we can be sure that there will be others that we will not think of. Which solutions are potentially valid? Which should we pursue? It’s a matter of creativity and personal judgement.
7. Solutions to wicked problems are not right or wrong. They are simply better or worse; good enough or not good enough. We can’t decide objectively whether the solution to a wicked problem is good or bad. We can evaluate solutions only in the social context of the stakeholders, all of whom are in some way competent to judge.
8. Every wicked problem is unique. Wicked problems include so many different factors, interlocking in such complex ways, that each problem is essentially novel. No two wicked problems are alike.
9. Every solution to a wicked problem is a ‘one-shot operation’. We can’t learn about a wicked problem without trying solutions, but because there is no opportunity to learn by trial and error, every attempt counts significantly. Solutions to puzzles can be easily tried and abandoned. Solutions to wicked problems are expensive and probably have expensive consequences – some of which may be irreversible. As Rittel said: ‘One cannot build a freeway to see how it works.’
10. The problem-solver has no right to be wrong. Any action we take to solve a wicked problem will have wide-ranging consequences. If we are dealing with a wicked problem, we shall be held liable for the consequences of our actions. We can’t avoid that responsibility.

First steps in tackling a wicked problem

Wicked problems can cause prolonged and painful stuckness. Whether the problem is personal or social, any solution to a wicked problem is likely to require radical and imaginative action to unstick our thinking. And, because a wicked problem involves a group of stakeholders, any solution is going to have to be collaborative.

Your aim must be to seek:

• shared understanding of the problem; and
• shared commitment to tackle it.

Seeking shared understanding of the problem

The first thing to do with a wicked problem is to discuss it. Bringing people together to discuss a complicated problem requires considerable skill. You’ll need to manage the conversation and the group’s thinking.

Building shared commitment

Commitment, as we saw in Chapter 4, is more than taking responsibility for a problem. Commitment is the highest level of problem ownership: at its most powerful, it generates a sense of unstoppable energy and concentration that we call the ‘flow’ state. How can we begin to generate ‘flow’ in a diverse group of stakeholders?

The problem matrix can help us. We’ve seen that one way to deal with a headache is to transform it into a puzzle. But two other transformations are possible: turning it into either a plan or a dream. To do either, you need to transform it from a presented problem into a constructed problem.

We saw earlier that presented problems and constructed problems are expressed differently. A presented problem is typically expressed as a statement of what’s wrong. A constructed problem is typically expressed as a phrase beginning with the words ‘how to’. The first move, then, in unsticking our thinking about a wicked problem is to express it as a constructed problem – as a ‘how to’.

‘How to’ unsticks our thinking in some remarkable ways:

• First, we begin to think forwards into the future. Instead of thinking about what’s wrong, we’re asking what we might do.
• Second, instead of considering one intractable problem, we’re considering multiple solutions.

We can use the ‘how to’ technique to involve different stakeholders in solving wicked problems. By focusing their minds on the future, on solutions and on multiple possibilities, the technique very quickly and simply focuses their minds on the issue at hand, dispelling blame and resistance and allowing different perspectives to be voiced.

The next chapter looks at the ‘how to’ technique in more detail. The first move remains the most important: to transform our perception of the problem so that it becomes either a plan or a dream – a constructed problem that we can feel committed to solving.