Have you ever tried anything a bit risky in your company: an idea that no-one had pushed before or a project that required people across the company to act in new ways? If you have, you probably felt you succeeded despite the system rather than because of it. Rather than help you in your endeavor, the people and processes surrounding you conspired to hold you back, and it was only through tenacity and resolve that you were able to prevail.

I have quite a lot of personal experience of how this happens and I have observed dozens of similar cases through my research. It is easy to conclude that the “system” in large organizations is broken. However, this is only half true. Large organizations do indeed suffer from acute limitations and bizarre pathologies, but they also have enormous strengths that we should not lose sight of. Being an effective manager requires a deep understanding of the limitations and strengths of the organization in which you are working.

Here is a cautionary tale from my own experience. I once agreed to do a Webinar for a big organization. I have done half-a-dozen Webinars before, typically for small companies trying to get people to pay for online content. In these cases, the process looks like this: they email me, I agree to do it, we arrange a date 2–3 months out, we do a quick technology check the week before, and then the event happens. It is all very easy, lecturing to an unseen audience in my own office. Frankly, I sometimes wonder how many people are actually listening. But that is a separate issue.

The process with this particular organization was rather different. The person I was talking to suggested a multispeaker event six months out. The other speakers and I all said yes. He then consulted with his board, who decided it was all a bit rushed, and couldn't be marketed properly, so we ended up delaying by a further six months. We all agreed.

A couple of months later I received the draft contract, 10 pages of detailed legalese, including clauses restricting my use of the presentation materials with other audiences, and requiring me to acknowledge them in all subsequent use of the materials. At which point it finally dawned on me what was going on – the Webinar was being treated like a book, one of those bulky paper things we used to take on holiday with us before the Kindle was invented. Once I had taken out the clauses that restricted my use of my own materials, the final version was then sent to me in hardcopy, by courier – one of those companies that was set up to distribute paper around the world before the Internet was invented.

The company had, in other words, taken its century-old process for selection and negotiating rights and applied it with little adjustment to these new digital and virtual offerings. My realization that this is what happened also explained the mystery around the six-month delay. Every publisher knows that books either get published in spring or fall; that is the way the publishing calendar works. So once our Webinar missed the spring window, it obviously had to be bumped to an autumn date.

I know I am being a bit harsh on the organization here. The people who work there are smart and knowledgeable and they understand these new technologies as well as I do. But they are being held hostage by their archaic management processes, processes that they themselves curse but are incapable of getting around. The net result, in this particular case, was a Webinar that cost a great deal to put on (in management time and legal fees) and with significant delays. When the event finally happened it went fine, but the size of the audience and the quality of the product were no better or worse than the low-cost, no-frills Webinars I had done before.

Understanding Bureaucracy – and Rising Above It

This experience reinforced for me just how inert and inflexible management processes are in most large organizations. A company can change its strategy, replace all its senior managers, outsource half its activities, and even get people thinking differently, but its core management processes – the way it allocates resources, evaluates people, negotiates contracts – will still be running under their own steam, just as they did decades earlier.

Management processes are the embodiment of an organization principle known as bureaucracy. We all know the word, and we often use it as a shorthand for everything that is bad about large organizations. Strangely, though, the original meaning of bureaucracy was entirely positive. German sociologist Max Weber brought the term into regular usage. He said organizations could coordinate activity in one of three ways: by using the traditional beliefs and norms of what had always been done (“traditional” domination); by relying on the personality of their leader (“charismatic” domination); or by developing a system of rules and procedures that transcended the idiosyncrasies of a particular person or situation (“legal” domination). This third model, which came to be known as bureaucracy, dominated the business world during the twentieth century, and was indeed very effective as a means of generating efficiency and creating outputs with consistent quality.

However, bureaucratic systems also have their limitations. They create a high level of alienation and disengagement among employees. They allow people to exercise their power in ways that subvert the organization's broader goals. They also breed management processes that take on a life of their own and never die out. At their worst, bureaucratic organizations are made up of unhappy employees, Machiavellian managers, and internally focused processes where no-one can see the forest for the trees.

What can we do to overcome these deep pathologies? One way forward is to attempt to make organizations less bureaucratic. If you have ever picked up a book by Tom Peters, Rosabeth Moss Kanter, or Gary Hamel, you will have seen plenty of advice on how to de-bureaucratize the business world, such as pushing greater front-line accountability, eliminating mid-management roles, encouraging spin-offs, and so on. These are all sensible suggestions but they don't “solve” the problems of bureaucracy, they simply keep them under control.

The other way forward, and my focus here, is for individuals to find more effective ways of working within this “broken” system. In other words, as a manager, you learn to accept that the system has certain limits, and you make sufficient changes within your sphere of influence to get things done. For example, in one research project I was involved in, the evidence showed clearly that the more complex the system, the more important it was for individuals to be multitasking, proactive types who were able to “rise above the nonsense” to make things happen.

So if the previous two chapters were about understanding and remedying our insensitivity to our employees, and our own personal biases, this chapter is about recognizing the limitations and pathol­ogies of the organization that surrounds us – and finding ways of coping. As suggested in Figure 6.1, bureaucracy has three primary pathologies.

Figure 6.1 The Problems with Bureaucracy and How to Cope

First, it results in alienation – a feeling of disillusionment or detachment on the part of the individual worker, when he is separated from the product of his labour. This was Karl Marx's insight, and of course it led him down a path towards class warfare and communism that proved to be a dead end¹. However, his original insight was spot on, and the need to find ways of helping workers to become more involved in their work and in the fruits of their efforts is as great as it has ever been.

Second, bureaucracy gives rise to political game-playing. The definitive insight here came from the French sociologist, Michel Crozier, who observed that so much of what happens in bureaucracies is decided in advance, that the only way people are able to gain some control over their working lives is to exploit “zones of uncertainty” where the outcomes are not already known². Thus, interactions in large organizations become a series of games and power struggles, in which people pursue their personal agendas, with little or no regard for their effect on the organization as a whole.

I doubt I need to convince you that big organizations are hotbeds of political maneuvring and game-playing. Some managers are simply self-interested, only contributing to a cause when it suits their immediate needs to do so. Others are schemers, taking delight in sabotaging the plans of others as a way of furthering their own agenda. Of course, not all managers act in these ways, but if even a minority do, it means everyone else has to adapt their behavior accordingly.

Third, bureaucracy allows inert procedures to emerge. As we saw with my Webinar story above, these processes take on a life of their own and are almost impossible to kill. They support the primary value-adding activities of the firm, but they are often two or three steps removed from the marketplace, so feedback from customers gets filtered out and lost. They have their own managers, who have strong vested interests in keeping their jobs. These processes are dependent on each other, creating a tightly woven matrix of activities that cannot easily be pulled apart. The budgeting process is perhaps the definitive example, a process that according to one book is “time-consuming, adds little value, and prevents managers responding quickly to changes in the business environment.³”

Most of the previous two chapters were about finding antidotes to alienation, for example, by getting inside the mind of our employees and looking for creative ways of making their work more fulfilling. So I won't give this issue any further consideration in this chapter. Instead, I will focus on the second and third pathologies. We can overcome political game-playing through personal initiative – by rising above the petty games of our colleagues and by coming up with creative ways of making a difference. We can also overcome our inert procedures by seeking out creative ways of challenging them or renewing them.

Take Personal Initiative – To Overcome Political Game-Playing

When something isn't working very well, you can take the path of least resistance and just put up with it or you can follow the path less travelled and try to improve it. This was the crossroads that Jordan Cohen, Director of Organizational Effectiveness at Pfizer, was standing at in 2005. He realized people were wasting a lot of time on low-value, run-of-the-mill work, like building spreadsheets, booking travel, and doing background research. Given the enormous cost pressure the company was under at the time, this seemed like an obvious area where greater efficiency was possible. “I noticed one of my team, Paul, an MIT graduate, spending time after hours doing tasks that were not just below his pay-grade but not core to his role either,” he recalls. “I knew there had to be a way to unburden him from this low-value work.⁴”

Pfizer wasn't an organization that encouraged bold moves. It was a leader in a highly profitable and highly regulated industry, and it exercised due caution at every turn.

Seeking to turn his observations about Paul into something tangible, he drew inspiration from Tom Friedman's The World is Flat, which opened up his eyes to the possibility of taking advantage of the global marketplace for services. He came to the realization that outsourcing could be done at a microlevel, around individual tasks. The pitch to his colleagues was: “I think I can help by delivering that to your inbox within 24 hours so you don't have to do it … it may cost a little bit of money but it will be just as good as if you did it” (note that he did not say, “I can outsource this for you” – he quickly learned that the term outsourcing had negative connotations). This resonated strongly across the company, and it gave him the impetus to turn his idea into an experiment.

Jordan first collected some data, and he was able to show that colleagues were spending 20–40% of their time on noncore work. He immersed himself in the practice of outsourcing, developing detailed knowledge of the types of services readily available, how to select and manage vendors, and how to ensure a high-quality service. By working evenings and weekends, Jordan was able to design a prototype program within just a few months – a service that colleagues could access at the click of a button, to secure one-stop support for such things as business research, data analysis, and creating documents.

Having a working prototype allowed Jordan's colleagues to start using the service and for him to learn how to make it work properly. As he recalls, “When I first started this, I didn't know what I was doing. By the time I finished, I was an expert. We weren't just looking at the output on a monthly or weekly basis, we were looking at it every night, and we were putting fixes in place immediately.” Plenty of mistakes were made – “I personally bought a lot of drinks for people, I apologized, that's what you do” – but within one year the service was operating in pilot form.

Moving from the pilot to a fully fledged operation took another year. Getting the name right was a challenge. Virtual assistance was Cohen's first proposal, but it sent the wrong message to the firm's secretaries who worried they were being replaced. Office of the Future became the working title, which gave way eventually to PfizerWorks. Cohen also had to get buy-in at every level, even meeting with the CEO to get the final endorsement to proceed.

Jordan didn't face any outright hostility in pursuing this initiative. In fact, no-one ever told him it was a bad idea, but there was a real risk of death by a thousand cuts. Some people were sceptical. Others were disinterested. Most had better things to do than help him. Faced with this mass indifference, he could easily have given up, but instead he chose to rise above it, and get PfizerWorks off the ground. So what were the tactics he adopted? What advice does he have for others who would like to transcend the narrow personal agendas of their colleagues?

Challenge The Rules – To Overcome Inert Procedures

Jordan Cohen's tale is one of patience and tenacity, of accepting the constraints of the system and working within and across them to get things done. All credit to him – many people would have simply given up somewhere along the way.

However, there is another way forward when faced with a wall of resistance to your well-intentioned initiatives: namely, to set about tearing the wall down, brick by brick if necessary. Of course, I am not going to recommend you pursue a sophisticated form of career suicide. Deliberately breaking the rules can work, but it can be very dangerous as well⁵. I recommend a more prudent approach. The starting point is to be very clear on your own degrees of freedom – your level of authority and the tolerance for change of those to whom you are accountable.

So here is an important question: Are you the boss? And when I say boss, I mean is there an area of business that you are fully accountable for? This could be an entire corporation or more likely a single business unit, function, or department. If you are, then you have the opportunity to “just do it.” Of course, there is still an issue of how you manage up, to keep those above you informed and on-board, but that shouldn't stop you from doing what needs to be done. Here are two examples of top-down approaches.

Making Your Experiments Stick

I spend a lot of my time working with managers on small-scale experiments of this sort. While they usually yield interesting and valuable findings, the reality is that most of them don't end up getting rolled out company-wide. For example, the expense-claim experiment at Roche was very well received internally, but at the time of writing it had not been implemented across the organization.

So I am often asked, doesn't this frustrate you? And doesn't it call into question the entire logic of experimentation? My answer has three parts.

First, management experiments that challenge the rules serve the important function of forestalling further bureaucratic creep. In my mind, bureaucracy is akin to a gravitational force, and the larger the organization, the stronger the force becomes. This means that large companies have to run as fast as they can just to stand still. They need managers throughout the system to be on their guard; questioning the way things are working, trying out alternatives, and pushing back on new rules and regulations.

Second, the discipline of designing and running a management experiment provides enormous personal learning. These experiments are often run under the auspices of a corporate leadership development program, and one of the key themes in such programs is typically closing the knowing–doing gap. It is useful to talk about this gap, to make people aware of it, but it is even more useful to do something about bridging it. Experimentation provides the bridge: it compels the managers involved to translate their abstract ideas into operational hypotheses and then into a series of discrete actions. These actions can then be scrutinized and evaluated against the original hypothesis. In my experience, this is a really effective way of enhancing the attentiveness and self-awareness of managers.

Third, management experiments are sometimes really successful. For example, the If insurance experiment that was discussed at the beginning of Chapter 5 led directly to a rethinking of the company's entire approach to coaching and mentoring. The experiments conducted by Chris Bayliss and Ross Smith above have both had an enduring impact on their organizations. Many other companies have also had good experiences with their management experiments. The best single source for reading up on these is the Management Innovation Exchange (MIX), a free online resource at www.managementexchange.org.

Of course, it should be no surprise that many management experiments don't seem to go anywhere. The essence of innovation in all walks of life is that most ideas fail. A pharmaceutical company needs to screen thousands of potential targets to identify dozens of candidate drugs, in order to yield a single viable new drug. Perhaps the ratios are a bit different in the field of management, but the principle that “failure” is a more likely outcome than “success” is one that we need to continuously remind ourselves of.

Making The Best of an Imperfect World

It is very easy to become frustrated by the apparent irrationality of the organizations in which we work. Some people yearn for more rules and procedures as a way of creating order and reducing the amount of time resolving disagreements. However, the more rules and procedures you create, the more depersonalized work becomes and the more introspective and self-defensive people become. To paraphrase Winston Churchill, bureaucracy is the worst organizing model in the world, except for all the others.

So what is the way forward? We can certainly try to identify alternative ways of coordinating our business activities, whether by using more market-like techniques or community-based principles of self-organizing. My previous book, Reinventing Management, had an entire chapter on this subject, but for the most part, we will all end up working for large organizations that operate in an imperfect way. Our job, as effective managers, is to develop the tactics and skills to function effectively in such a system.