For real breakthroughs that push technology beyond the edge into unexplored regions, project teams are more likely to succeed by turning to the flexibility of independent action and the full force of individual creativity, unhampered by command-and-control. The trick is to bring out and capitalize on the inventive energy of independent thinkers, encouraging free exploration and individual initiative, to foster a kind of creative chaos that hovers on the supercharged edge of running completely amok, a sort of controlled insanity that breaks out of accepted modes of thinking and challenges assumptions about limits and possibilities. All these things run absolutely counter to the closed patterns of corporate pyramid power because they undermine authority and throw tradition to the winds—profoundly threatening to the traditional management mentality, but precisely what is needed to break new ground.

The key ingredients for a breakthrough initiative are, in fact, the exact antithesis of the tradition-bound hierarchical model. Instead of putting stability ahead of change, instability is promoted, becoming the driving force to overcome blindly accepted practices and unquestioned notions. Instead of putting corporate and collective interests above individual ones, individual freedom of expression and action come first. Where the traditional pyramid tries to rein in cantankerous coding “cowboys” and “cowgirls,” breakthrough teams love them and let them run free. This free-wheeling atmosphere stimulates creativity and tends to promote the “personal best” performances that can generate breakthroughs.

Breakthrough teams actually depend on individual initiative to coordinate their activities. Decisions are not centralized but are made independently, close to the action, by whomever encounters the problems and has the know-how to resolve them. What keeps such a group on course is a kind of friendly competition; what keeps them from running off in every direction at once is their common interest in and love of the game and their mutual respect for each other as players.

You are most likely to find this model operating in smaller high-tech companies, entrepreneurial start-ups, and the research and development divisions of larger organizations. They can be remarkably successful. Indeed, many a muscle-bound corporate behemoth relies on an undisciplined “corporate skunkworks” for the new ideas and products that keep the collective engine primed and running.

Consider this scenario in an archetypal Pacific coast company drawn from real experience. You enter the lobby to find the phones being handled by a bearded, middle-aged guy who turns out to be the veep for R&D. You tell him you are a GUI consultant and he waves you down the hall to the left. Ducking a barely subsonic frisbee that sails past, you try to find someone in charge. No luck. Recognizing a screen on a workstation in one office, you approach someone who is browsing through a familiar class library and turns out to be the receptionist. But conversation becomes increasingly difficult as the pace of the hallway frisbee game picks up. A couple of programmers, who are busy debugging work-arounds to an operating system problem, holler for quiet but end up getting drawn into the melee of flying discs. Soon the entire department has adjourned to the adjacent parking lot for a furious five-frisbee tournament.

To a casual observer, it may look like bedlam or kindergarten, but during this particular frisbee game a couple of programmers get inspired by the pattern in which the group keeps multiple frisbees flying without interference. They devise from it a novel solution to a nagging problem in the groupware that the department has been developing. Was it dumb luck? Were they just fooling around on the job? Impossible to tell in such a group. Work is play and play is work. Is the receptionist a software engineer? You never know.

And what about the V.P. of R&D, is he part of the support staff? He is if he's any good. Effective managers of this kind of creative chaos know that their real job is to provide resources and support, run interference for the group, and stay out of the way. The best such managers and team leaders are also likely to be techies themselves, basically one of the bunch, but particularly respected for their own programming prowess or other technical talents.

Of course, it doesn't always work, and even when it is working, inventive nuttiness can have its downside. It can be darned difficult holding a planning meeting or a code walkthrough when staff are always coming and going and those in the room may be preparing e-mail on a laptop or playing chess on a pocket board. And communications within a breakthrough team can be haphazard at best, even when everyone is in the same room. Not that these creative independents are uncommunicative, just that information can too easily get lost in the chaos. (“Oh, yeah, the client did message us last week about a change in the interface protocol. The note got filed someplace. I think.”)

The brilliant and decidedly independent psychologist David Kantor was among the first to observe that apparent randomness in human groups disguises an underlying, complexly patterned logic (Kantor and Lehr 1975). In this, his studies anticipated more recent work applying chaos theory to groups. Kantor recognized two variants of apparent randomness, one he called creative and the other chaotic, in the older and more traditional sense of chaos. The difference between “random creative” and “random chaotic” is crucial; it is the difference between success and failure in achieving a software breakthrough.

In extreme cases, a breakthrough team can become a breakdown team. Without the right ingredients, friendly competition can become unfriendly, unproductive, and even desperate. Would-be contributors work at cross purposes and fight for resources. Eventually, people may spin completely out of control, leaving the organization and project in shambles.

The first necessary ingredient to avoid chaotic collapse is good people, bolstered by good training and tools. Members of the team must have skills and abilities that are up to the challenge of the project and that are recognized by other members of the team. Mutual respect for technical competence and an implicit trust in the ability of other members to contribute to the effort are the essential glue holding the breakthrough team together.

The second necessary ingredient for success is sufficient, even abundant resources. The almost brownian motion of whole teams of innovators can lead to extraordinary solutions, but not typically with the greatest efficiency. New algorithms have to be tried out, novel data structures have to be implemented, and clever screen layouts have to be prototyped. The process of freely creating and trying out alternative ideas is essential; even dead ends that have to be abandoned are often an important part of the group learning process. Creativity comes at a cost and requires risk taking.

How do you lead a bunch of inventive independents? Not by taking charge. Leadership of breakthrough teams is a rather special role, radically different from management in traditional tactical teams. The most effective team and project leaders are peers who are highly respected as programmers and problem solvers, who are charismatic trend-setters to whom others naturally look. They lead by example rather than edict—which would almost certainly be resisted—and are able to foster an atmosphere of high mutual regard among team members. They are also good at getting needed resources—work stations, software, training, more time—for the team, keeping it well fueled and heading off any potential breakdown into counterproductive competition for scarce resources.

For success, breakthrough teams also need autonomy: freedom from interference, freedom to explore unanticipated angles and approaches. Good leaders of such teams say, “Let the games begin!” then stay out of the way. The best ones also make sure that nobody else gets in the way either.

A project team organized and managed for creative breakthrough can be a fun, interesting environment in which to work, but not everyone can work well in such a setting. People who need clear directions, well-defined goals, and straightforward expectations are likely to be more comfortable in a traditional hierarchy. The kinds of developers who perform best in breakthrough teams may be either artistic or intellectual, but they are sure to be seen as independent-minded. They are self-starters who don't wait for directions, but more than that, the best performers are also perversely persevering. Good tactical team members are often those who are especially responsive to direction from leaders, but good innovators are likely to be more individualistic, even resistant to authority and direction.

It probably would make little sense to turn an entire cadre of creative cowboys and cowgirls loose on a routine database application for the accounting department; the poor accountants might themselves be driven to stampede. On the other end of the scale, massive projects marked by countless components and complex interdependencies—the civilian space station software, for instance—are not the bailiwick of breakthrough teams, even when there is a clearly recognizable need for innovation. Teams of creative independents, for obvious reasons, fit better with smaller and simpler projects that do not require close coordination of too many highly interdependent parts.

The large-scale complexity of many modern software projects is better approached by other project models to be discussed in later chapters. In the meantime, the Nanomush Corporation and International Behemoth Management, Inc., types will have to continue to muddle their way through with their chaotic cowpokes and their megapyramids.

From Software Development, Volume 1, #5, May 1993.