Quality has become a watchword of the hour in software development. Some of the concern with quality is genuine, and some is merely panicky chief executives lunging after the bandwagon rolling by; some “quality programs” will make a difference, and some will only make for good public relations. As issues of software quality come to the fore, matters of maturity will loom larger, because an effective commitment to quality requires maturity from organizations and individuals.
Managers of software development face many new challenges as software systems become larger and more complex. Organizations are moving toward greater “process maturity” using more disciplined and sophisticated models of software development. The question in the minds of many managers is whether development professionals will live up to their name and mature with the methods. Programmers, analysts, and designers are often a breed unto themselves, and managing an entire herd of such mavericks can tax the resources of even the best managers. The problems of dealing with diehard independents who would rather quit than collaborate are major issues facing software development groups today.
Even with the surplus of loners in the field, most software is produced by groups of people working together. Some of it is produced by groups of people working separately. Some is even produced by groups of people working against each other. Very little software is developed by individuals working alone.
This obvious, simple, and seemingly inconsequential fact would not be worth stating were it not for the way in which the field of software development is dominated by myths of giants, by a mythology that glorifies the brilliant genius who single-handedly conceives and codes clever new systems in sweaty and sleepless weekends of nonstop programming. We are blessed and cursed by the promethean images of these nerdy pioneers bringing us new languages, new tools, and new paradigms for computer applications. And we are blessed and cursed by a larger cadre of lesser godlings, programmers of more limited talent but nevertheless indomitable spirit, determined individualists who insist on doing things their way, alone, without interference, without help, without the hindrances of supervision, methodology, or discussion.
Management and organizational consultants often refer to such people as “cowboys.” Cowboys, the last of the rugged and untamed, are found in various fields, but nowadays many of them are punching assembly language cattle on the silicon frontier. They have also been called “human cougars,” for their solitary and sometimes wary ways. They are the mavericks who either go it their own way or not at all.
In case there are any doubts, it is important to make clear that I am a maverick myself, or so I have been told. In fact, I have been officially branded a maverick by methodologist-become-historian Paul Ward in his history of structured analysis (Ward 1992). He has personally assured me that it is to be taken as a compliment. I certainly have been a nonconformist most of my professional life. Modern software engineering may have embraced many of the basic principles of structured design, with CASE tools and integrated development environments enshrining data flow diagrams and structure charts as the technical iconography of development orthodoxy, but such was not always the case. Difficult though it may be to believe, these were once the unorthodox, even radical ravings of a maverick.
I believe in mavericks; many of my best friends are mavericks. And I believe in the maverick imagination, in individual creativity as the well-spring of nearly all genuine innovation. I also recognize it as the well-spring of much monumental lunacy. For every Einstein there are a gaggle of Velikovskys. Sometimes the innovation and idiocy even spout from the same font, as from a Tesla or a Wilhelm Reich. One way or another, mavericks have enriched our lives, if not always by invention then by entertainment.
Being a cowboy is not the same as being independent or an individualist. The definition does not hinge on whether or not someone uses a particular software methodology or even any methodology at all. Being a cowboy is a frame of mind and a style of life. Cowboy coders are simply those oppositional developers who can't abide being fenced in by standards, constraints, or discipline, who resist all efforts at being reined in by supervision or collaboration with others, who put idiosyncratic originality above usability or reliability.
Of course, not every programmer who prefers to work alone is a coding cowboy. Some people are loners by temperament. Some are perhaps better described as hermits, others just prefer their own company, and many simply find the company of others distracting or even overwhelming. Some of my best work has been done with no one in the room but me and my trusty computer.
Why is it important to manage mavericks? Why not merely turn them loose, let them join their cowboy companions as contract programmers and independent consultants? For one thing, there are so many of them. For another, a lot of them are good, and potentially could be a heck of a lot better if their creative contrariness could be tempered by a little more cooperation. Mavericks and coding cowboys have a real contribution to make to software development. Good management creates a context for capturing and utilizing this contribution for the benefit of both the organization and the cowboys.
Some managers advocate a laissez faire approach to coding cowboys. Expecting discipline or imposing standards just keeps programmers from reaching their full potential as brilliant programming cowboys.
I think managers have better options—at least I hope so. The free-rein approach probably accounts for a lot of the poor performance and unreliability of the products being shipped by many such major software companies. It shows also in the user interfaces of major software products that suffer from creeping featurism and are covered with a hodge-podge of unreconciled hooks and handles contributed by each and every member of the old programming posse.
As the size and complexity of software products grows, it is more important for project managers to learn how to use the isolated developer wisely. Even where the dominant project model is collaborative teamwork, the best managers will find ways to accommodate the needs of loners. It may not be possible to allow them to work in complete isolation, but it also may not be necessary to make them attend every meeting or code walkthrough.
Loners can do good work and they can do poor work. It is up to the manager to find ways to utilize them so that they do good work. Part of the trick is in how the work is broken down and assigned. Those who work best in isolation need to be given tasks that are more or less separable, portions of the system that can be defined as black boxes with simple interfaces and well-defined external specifications. They need to work on components that are only weakly coupled with other parts of the system, that do not require close coordination or frequent communication with others.
Another part of the trick is careful monitoring and review. Limiting the interdependencies between independents and the rest of the development team does not mean ignoring them. In fact, when part of a system is developed separately, it is even more important to monitor the interfaces and interconnections that remain. Managers often understand this when working with outside contractors or telecommuters who work at home, but tend to forget it when dealing with the independent who sits in the office down the hall.
When work is done openly in groups, increased visibility tends to reduce defects and increase quality (See Chapters 26 and 33). Work completed in some degree of isolation from a development group therefore warrants greater scrutiny in terms of conformance to external specifications and closer inspection in terms of implementation quality. It is not enough that the code works as far as acceptance testing can verify; quality assurance must also study the code itself for conformity to standards of clarity and reliability.
Working alone does not imply working without discipline or without good methodology, any more than working in a group guarantees results. Working in a group only guarantees there are more people involved. From the manager's perspective, however, using systematic development methodology assumes greater importance when there are developers working in isolation. Regression testing and implementation walkthroughs can only do so much in assuring quality. True quality cannot be achieved after the fact; it has to be designed and built in from the beginning. Systematic and formal methods for software development are proven ways of building in quality. The work of even the most independent coding cowboy will probably be improved and can certainly be trusted more if a systematic methodology is used.
In order to get the coding cowboys to use systematic development methods, management may have to compromise on which method is used. It is better that independent operators use their own idiosyncratic methods than no methods at all.
Design models not only can speed development and improve quality but can also aid traceability by providing a partial record of a developer's thought processes, an audit trail of the derivation of problems and solutions. Requiring appropriate design documents—data flow diagrams, structure charts, structured flow charts, and the like—needs to be part of the specifications for subsystems that are to be developed independently.
Project managers may find analogies to the cattle range useful to help keep in mind an assortment of approaches for dealing with programming mavericks. Trail bosses had to ride herd on their charges, watching for strays that separated themselves too far from the group, gathering them in periodically. They kept close watch on their cowhands and the cattle they herded. They also left room for cowboys to let off steam on Saturday nights.
Meilir Page-Jones has developed a very practical schema for understanding the ages of software process maturity. Some groups operate in the age of anarchy, developing software without the benefit of any systematic approaches or even codified wisdom. Everything rests on the skill of the individual. The age of folklore is characterized by a culture of collective wisdom, accumulated knowledge that is often embodied in stories about successes and failures or rules of thumb extracted from past experiences. The age of methods is based in systematic, although not necessarily formal, approaches to software development that go beyond folklore. The age of metrics is based on measures for evaluating quality and productivity and organized feedback for improving the development process based on measurement. Finally, we reach the age of engineering, in which software development becomes a true engineering discipline, a process under continuous improvement using methods that are not based in folklore or armchair speculation but on theory validated through study and research, in which design decisions and trade-offs are systematic and derived from models and metrics that embody the results of a growing body of knowledge.
Engineering is what you get when mature individuals in mature organizations use mature methods. Anarchy is what you get when you simply throw a group of coding cowboys into the corral together and point them at a problem.
Unfortunately, when mavericks are put into groups they are prone to becoming oppositional. It is not uncommon for them to end up working at cross purposes. Without creative leadership, a whole group of coding cowboys is apt to lead to unmanageable chaos.
When teamed with other less contrary types, coding cowboys can have a tendency to undermine teamwork. Some may hold back and refuse to participate in group problem solving, others may take up most of the air time with their own favorite topics and approaches. Often they are critical of any ideas they did not generate and end up in conflict with the rest of the group.
In the worst cases, the contrarions end up being pushed into a corner and scapegoated for their negative attitudes. In turn, coding cowboys may come to view the rest of a team as a bunch of groupthink bozos who couldn't code their way out of a cardboard box and don't appreciate true genius when they see it.
Managers need to know how to head off such polarization for the sake of the team and the end product. Direct confrontation may not be the preferred style of management for handling coding cowboys. It can be risky to face down cowboys. They have deserved reputations for strong opinions, sensitive egos, and itchy trigger fingers. Managers are unlikely to be able to win such a face off and may only antagonize their cowboys into even more entrenched opposition.
The real challenge facing managers is what to do about the mavericks and strays. Some would argue that they need to be cut out of the herd and put out to pasture, but this option is not always available and may not serve the best interests of the organization. My boss says that the job of the manager is to get rid of the obstacles that keep others from doing their jobs well and from fulfilling their greatest potential. Lucky me. Part of the potential of well-managed mavericks is to bring their creative energy and critical abilities to the organization.
Some kinds of teams thrive on opposition, others cannot abide dissent. It depends on the basic model by which a team is organized (Constantine 1993c; Larson and LaFasto 1989). Project teams that are based on top-down management and work assignment, that rely on traditional authority for direction, or that are headed by highly directive leaders, will have greater problems with mavericks and coding cowboys. The work of such traditional tactical teams (see Chapter 11) can be hampered by opposition, and their controlled style of organization can push mavericks into rebellion.
A free-wheeling “breakthrough” team (Chapter 12), on the other hand, can be energized by mavericks, whose independent spirit and creative individuality are just what the group needs. The issue here is how to foster creativity rather than chaos. The key to getting useful work out of such a team is promoting high levels of mutual regard and respect for each other's competence. Teams where members see their teammates as skilled professionals with good ideas can engage in healthy competition without degenerating into a free-for-all.
Teams that work by building technical consensus (see Part I) can also be good settings for mavericks. A technical consensus is one in which the contributions of all team members are taken into account in arriving at a common decision that all members feel they can support, even if they do not necessarily agree on every detail.
Opposition or criticism should never be confused with disloyalty. In fact, critical feedback is often the most valuable information and the most important to hear. Healthy skepticism and critical perspectives should be encouraged, not disparaged. The quality of group problem solving has been shown to be critically dependent on this kind of “negative feedback.” Groups that include a “resident critic” or “devil's advocate,” or that exploit dialectical processes of opposing ideas and active critique, perform better (Constantine 1989; Priem and Price 1991).
Managers should remember that, just as authority tries to control opposition, opposition resists control and authority. So, instead of controlling, co-opt! One way to co-opt opposition and turn it to the advantage of the team is to institutionalize it. In one approach (Constantine 1989, 1991a), the critic or “devil's advocate” is regarded as a valued contributor whose role in the group is critical to success. Like the “grit in the oyster,” critical monitoring of the work of the team is the necessary irritant that forms the basis for producing programming pearls (Thomsett 1990). As a team member, the maverick is formally charged with the responsibility to critique decisions, take alternative viewpoints, note exceptions, problems, and limitations, and make sure that the group considers alternatives and explores difficulties with proposed solutions.
Mavericks are often better at criticizing than being criticized. When they go off on their own, however, their work needs to be checked and reviewed, which they may resent as intrusive and distrusting. If critical feedback to the coding cowboy is needed, it is best rendered as impersonal as possible, appealing to the analytical and critical skills of mavericks, challenging them to be creative in solving whatever problems are uncovered.
It is not surprising that the biggest limitations on mavericks are self-imposed. They can get stuck in the need to be unique, to do things differently, even at the expense of practicality. Working alone also limits the size of what they can tackle. Mavericks can end up alienating coworkers.
We now know that diversity in teams promotes better problem solving and decision making. Several decades of research on group dynamics has demonstrated that heterogeneity beats homogeneity almost all the time. Teams of men and women solve problems better than single-sex teams. Groups that include varied specialties and training fare better. Diversity wins, whether it is diversity in personality, in interpersonal style, or in culture or ethnic background.
Clearly, then, we do not want to get rid of all the mavericks. We want teams in which a variety of leadership styles and responses to leadership are displayed—including resistance to leadership. We need movers and shakers on our teams, innovators who will generate ideas, and champions who will drive them forward. But we also need critics and skeptics who counter unbridled enthusiasm with cool doubt, who keep uncritical support from running away with unproved notions. Opposition to leadership can even serve the team by keeping leadership within bounds.
We are hampered in exploiting opposition by simplistic notions of teamwork that are based on ideals of perfect harmony and fantasies of cooperation without conflict. For the best collective performance, contention may be the most essential ingredient in a creative process that results in a quality product.
And what is a software sage? A software sage is someone who recognizes opposition as opportunity, and who sees flexibility in frustration. A software sage is the wise old ranch hand who has been through the range wars, has been there out on the lone prairie, and has returned to the homestead. A software sage may even be a reformed cowboy or cowgirl, one who has never lost respect for the individual but who has come to value cooperation.
Revised from American Programmer, July 1993.