It is the long history of humankind (and animal kind, too) that those who learned to collaborate and improvise most effectively have prevailed.
—Charles Darwin
After a collective of over 50 years working as software engineers, we want to have an honest conversation with you, the reader of this book, on why agile software projects and teams fail. Over the course of our careers, we navigated a variety of consulting companies, large corporations such as IBM, mid-sized, aggressive industry leaders such as Shutterstock and AlphaSights, and small, fast-paced startups.
We have contributed code to products in a wide range of industries—from TV networks to online grocery delivery, ecommerce, research laboratories, IoT, online marketplaces, education, and fintech.
But no matter what the industry, product, or team size, the projects that succeeded all had one thing in common: good leaders that hired good professionals, and then gave these professionals the autonomy to do their job.
Similarly, projects that failed were led by people who either lacked the necessary knowledge or the required attitude.
The engineers working on these projects had either (i) free reign and no standards to follow, were exposed to constant power struggles by their colleagues and turned the codebase into a swamp or (ii) were so enshrined in red tape and lack of autonomy that they could not produce.
As software engineers, we like to tell ourselves that the codebase defines a successful product: clean, easy to understand, testable, well-structured. However, this is only partially true. The main pillar of any product is the people behind it, and the main challenges in software engineering today are human, not technical. A bad codebase can be refactored and made robust by good engineers and a leader who knows how to facilitate it. At the same time, a good codebase can be ruined in a matter of weeks by lack of direction, lack of oversight, and pent-up frustration. The latter not only impacts code, but inevitably leads to wider problems within the team. If not contained by leaders, these problems quickly expand beyond the team. Rot spreads, and once settled in within a company, it can become very difficult to clean up. Like steering a massive oil tanker, fixing dysfunctional teams takes a tremendous amount of time and force, and often entails a heavy cost.
Important Terms to Remember
Before we dive into the details of the above, we should define some terms. Specifically, we should clarify what we mean by successful and unsuccessful products and how we are to understand the term “dysfunctional teams.” We will begin with the latter.
Dysfunctional Teams
At one point or another in our careers, we have probably all been part of a team that was dysfunctional. We all “felt” that things were not working as they should be, but what exactly does that entail? From our experience, teams are dysfunctional if the individuals composing the team cannot work together toward achieving a common objective. Similarly, an organization that has a great development team can be considered dysfunctional once those people involved in “achieving the team’s mission” are not working in tandem.
In other words, the people composing the company are not building a product/solution together. Instead, any advancement toward the goal requires an inordinate amount of effort.
Team members compete and direct their force and energy against each other rather than channel it toward achieving the objective. Dysfunctional teams leave members emotionally drained and produce output that is of poor quality.
- 1.
Absence of Trust – The fear of being vulnerable with team members prevents building trust.
- 2.
Fear of Conflict – The desire to preserve artificial harmony stifles the occurrence of productive ideological conflict.
- 3.
Lack of Commitment – The lack of clarity of buy-in prevents team members from making decisions they will stick to.
- 4.
Avoidance of Accountability – The need to avoid interpersonal discomfort prevents team members from holding one another accountable.
- 5.
Inattention to Results – Pursuing individual goals and personal status erodes the focus on collective success.
- 1.
Frequently produce high-quality work
- 2.
Maintain a friendly, collaborative environment
- 3.
Use friction and disagreements to their advantage
- 4.
Interact well with other teams
- 5.
Are committed, have a clear hierarchy, and hold members accountable
- 6.
Trusting and attentive
Worded differently, members in well-functioning teams know that they are all in the same boat, understand their role and the impact it has, and thus produce successful products.
Successful and Unsuccessful Products
This leads us to our second definition: What are successful software products? From a bird’s eye view, successful products are those that provide value to the user, providing solutions to existing problems while helping a company to achieve its goals.
Creating a successful product does not mean creating a perfect product. Indeed, the most successful products are often far from perfect and require many iterations and generations to reach widespread adoption. However, they solve a problem and do so in a manner that is scalable. Conversely, unsuccessful products are ones that fail to solve a problem—either due to bad quality, because their creators failed to understand the problem that they were trying to solve in the first place, or because they take so long that they become irrelevant by the time they’re completed.
Leaders
Last but not least, we need to clarify what we mean when we refer to a leader. Roles across companies often have different meanings. In startups, there might be no engineering manager, and the CTO or founder might implicitly assume the role of principal software engineer, manager, and team lead. In other companies, roles might be better defined, but titles change. Some companies have no team leads but only senior software engineers or managers. Other companies might call tech leads “DRIs” (Directly Responsible Individuals), and their responsibilities might overlap with those of a principal software engineer or team lead.
Then again in other companies, the team lead might assume more of a managerial role. And so on, so forth. Therefore, we want to encompass all of these roles under the umbrella term leader. From our perspective, a leader is anyone who has the ability to direct other people and influence their decisions, whether they hold a formal title or not.
A leader is a guiding hand, who can change processes and standards within an organization, and who acts as a facilitator between different stakeholders. Leaders can orchestrate far-reaching change—either through direct decisions or because their decisions have a cascading effect.
With these definitions in place, we can gain a better understanding of the scope of this book, which is concerned with identifying and rectifying the most common mistakes that lead to dysfunctional remote agile teams and hence unsuccessful products.
Pillars at a Glance
An illustration represents Leadership, people, feedback, focus, quality, bureaucracy, expectations, and ethics as the eight pillars of successful leadership.
Summarizing the nine pillars for successful leadership of remote agile teams
Leadership
The first of these pillars is leadership and management. People are the most important asset that a company has. This means that, when looking at the big picture, good people management is generally more critical than good technical management. Being a good leader therefore requires being good at facilitating interactions between people. And this can only be achieved when striking the balance between micromanaging (bad), maintaining presence, and technical oversight (good). At the same time, it involves knowing how and when to shield your team; how to funnel information; how to manage deadlines; and how and when to provide autonomy and calibrate expectations.
People
The second pillar revolves around hiring the right people. This requires knowing how to identify good and bad engineers, and understanding what the cost of each incurs. Furthermore, being good at hiring people means understanding and valuing experience. Many times, recruiters focus solemnly on the technical abilities of engineers, neglecting the fact that decisions made by engineers, while technically correct, can fundamentally impact and change the project down the line. Understanding what decisions can be changed later, being exposed to dealing with other people’s past decisions as well as navigating “unknown unknowns,” is more important than technical knowledge, as the latter can be learned quickly but the former can only be gained over time. Identifying engineers that have lived through their past mistakes, and hence are unlikely to repeat them while working for you, is a crucial skill set when it comes to attracting talent.
Feedback
The third pillar is feedback, and involves realizing that feedback is a tool for continuous improvement—both from a personal and organizational perspective. Being “good at feedback” means listening to software engineers (e.g., understanding what is not being said) and acting on the feedback provided (knowing how to identify feedback that helps the team’s purpose and communicating this effectively).
Focus
The fourth pillar concerns focus. That is, balancing the costs of change and inaction. Setting a goal on Monday and then changing it on Tuesday while expecting great results Wednesday just won’t work, and leaders need to understand this. At the same time, the company cannot become a massive oil tanker that is impossible to maneuver. It is important for the product to continuously evolve—one iteration at a time. Products that standstill (due to bureaucracy or inaction) will cause them to eventually disintegrate and fail. Managing these trade-offs between engineering and changes in the product is crucial and raises the question to what degree changes are permissible. The answer is rooted in a quote by Steven Covey: “The main thing is to keep the main thing the main thing.” In other words, to succeed one needs to maintain focus and consistency. This also entails understanding the difference between starvation and drowning:
Death through absence or excess. When we starve, we die because we do not have enough of one thing. Drowning, on the other hand, means dying because we have too much of something (in this case water).
Quality
Quality is the fifth pillar and is concerned with how to set high standards for your team. We all tend to sink to our lowest permissible level. If you set low standards, the team will adopt low standards. If you set high standards, your team will try to meet those. At the heart of this notion is the “broken window theory,” adopted from criminology.
In short, and adapted to the context of software engineering, it goes like this: When you start working on a poor codebase or join a dysfunctional team, then you won’t feel bad for misbehaving or writing further garbage code. However, when you start working on a good codebase or join a high-performing team, then there is a psychological factor that inhibits you from degrading the environment since it would be very easy to link the messy code or snarky comments back to you. You would be “the only guy pooping in the park,” so to speak. Therefore, communicating clearly that code quality, usability, and security are priorities is important as this is the expectation that your team will have to meet.
Expectation Management, Bureaucracy Reduction, and Ethics
The last three pillars are equally as important as the first and involve expectation management, bureaucracy reduction, and ethics. Focusing on the latter, we have found that weak ethical positions are most often due to the following factors: (i) excessive ego, (ii) lack of empathy, and (iii) a leadership that is excessively obsessed with metrics (which oftentimes they do not fully understand). The latter means that charts become an end in itself, causing people to lose track of what is or isn’t ethical. The journey to this stage is always the same: Leaders might initially believe in their product or goal, but then lose themselves by focusing too much on their ego, metrics, and the solution they have (instead of focusing on the problem). Oftentimes, leaders try to aim for good metrics rather than good products. That is, instead of building a good product that then moves the needle, they try to move the needle directly. Moving metrics therefore becomes the goal and is placed above loyalty to the customer. Eventually, this becomes difficult to correct for a company as a whole.
Here Macbeth comes to mind: “I am in blood / Stepped in so far that should I wade no more, / Returning were as tedious as go o’er.” Applying MacBeth’s point of no return to the context of building software products means building a product that is inherently low quality since it is geared at moving some metric (or personal goal), rather than geared at providing value to users. Once this poor product reaches a certain stage, it becomes so difficult to salvage, continuing down the wrong path almost becomes easier than turning around. In order to avoid this trap, leaders should (i) create an environment which minimizes the negative impact of big egos and (ii) ensure that each decision they make allows the company to earn money by providing value; not by driving metrics.
Conclusion
Last but not least, it is important to note that this book is geared toward those who are in, or want to get into, a technical leadership position within the field of software engineering—senior software engineers, startup founders, engineering managers, or team/tech leads. This does not mean that less technical leaders will not find this book interesting (on the contrary: we hope they do). However, it does imply that some of the topics we discuss might not translate higher up the hierarchy (e.g., to the C-Suite level). Indeed, depending on the size of your organization and structure, some aspects of our conversation might apply differently, or might not apply at all. We therefore urge you, the reader, to use your own good judgment on whether the knowledge we share here makes sense for your context. After all, we are trying to have a conversation with you and share our collective experiences. We are not trying to fool you into believing that “this is all there is to know.” Leading people and building products is difficult and extremely complex, and no one book can teach you all that there is to know. All that a single book can do is teach concepts, change your perspective a little, or awake new thoughts in you. As Franz Kafka famously said: “A book must be the axe for the frozen sea within us.” We hope that this book serves as such an axe.