Chapter 1. The Impact of People and Leadership on Scalability

Fighting with a large army under your command is nowise different from fighting with a small one; it is merely a question of instituting signs and signals.

—Sun Tzu

Here’s a multiple-choice pop quiz. Which of the following is the most important factor in ensuring the long-term scalability of the products you produce: people, process, or technology? Was your answer “people”? If so, we commend you. If it wasn’t, then consider the following: To our knowledge, the “Robot Apocalypse” has not yet happened, so our products do not build themselves. Any value or liability within our products is a result of human endeavors. People design, code, and configure the systems that run our products and are solely responsible for any defects in design, code, or configuration. People alone are responsible for the product architectures that either successfully process incredible transaction volumes or fail miserably, crumpling beneath the weight of consumer demand. All too often, however, people are the things that we as technologists and engineers overlook when striving to create scalable solutions or solve problems related to scalability. People are woefully overlooked and underappreciated. Organizational reviews of employees are once-a-year, check-the-box exercises in which cursory assessments are written in haste. Managers and leaders are frequently untrained or undertrained in the performance of their duties. In this chapter, we explain why the people, structure, management, and leadership in your organization all have an enormous impact on your ability to scale.

The first edition of The Art of Scalability eschewed real-world cases in favor of a fictional company, AllScale. This fictional company was an aggregation of many of our clients, their challenges, and their successes. While AllScale provided value in highlighting the lessons we hoped to convey, it was nevertheless a piece of fiction with limited ability to connect users to real-world successes and failures.

In this second edition, sometimes the stories we present are vignettes about an individual, as in the case of Steve Jobs at Apple or Jeff Bezos at Amazon. In other stories, we outline the success or failure of a company broadly and then point to key elements of that success or failure to help make a point.

All of the greatest successes in building scalable products have at their heart a great set of people making many great decisions and every once in a while a few poor choices. Ignoring the people element of scale is a very large mistake—one that is often found at the root of products that fail to meet end-user demands.

Given that people are the heart and brains behind scalability, it makes sense that we should spend a great deal of time and effort on attracting and retaining the best folks available. As we will discuss in Chapter 5, Management 101, this quest is not just about finding the people with the right and best skills for the amount you are willing to pay. Rather, to be truly successful, you must have the right person, with the right behaviors, in the right job, at the right time.

The right person speaks to whether the individual has the right knowledge, skills, and abilities. Putting this person in the right job at the right time is about ensuring that he or she can be successful in that position and create the most shareholder value possible while tending to his or her career. The right behaviors mean that the person works and plays well with others while adhering to the culture and values of the company. Bad behaviors are as valid a reason for removing a person from the team as not having the requisite skills, because bad behavior in any team member creates a vicious cycle of plummeting morale and productivity.

Perhaps no story better emblemizes the traits of the right person and the right behavior at the right time than both the firing and the subsequent rehiring of Silicon Valley legend Steve Jobs. Walter Isaacson’s masterful biography, Steve Jobs, paints a picture of a man who, in the spring of 1985 at the age of roughly 30, was destroying the company with his childish, rude, selfish, and often disruptive behavior. Morale was plummeting within Apple. While clearly John Sculley (Jobs’s replacement) was not the answer, Jobs’s lack of teamwork and constantly changing views of an appropriate product left the company devoid of focus and identity. After leading the launch of the world-changing Macintosh, Jobs apparently refused to believe or accept that the Apple II continued to drive a majority of the company’s sales. His derisive behaviors sowed discontent within the product teams—dissent that ran counter to a culture of learning from mistakes and capitalizing on successes throughout the company. Jobs’s behaviors, at the time, were simply not correct for what the company needed as a leader.

Now fast forward a little more than a decade. In 1996, Jobs was brought back in as Apple’s CEO, through the acquisition of NeXT and the subsequent ouster of Gil Amelio. Perhaps as a result of his experiences at both NeXT and Pixar, Jobs had matured. While he retained the maniacal focus on his vision of the perfect product, he had at least tempered many of the liabilities that resulted in his abrupt dismissal in 1985. Jobs returned to Apple as a more disciplined man, with all of the innate skills he had as a brash 30-year-old, but also with many more of the leadership attributes necessary to turn around a company. This is an example of how (as we describe later in this book) leaders can, in fact, be made—or at least made better. While Jobs was the right person in 1985, he did not have the right behaviors at that time. When he returned in 1996, he was clearly the right person with the right behaviors in the right job at the right time. Note that the “right behaviors” here doesn’t mean that Jobs was a nice guy; in fact, Isaacson indicates he was anything but a nice guy. Rather, the right behaviors means that the net results of his behaviors were generally a positive influence on Apple’s growth.

• How easily can I add or remove people to/from this organization? Do I need to add them in groups, or can I add individual people?

• Does the organizational structure help or hinder the development of metrics that will help measure productivity?

• Does the organizational structure allow teams to own goals and feel empowered and capable of meeting them?

• Which types of conflict will arise, and will that conflict help or hinder the mission of the organization?

• How does this organizational structure help or hinder innovation within my company?

• Does this organizational structure help or hinder the time to market for my products?

• How does this organizational structure increase or decrease the cost per unit of value created?

• Does work flow easily through the organization, or is it easily contained within a portion of the organization?

The question of how easily people are added is an obvious one. It is very difficult to increase the amount of work done by an organization if the structure does not allow for bringing in more people to perform additional work. Great organizational structures support the addition of people in small or large numbers, allowing new teams to be formed or people to be added to existing teams. They also enable the organization to flex “down” when and if the company falls on hard times.

The question regarding metrics is important because organizational output is a function both of size (the number of people doing work) and efficiency (the output per individual or team). Sometimes the means of achieving greater output isn’t “more people” but rather “more work per person” or “more total output at the same team size.” Without understanding individual or organizational efficiency (metrics such as throughput, availability, time to market, and cost per unit produced), how can we determine if it makes sense to spend money on productivity tools or additional headcount? Most of us wouldn’t consider driving a car any considerable distance if that car didn’t at least have a gas gauge and a speedometer. The same should be true about our organizations: we shouldn’t run them without key performance indicators that help us achieve our desired results. Management means measurement, and a failure to measure means a failure to manage.

If the structure of your organization makes it difficult to establish measurements for individual performance, you will not be able to measure output. If you cannot measure the output and quality of the organization’s work, you can’t react to sudden and rapid deteriorations in output.

Team members’ perception of whether they “own” goals and are empowered to deliver on them speaks to the autonomy of the team in achieving and owning its goals or, conversely, to its reliance upon other teams to meet its goals. Empowered teams generally experience higher morale, lower employee turnover, and faster time to market than teams that do not feel empowered. Fundamental to empowerment is the ability to own the decisions necessary to achieve a goal. To feel real ownership of their work, teams must feel they have the tools and capabilities to act upon their decisions. The teams that report the highest level of empowerment and ownership over goals are those that are cross-functional in nature and have all the skill sets necessary to achieve the goals set for their teams.

The question about the types of conflict that arise within this organizational structure, and whether that conflict helps or hinders the mission, addresses the two primary types of conflicts within an organization (cognitive and affective) and their relationships with the organization’s outcomes. Affective (“bad”) conflict is role- or control-based conflict and often arises between teams. Affective conflict is often about “who” does something or “how” something will be done. In organizational structures where multiple groups are necessary to deliver a product, such as “Operations,” “Development,” and “Quality Assurance,” the real question may be about “who” gets to define when something is ready for launch or “how” it should be developed to meet customer needs. How much input does the infrastructure team have into the way in which applications will interact with databases? How much input does the software team have into the definition of the deployment architecture? Who should make those decisions? Affective conflict rarely adds value to products; instead, it almost always delays the launch of products and increases their costs. Further, when left unhandled, it decreases employee morale, increases employee turnover, and diminishes the level of innovation within a company.

Cognitive conflict, in contrast, if handled properly, is often called “good conflict.” Most often cognitive conflict is related to “why” something must happen or “what” a company needs to do achieve some desired outcome. Imagine some of the best brainstorming sessions in which you’ve ever participated, or a genuinely effective prioritization meeting where a team really came together, made difficult decisions, and felt good about the results. Cognitive conflict expands our range of strategic possibilities. It increases the probability that we will make the right decisions by relying on diverse knowledge, skill sets, and experiences to cover the intersection of that which is imaginable and possible. More than likely you’ve had at least one experience where you’ve been in a room with someone who lacked your technical knowledge and expertise, but who nevertheless asked a question that completely changed your approach to a problem.

These two types of conflict, when joined together, help to define how we should think about forming teams. To reduce affective conflict, we want cross-functional teams capable of owning the outcome to a goal in its entirety. That means embedding each of the necessary skill sets into the team tasked with delivering some solution. This same approach—that is, the embedding of diverse skill sets within a team—also allows us to increase the levels of cognitive or positive conflict within a team. Whenever we can create such cross-functional teams, we know that morale increases, empowerment increases, employee turnover decreases, time to market decreases, and innovation increases.

The next two questions “How does this structure help or hinder innovation?” and “How does this structure help or hinder time to market?” are at least partially informed by the previous answers. Everyone might agree that they are important questions to answer, but their answers are a bit involved. We will address these issues in Chapter 3, Designing Organizations, during a more in-depth discussion of organizational structure.

The relationship between organizational structure and the cost per unit of value created brings up the issue of what we like to call the “organizational cost of scale.” In his book The Mythical Man-Month, Fred Brooks indicated that there is a point in the life of a software project when adding people to a project can actually cause the project to be delayed further (i.e., be delivered even later). Brooks points out that one reason for this further delay is the communication overhead associated with the addition of each new team member to the project. As the team size grows, the increase in the overhead per team member is not linear. In other words, the more people you add to a project or task, the greater the overhead per person in communication and coordination costs.

Have you ever been in an organization where you receive hundreds of internal emails each day and potentially dozens of meeting invitations/requests each week? If so, you have undoubtedly spent valuable time deleting the emails and declining requests that aren’t relevant to your job responsibilities. The more people within your company, group, or team, the more time you must spend reading and deleting emails as well as going to meetings rather than doing “real” work. This scenario perfectly illustrates why adding people can often decrease the output of each individual within an organization. As in the preceding example, when you add people, the email volume grows and communication time correspondingly increases. Figure 1.1 depicts an engineering team attempting to coordinate and communicate. Table 1.1 shows the increase in overall output, but the decrease in individual output, as the team size increases from one to three. In Table 1.1, the individual loss of productivity due to communication and coordination is 0.005, which represents 2.4 minutes of coordination activity in an 8-hour day. This isn’t a lot of time, and most of us intuitively would expect that three people working on the same project will spend at least 2.4 minutes each day coordinating their activities even with a manager! One person, in contrast, need not perform any of this coordination. Thus, even as individual productivity drops, the team output increases.

Figure 1.1 Coordination Steals Individual Productivity

Table 1.1 Individual Loss of Productivity as Team Size Increases

You can offset but not completely eliminate this deterioration in a number of ways. One possibility is to add management to limit nonessential coordination. Another possibility is to limit the interactions between individuals by creating smaller self-sufficient teams. Both of these approaches have benefits and drawbacks, as we will discuss in Chapter 3. Many other approaches are possible, of course, and anything that increases individual throughput without damaging innovation should be considered.

“Does work flow easily and quickly through the organization, or is it easily contained within a portion of the organization?” is meant to focus on the suitability of your organizational design to the type of work you do. Does work flow through your organization as efficiently as a well-defined assembly line? Waterfall processes resemble assembly lines; if your company employs such a process, having functional teams that hand off work to one another can make a lot of sense. Do your product architecture and the process you employ (such as Agile) allow the work to start and end within a single functionally diverse team? To achieve such a structure, the components of our architecture must work independently with minimal communication overhead between teams. An important point here—and a lesson that we will explore later in the book—is that our organizations, processes, and technology must work well together. Agile processes and functionally oriented teams working on monolithic, highly interdependent architectures can run into as many problems as waterfall processes with cross-functional teams working on largely independent and horizontally scaled product architectures.

One well-known company, Amazon, clearly demonstrates the value of ensuring the answers to our questions align to our end goals. You have likely heard of the “Two-Pizza Team” concept made famous at Amazon. In one off-site retreat, the executives at Amazon were discussing the need for better communication among teams. Jeff Bezos, the founder and CEO, jumped into the discussion by saying, “No, communication is terrible!”¹ Bezos was recognizing the cost of communication overhead as described in Table 1.1 and which Fred Brooks identified in The Mythical Man-Month. Bezos’s fix was to create the “Two-Pizza Team” rule: no team should be larger than can be fed with two pizzas. The idea here is that communication occurs primarily in the team and, as a result, communication overhead is significantly reduced. To be successful, however, the teams must be empowered to achieve their goals. Furthermore, each team must be given one or more key performance indicators (KPIs) that are indicative of overall success. The teams are often staffed cross-functionally so they have the skills necessary to achieve their KPIs without outside help.²

1. Alan Deutschman. “Inside the Mind of Jeff Bezos.” Fast Company. http://www.fastcompany.com/50106/inside-mind-jeff-bezos.

2. Stowe Boyd. “Amazon’s Two Pizza Teams Keep It Fast and Loose.” GigaOm. http://research.gigaom.com/2013/08/amazons-two-pizza-teams/.

Bezos’s hope in creating the “Two-Pizza Team” rule was to engender higher levels of innovation, faster time to market, and lower levels of affective conflict within teams. He explicitly designed the teams with KPIs in mind for the purposes of achieving measurability. The solution is highly scalable. Need more capacity? Add more teams dedicated to specific aspects of the product. Finally, work is easily contained within each team. With one simple construct, all of our key questions are addressed.

Having discussed why organization design is important to scale, let’s now turn our attention to why management and leadership are so important.

Management and leadership can either multiply or detract from your ability to scale organizations in growth environments. Although they are often perceived as existing within the same context, they are really two very different disciplines with very different impact on scalability. Many times, the same person will perform the functions of both a leader and a manager. In most organizations, a person tends to progress from a position of an individual contributor into a primarily management-focused role; over time, with future promotions, that person will take on increasing leadership responsibilities.

In general, we like to think of management activities as “pushing” activities and leadership as “pulling” activities. Leadership sets a destination and “waypoints” toward that destination. Management then gets you to that destination. Leadership would be stating, “We will never have a scalability-related downtime in our systems”; management would be ensuring that this scenario never happens. Every company absolutely needs both management and leadership, and needs to do both well. Table 1.2 is a non-exhaustive list of some activities and their classification as either leadership or management oriented.

Table 1.2 Example Leadership and Management Activities

Far too often, we get caught up in the notion of a “management style.” We might believe that a person’s “management style” makes him or her more of a leader or more of a manager. This notion of style represents our perception of an individual’s bias toward the tasks that define either leadership or management. We might believe that a person is more operationally focused and therefore more of a “manager,” or we might perceive someone as being more visionary and therefore more of a “leader.” Although we all have a set of personality traits and skills that likely make us more comfortable or more capable with one set of activities relative to the other, there is no reason we can’t get better at both disciplines. Recognizing the distinction between the two disciplines is a step toward isolating and developing both our management and leadership capabilities to the benefit of our stakeholders.

As we have indicated, management is about “pushing.” Management ensures that people are assigned to the appropriate tasks and that those tasks are completed within the specified time interval and at an appropriate cost. It ensures that people get performance-oriented feedback in a timely manner and that the feedback includes both praise for great performance and information about areas needing improvement. Management focuses on measuring and improving everything that ultimately creates shareholder value, such as reducing the cost to perform an activity or increasing the throughput of an activity at the same cost. Management means communicating status early and often, and clearly identifying what is on track and where help is needed. Management activities also include removing obstacles or helping the team over or around obstacles where they occur on the path to an objective. Management is important to scale because it is how you get the most out of an organization, thereby reducing cost per unit of work performed. The definition of how something is to be performed is a management responsibility, and how something is performed absolutely impacts the scale of organizations, processes, and systems.

Management as it relates to people addresses the need to have the right person, in the right job, at the right time, with the right behaviors. From an organizational perspective, it is about ensuring that the team operates cohesively and effectively, includes the proper mix of skills, and has appropriate experiences to be successful. Management as applied to an organization’s work makes sure that projects are on budget, are on time, and deliver the expected results upon which their selection was predicated. Management means measurement, and a failure to measure is a failure to manage. Failing to manage, in turn, guarantees that you will miss your organizational, process, and systems scalability objectives—without management, no one is charged with ensuring that you are doing the things you need to do in the time frame required.

Leadership, by comparison, has to do with all the “pulling” activities necessary to be successful in any endeavor. If management is the act of pushing an organization up a hill, then leadership is about selecting that hill and surmounting it first, thereby encouraging the rest of the organization to follow suit. Leadership is about inspiring people and organizations to be better and do great things. It creates a compelling vision that tugs on the heartstrings of employees and “pulls” them to do the right thing for the company. Leadership identifies a mission that helps codify the vision and develops a causal mental roadmap that helps employees understand how what they do creates value for the shareholders. Finally, leadership defines the objectives to be met on the path toward the organization’s goals and establishes KPIs. Leadership is important to scale because it not only sets the direction (mission) and destination (vision), but also inspires people and organizations to journey toward that destination.

Any initiative lacking leadership (including initiatives meant to increase the scalability of your company), while not doomed to certain failure, will likely achieve success only through pure dumb luck and chance. Great leaders create a culture focused on ensuring success through highly scalable organizations, processes, and products. This culture is supported by incentives structured around ensuring that the company scales cost-effectively without experiencing user-perceived quality of service or availability issues.

• The right person in the right job at the right time and with the right behaviors is essential to scale organizations, processes, and systems.

• Organizational structures are rarely “right or wrong.” Any structure is likely to have pros and cons relative to your needs.

• When designing your organization, consider the following points:

The ease with which additional units of work can be added to the organization. How easily can you add or remove people to/from this organization? Do you need to add them in groups, or can you add individual people?

The ease with which you can measure organizational success and individual contributions over time.

How easily goals can be assigned and owned by groups, and whether the groups will feel empowered to deliver these goals.

How conflict will emerge within and between groups, and whether it will help or hinder the company mission.

How the structure will help or hinder both innovation and time to market.

How the organizational structure will increase or decrease the cost per unit of value created.

How easily work flows through the organization.

• Adding people to organizations may increase the organizational throughput, but the average production per individual tends to decline.

• Management is about achieving goals. A lack of management is nearly certain to doom your scalability initiatives.

• Leadership is about goal definition, vision creation, and mission articulation. An absence of leadership as it relates to scale is detrimental to your objectives.