Chapter 5. The Culture of Collaboration: Organizing Teams
Imagine that your company is a person. What adjectives do you think your customers would use to describe its personality? That’s your company’s brand perception. What adjectives do you think the employees would use to describe it? That’s your culture.
The company culture is the true personality of your organization. It influences how it behaves, what it does, and the way it works. It is anchored on shared assumptions by employees, which heavily guide what they do and how they do it. Any plan, any activity, any initiative that is not aligned with that personality feels unnatural and doesn’t stick.
Without a culture aligned with AI, any attempt at embracing it will also feel against the grain and won’t stick, either. You can target a few use cases and even be successful in some of them, but you won’t be able to truly transform your organization with AI, any more than I could transform myself into a dancer by memorizing a couple of dance moves—it’s just not who I am or what I’ve been preparing for.
Fortunately, unlike me, company culture can be changed. It’s the result of many factors that we can act on: the way the company is organized, the leadership style, the communication strategy, talent management, training, rewards… these are all aspects we can control that help shape the culture of the organization. This chapter will focus on one important factor: the team structure and collaboration model.
From Projects to Products to Platform
Traditionally, enterprise IT has been managed as a set of individual projects. It is very tempting to apply the same model to AI—you can identify the high-priority use cases, kick off a project for each of them, and assign them people, goals, and plans. That approach sounds practical and easy to manage, but it will actually undermine your broader vision to transform the organization with AI.
Project-based IT management is being replaced in many organizations by product-based IT management. The concept is simple: instead of managing the IT services as individual projects that have a beginning and an end, we approach them as products that are in continuous evolution. A product management strategy focuses more on the needs of the user, continuously providing value to better serve those needs. A project-based approach focuses more on the tasks, and successfully delivering the results. After we deliver those results, we move on to the next project.
Building a house is a great example of project-based management. You can design the house in great detail before you even start to build it, and you can plan every step in the construction with very high accuracy. After the house is built, you move on to the next one.
Building software is very different. It is difficult to design it at a high level of detail in advance, and it is even more difficult to plan every step in its construction. To make matters more interesting, the needs will be changing all the time, even after it’s delivered. The equivalent in the house construction example would be getting a request from the homeowner to add one more bedroom after the house is built, or to change the entire orientation of the house, or (I’m sure IT readers will be able to relate) to add 10 more floors to a single-floor house.
AI is even more challenging. In such an emerging field, it’s impossible to perfectly plan every step along the way, and it’s even difficult to guarantee that a particular outcome will ever be achieved.
AI scenarios are in continuous evolution. As data changes, the AI on top of it must change too. A project-based mindset for AI would have bad consequences; projects wouldn’t meet the initial expectations and would soon be rendered obsolete by changes in the requirements or the underlying data.
In a sense, even a product-based management strategy is not sufficient for the deep transformation required by AI. Recall that the different parts of an AI organization are interconnected. As you transform your technical departments, they have to serve the business units. As you transform the business units, they have to serve the employees. The resulting structure looks more like a platform, as shown in Figure 5-1.
Figure 5-1. Platform structure in an AI organization
In platform-based management, not only you should consider the final users of your products, but every product should also serve as the foundation for other products in your organization.
For technical departments, this means that in addition to bringing AI to the applications, they should also provide core AI services to the business units. For business units, this means that as they transform their business processes with AI, they should also enable business users to create and apply AI solutions on top of them.
Microsoft is an extreme example of a platform-centric company. Our product portfolio is layered into tiers, from the core platform services in Azure to the business services in Dynamics 365 to the user services in Microsoft 365. Each layer is its own entity, and is designed to serve the layer on top of it.
In the platform approach, you should think of every product as having two audiences: the final users, and other products in the company. At Microsoft, we refer to that concept as first party equals third party (1P = 3P): the idea is that every product we develop for a particular business should also be considered as a platform for customers. The other way around is also true—every service we develop in the platform should consider the other businesses as their customers. This requires a shift in mindset:
-
Product definition should be done not only with the needs of the team leading the product in mind, but as part of a much broader exercise including other teams in the company.
-
Every product should be designed with composability in mind and be fully accessible to others. Different products should be able to integrate.
-
Common services and assets such as identity and data should be unified as much as possible in order to make it possible for multiple products to come together.
-
Duplicated capabilities should be identified early and avoided as much as possible.
-
Components such as AI models should be reused across all layers in the organization and easily discoverable through mechanisms like repositories.
Architecting Teams for a Platform Approach: MLOps
A project-based approach is a dream come true for management. You have full flexibility to move people between projects, optimizing resource utilization and increasing specialization. Teams can be organized by function (developers, data scientists, program managers, etc.), and a resource manager can assign the right people to the right project in real time.
It’s a pity that doesn’t work for AI. The resulting organizational model would try to build AI like a house. Business owners would create the specifications and plan for an entire project. After spending weeks defining this plan they would hand it over to the technical team, who would try to execute it. But without close involvement from the business, the project team would have to make various implementation decisions along the way, and the resulting outcome might not be valuable for the business anymore. Even if it were, the project would stall at some point when the business requirements or the data underneath changed.
Rather than organizing teams by functions, a much more appropriate way of tackling this problem is by creating agile, autonomous teams, with the business involved throughout the whole process. With this approach, all the roles required to manage the product end-to-end are represented in each team: developers, data scientists, product managers, and whatever else is required for the product. Because they operate autonomously, the teams are much more agile and better able (and motivated) to deliver a successful product.
Teams structured like this and driven by agile methodologies are already being used by almost every software company on the planet, including Microsoft. We even changed our office configuration to support this model, creating common areas where these multidisciplinary teams could work together. In a sense, each of these teams operates like a small startup inside a big corporation. They are highly autonomous, motivated, and driven by the success of the final product rather than by individual goals.
A good test to assess whether your organization is working with this model is the following: ask a data scientist in your company what they are paid for. Very likely, they will say something along the lines of “creating high-quality models” or “delivering AI solutions on time.” If that’s the case, your organization is probably not functioning in agile, autonomous teams.
All the members of an autonomous team should feel responsible for the business outcomes of the product they are working on. Every member of the team should reply to the question of what they’re paid for with the business metrics initially defined for that product, be they revenue, customer satisfaction, or anything else (see Chapter 3).
These teams not only include the product definition and product development components, but also the operations component. DevOps (or the equivalent for AI, usually referred to as MLOps) is an approach that brings operations and development together. A key aspect of it is continuous deployment, which allows the team to continuously deliver value from development to production, instead of waiting days or weeks for a different team to do so.
When you develop a product this way, you can bring one more stakeholder into the product development process: the user. DevOps is the critical piece that makes these usually autonomous stakeholders work in cooperation with product development. It allows you to include all your constituents in the development process, to make sure what’s being built is what the user actually wants.
Who are the constituents? In a platform approach, we need to include several of them:
- Business users
- The middle tier of the platform. In the case of a technical team, the business units should be part of it and should be involved in all stages of product development.
- Final users
- Employees for internal products or, for external-facing products, real customers. An absolute must-have for agile development is to involve the end user in the product as soon as possible to get early feedback.
- Other products in the company
- This allows you to ensure the product is aligned with the rest of the platform. It means that you will avoid duplicating work or creating different products that don’t integrate or compose correctly.
When you include these constituents throughout the entire development cycle, you make sure technology and business outcomes are always in sync and don’t diverge during the development process. You can continuously track your progress with regard to the strategy you defined initially and compare it to the goals you set.
The Agile Loop of MLOps
For this to happen you need to be able to capture as much information as you can from the system in production, so the loop is closed. Telemetry is one the most foundational components of DevOps. When we bring the user into the development process itself, we need to use telemetry to understand how the system is being used. This telemetry can be explicit, like when we directly ask the user about their experience (via survey questions, like/don’t like buttons in the interface, feedback options, etc.), and also implicit, with data generated as the product is used (clickthroughs, abandon rates, user flows, and so forth).
When telemetry is heavily used, the product definition can also be driven by data. How existing products are used is an excellent indicator of the users’ needs for future products or features. Product managers can use data-driven decisions to complement user research or their own intuition.
After the initial releases, telemetry will also be critical to evolve an AI solution. Model accuracy will degrade over time as conditions change. A monitoring dashboard consolidating all the telemetry data from your AI models is critical in an AI organization, as well as an established process that can react quickly to sudden drops in model performance.
When designing the MLOps solution for your organization, you should also assure traceability. Traceability will help you connect a specific model at a specific time back to the entire product lifecycle. For example, if a model is showing a lack of accuracy or bias in its decisions, you should be able to identify the version of the model, the history of changes, the team members responsible for those changes, and even the data that was used to train the model.
Telemetry will help you identify issues very quickly in your AI system, but traceability will help you identify the causes of those issues, which team members should be able to fix them, the model version and source code to fix, and even the data that was used.
Once the issue is corrected, you still have to deploy the system again to production. MLOps should also manage the continuous integration process. As changes are made to the product, the entire process from packaging the application to deploying it to production can be automated. Approval gates can be set up so updates don’t move through the process unless certain automatic or manual validation criteria are met. These can include performance requirements (e.g., compute or memory resources consumed), quality requirements (e.g., accuracy of the model), or even requirements for responsible AI, which you’ll learn about in Chapter 8.
With the right MLOps infrastructure in place, the definition/development/operations loop shown in Figure 5-2 can be dramatically accelerated. The result is a product that is developed with the stakeholders (business, users, and other products) closely integrated in the process, and in continuous evolution with telemetry data coming from its real usage.
Figure 5-2. MLOps loop
This agile loop and the multidisciplinary nature of the development will be the foundation for how autonomous teams operate in an AI organization:
-
Definition includes aspects such as the business understanding, experimentation with initial modeling, and data acquisition. It is driven primarily by business stakeholders in close collaboration with developers.
-
Development is focused on the actual implementation of the solution, including activities such as data preparation, modeling, and training. It is driven primarily by developers and it follows a continuous integration approach in which the solution is incrementally developed and validated.
-
Operations manages the system in production. It is also fully connected with development through continuous deployment, and provides telemetry back to the development process to help evolve the solution over time.
Services like Azure Machine Learning can help your team set up an entire MLOps system like the one described here, taking care of the infrastructure required to implement this agile loop.
Creating an AI Team
Beyond these teams that are operating autonomously in your organization, having a centralized AI-focused team of some sort will be very helpful to bootstrap the initial stages of your AI transformation. It will provide a center of gravity for your organization as well as strong accountability.
However, when creating such a team, you can explore different options for its scope. At the very minimum, this team should be the cross-company virtual leader for the AI transformation. That includes responsibilities like rallying the organization around AI, defining the strategy, identifying and planning the key AI initiatives and products, and evaluating technology solutions. This team should have a strong background and training in AI, and enough business experience and internal connections to apply that expertise to the business.
Beyond leading AI in the organization, a natural progression in scope for this team is to support the rest of the organization with additional functions such as:
- AI research
- Incubating technologies that can be used by the rest of the teams
- AI architecture
- Designing high-level technology components, as well as the interconnections between them
- AI production
- Providing core services to support the operations teams
- MLOps
- Providing services to the development teams, such as collaboration, security, telemetry, and continuous integration
- AI training
- Delivering AI training to the rest of the organization
Providing these functions centrally is usually advisable. The concept of autonomous AI development teams is extremely powerful, but it doesn’t scale if you don’t complement those teams with supporting functions. Imagine dozens of autonomous teams doing the same foundational research, or creating their own DevOps and production systems. Decentralizing these functions would result in duplicated work and a lack of visibility and control across teams.
Finally, the ultimate level of centralization is for the AI team to host these autonomous development teams themselves. This means that the AI team will also be the one developing solutions, which makes it extremely independent and agile but also brings the risk of a weaker connection to the business. In this situation, you will need especially strong connections between the business units and the centralized team, plus heavy endorsement from the leadership team to make sure the business units are accountable, as well.
Where Should the AI Team Report To?
Where to create the AI team is always a tricky decision. In some organizations it makes sense to add these functions to the IT department under the chief information officer (CIO). That can be a fitting arrangement if the IT department is already responsible for developing software or providing the supporting functions for other groups doing software development. In other companies, these functions are consolidated under the chief data officer (CDO). That approach can also make a lot of sense given that the CDO’s office is already on point for data and advanced analytics, which have a natural progression to AI. In other cases, these functions are the responsibility of other parts of the organization, such as the chief technical officer’s (CTO’s) office or the research team. The final decision really depends on your company’s situation and aspects like:
-
The change management needed. What organization has the closest functions and skills already?
-
The existing dynamics in the organization. Is the team already perceived as the leader driving the AI transformation across the business units?
-
The technical/business balance. Does the team have enough technical skills, combined with a great connection to the business?
-
Resource ownership. Does the team own a good percentage of the resources needed to execute the AI transformation, or will it depend on help from other teams to make it happen?
-
Leadership. Does the team’s leadership have enough background and expertise to lead this transformation?
Some companies for which there is no clear fit for these functions are opting for creating a new department focused on AI, led by a new C-suite role—the chief AI officer (CAIO). A fresh start with this new department can create the required focus and specialization, which is sometimes difficult to get in existing departments with a lot of baggage.
It’s not uncommon to follow hybrid approaches, as well. The IT department can provide the core infrastructure functions such as the production systems and MLOps services, and the CDO’s office can provide functions related to data and architecture. Development teams can be expanded from existing data science and application development teams in a central department or the business units. The AI team in that case can be scoped to the leadership role and orchestration across all those different teams, and in extreme cases it can even be relegated to just a virtual team across them.
No matter what approach you take, you will always end up with a new department, team, or virtual team that will become the primary driver of the AI transformation across the entire business. In all cases, this team’s success will rely heavily on its level of empowerment and connection to the business—and even a new department led by a fully empowered CAIO will need to battle internal resistance and the inertia of the current business.
That’s where the CEO has to step in. Every big transformation requires the company’s leadership to provide a clear vision and prioritization. Without the CEO clearly articulating the importance and the vision behind the AI transformation, the internal resistance would likely be impossible to overcome. The entire company needs to understand the reasons for the change and the criticality and extent of it. This can take the form of a simple communication like the famous “Internet Tidal Wave” memo from Bill Gates or the “For the cloud, we’re all in” communication from Steve Ballmer, but it’s usually much more than that. It’s continuously reinforcing the vision in every communication, both internal and external. It’s updating the company’s vision and mission. It’s changing the processes and even the employee compensation. It’s bringing the board along on the journey.
Without full buy-in from every business unit and every employee, the goals of the AI team pushing for the AI transformation would be impossible to achieve. The use cases would be the wrong ones for the business. The lack of involvement from the business users would result in products that don’t meet the requirements.
Once that leadership endorsement is in place, success will rely on the team’s ability to earn the trust of the business. Multidisciplinary, agile teams are usually very helpful in making that happen. Providing continuous value and driving a shared process for prioritization can help avoid misalignment with the business. Business units are usually in the performance zone: they’re particularly motivated by short-term revenue generation and cost-saving projects. Because of that, balancing these types of short-term projects with longer-term moonshots is also very helpful in order to have a healthy AI/business partnership.
Only companies with very strong AI-supporting functions as a foundation and full buy-in from the leadership will be able to develop a strong integration with the business and succeed in the journey to becoming an AI organization.
Building Versus Buying
Building your AI products in-house is not an option for many organizations. The skills gap or just the economics may mean it’s not feasible to rely only on internal resources.
The decision of whether to build or buy is not all or nothing, though. Many scenarios can be addressed with the existing talent in the organization. For example, most of the examples of bringing AI to applications that you saw in Chapter 3 can be handled by existing application developers, using prebuilt AI services and conversational AI platforms and drawing on the skills they already have. For organizations with existing data science teams, it is also a natural progression to target custom AI scenarios applied to business processes—and business users and analysts who are already doing business intelligence are great candidates for using self-service AI scenarios.
However, you will usually need to complement this strategy, either by acquiring prebuilt software-as-a-service (SaaS) solutions or by outsourcing custom development to third-party vendors. Good candidates are products supporting your horizontal processes, such as sales, marketing, customer service, or HR. The use cases in those processes are relatively common in the industry, and it’s often easy to find packaged solutions that provide immediate value.
On the other hand, vertical processes specific to your industry and your company are excellent candidates to develop in-house or in a close partnership with a vendor that can custom-develop the solutions. These vertical processes are usually the ones that differentiate your company from the competition, and you want to put as much attention and customization into them as possible.
In all these approaches, the role of the internal AI team is critical. A strategy involving in-house development, third-party SaaS solutions, and custom-made outsourced products cannot be successful without a strong foundation of supporting functions. The resulting architecture would become unmanageable very quickly, siloed into specific use cases that aren’t composable or reusable. All the supporting functions, from data management to MLOps to production to architecture, should be centralized in the company and shared across any built, bought, or outsourced solution.
Once you find the right organizational structure and collaboration model for your team, you can start creating your first AI solutions. But before that, you need to have a foundational data estate that can feed your AI solutions. That is the focus of our next chapter.