Assess

To recap, the Assessment Component includes assessments of the following:

• AI readiness

• AI maturity

• AI key considerations

Following are some questions around these categories that your assessments should answer:

• What are the gaps in my company’s readiness for AI?

• Is my company able to pursue scientific innovation (i.e., be agile, exploratory, and experimental)?

• How would I characterize my company’s data and analytics maturity? What about AI maturity? What gaps do I need to address?

• What are the AI-specific key considerations that we should understand and address?

• What are the critical assumptions and potential risks we need to understand and test?

From the AIPB process categories, the assessment category is applicable to answering these questions. Assessments should result in a plan—that is, a strategy. As such, the outputs of this component are defined by AIPB as an assessment strategy, which includes strategies for filling gaps related to AI readiness and maturity as well as strategies for addressing AI key considerations. Neither should be considered hard requirements to move forward as discussed, but they shouldn’t just sit on a piece of paper as a bullet point list of gaps and considerations either. Turning assessments into an assessment strategy is critical; it helps planning and, most importantly, helps to avoid potential failure while ensuring maximum success.

Many AI initiatives fail, as already discussed. I would argue that most do so because of either lack of assessment strategy, as we’ve defined, or because of lack of vision, strategy, and the ability to execute, as we cover next. In either case, don’t wait until it’s too late to identify and address potential points of failure.

Lastly, given the business-level and strategic nature of these assessments, the experts involved should be mostly managers (including domain experts) and appropriate SMEs. This includes executive leadership, functionally appropriate executives and managers (e.g., CAIO, CDO, CAO, VP AI/Data Science, Director AI/Data Science), and appropriate individual contributor team leads brought in as needed for specific expertise and analysis.

Vision

In my opinion, a vision for a new business, product, service, or feature is the high-level why, how, and what. The why can be expressed in terms of goals, benefits, or outcomes. The why can be driven by solving a specific problem, meeting a certain need, or eliminating a given pain point, but it can also be driven by wanting to create better human experiences without requiring a problem, need, or pain point. The goal is to define the “right” why’s, which can mean using a technique like the five whys to get there.

A great example are the swipe and pinch touch interactions that were introduced by products such as the first iPhone. I don’t think many people would have considered it as being a solution to a problem per se, nor do I think many people ever thought that type of interaction was even a possibility at the time. That said, as soon as people experienced those touch interactions, it became a delighter for the masses. Further, many people viewed these new interaction possibilities as solutions to problems that they didn’t even realize they had. Delighters can fuel a great vision in the same way that solving problems and eliminating pain points can.

Returning to AIPB’s emphasis on people and business, the key is to define the why for both. For businesses, the why is most commonly framed as a business case: how will it help the business in terms of goals, KPIs, objectives, ROI, or some similar measure. This is where most frameworks or models stop.

AIPB must also define the why for the people that are benefactors of the solution, which can be a customer, end user, or internal employee. There are very few, if any, innovative technology applications that don’t have an impact on a human in one way or another. Even full-blown automation affects people, although not always in a beneficial way if it is used to eliminate jobs without any job reassignment, retraining, or reskilling. We examine the topic of automation and job displacement in a later chapter, but for now keep in mind that this book is about how we can use AI to benefit people and businesses, and that includes beneficial augmented intelligence, job reassignment, retraining, and reskilling, as needed.

So, the key then is to define the business case and also the intended benefit for people who will experience the AI solution directly, regardless of whether the AI solution takes the form of insights, recommendations, predictions, augmented intelligence, optimization, or automation. All of these forms of AI can benefit stakeholders, so the goal is to identify who the stakeholders are and how they benefit.

So how do you formulate your AI vision and what is the final output? Let’s begin by asking the right questions:

• In which of the following am I most interested? Achieving a company goal, reaching a specific KPI target, solving a problem, eliminating a pain point, creating better human experiences, saving lives, maximizing patient outcomes, or helping people with disabilities, for example?

• If I develop an AI-based solution based on the answer to the previous question, how will it benefit my business? How will it benefit people?

• What AI opportunities will help me make these benefits a reality (assuming more than one), and how do I choose and prioritize them?

• At a high level, what will the solution be?

• At a high level, how will I be able to make this solution a reality and a success?

• What is the potential ROI?

You’ll notice I use the phrase “at a high level” for a couple of the questions. The reason is that we have not yet gone through the process of actually creating a strategy for making your vision a reality, and therefore we might not know many of the considerations, risks, details, designs, and so on that we’ll discover along the way. We’ve also not yet gone through the exercise of determining the desirability, viability, and feasibility of a promising idea. Vision development is a starting point of an iterative process, and, as I mentioned earlier, it might need to be modified and improved based on downstream activities. This is perfectly fine and, if anything, will help ensure success.

In terms of answering these questions and producing the desired output of the vision phase, I recommend managers (which includes domain experts) as the primary experts to participate in the process; for example, executive leadership as well as functionally appropriate executives and managers. You also should bring in other experts as needed (particularly AI practitioners from the scientists category), and the process should be collaborative and interactive, involving high-level assessment, ideation, and prioritization because there might be multiple great opportunities.

For the AIPB methodology vision phase, and from the AIPB process categories, I recommend the ideation and vision development category. Many of the recommended methods in this category are commonly and successfully employed by product managers and UX folks.

The output of the vision phase should be a people and business-focused vision statement that specifies the answers to the questions posed in an easy-to-understand way that can be delivered as an elevator pitch. It should start with why and describe the purpose and goals of the AI application (for both people and business), how it will be built, and what the solution will be.

A great example of something similar is Amazon’s “Working Backwards” customer-centric methodology.⁵ It’s an approach designed to work backwards from the customer in order to come up with product ideas, as opposed to the other way around.

The output of the process is a one-page internal press release written by a product manager that announces the not-yet-existent finished product. The idea is that the press release must indicate who the target customer is, what benefits the product will provide to them, how existing products fail, and how the new product will be better than the existing alternatives.

Ian McAllister from Amazon notes:

“If the benefits listed don’t sound very interesting or exciting to customers, then perhaps they’re not (and shouldn’t be built). Instead, the product manager should keep iterating on the press release until they’ve come up with benefits that actually sound like benefits. Iterating on a press release is a lot less expensive than iterating on the product itself (and quicker!).”

Ian McAllister, Director

McAllister suggests writing these press releases in what he calls “Oprah-speak.” In other words, write it in the way you would imagine Oprah Winfrey explaining it to her audience.

Although not exactly phrased in terms of the type of vision statement that I’m recommending for AIPB, it’s certainly a great example. I especially like his term “Oprah-speak”; that is, if Oprah can deliver the vision to an audience and have everyone understand it, it’s probably well built. I provide an example of developing an AIPB vision at the end of Part II.

Strategy

With an AI vision in the form of a vision statement in place, the next step is to develop an AI strategy. Developing an AI strategy (or data and analytics strategy in general) is something with which I find many companies, executives, and managers struggle with.

AI applied in the real world is relatively new, not generally well understood, engrossed in excessive marketing hype, needs a certain amount of technical expertise that is in short supply, and requires acceptance of uncomfortable concepts such as nondeterminism and uncertainty to truly understand and develop a strategy based on it. These are some of the many reasons why AI leadership is critical to successfully developing a vision and strategy around AI.

So, what exactly do I mean by AI strategy? In the context of AIPB, an AI strategy is the plan for execution of your AI vision in order to make it a successful reality. It also represents a plan for executing the end-to-end process of taking an idea through to becoming a continually optimized solution in production. An AI strategy guided by AIPB should take the form of a solution strategy and prioritized roadmap.

The solution strategy (i.e., the plan) should define the people, processes, and resources (e.g., tools) required to do the following:

• Make your AI vision a successful reality

• Execute your assessment strategy initiatives while executing your AI strategy

• Iteratively execute the Five Ds (discussed shortly)

The purpose of the AIPB solution strategy is to successfully execute your AI vision by helping to develop an AIPB prioritized roadmap. It’s also to ensure successful downstream AIPB Methodology phases, outputs, and resulting AI-initiative benefits and outcomes. Having the ability to create and execute a successful AI vision and strategy is a critical part of pursuing AI initiatives.

The actual format of the solution strategy is less important than the strategy itself. We can materialize it as one or more documents, diagrams, whiteboards, or any other format as long as it serves its purpose and gets the job done. A prioritized roadmap, on the other hand, should have a well-defined format, as we’ll later discuss.

From a product strategy, design, and development methodology perspective, I use my own version of a model that some refer to as the Five Ds for describing the iterative and end-to-end process of building successful AI products and product features. Figure 3-7 shows the stages of the Five Ds.

Note that actually performing some of the work outlined by the Five Ds (e.g., design, develop, deliver) is covered by certain downstream methodology phases of AIPB, so the purpose of the Five Ds during the strategy phase of AIPB is to develop a plan for successfully executing these downstream phases.

Figure 3-7. The Five Ds

To create a solution strategy as part of your AI strategy, answer the following questions for each stage of the Five Ds. They are posed in the context of the AI solution defined by your vision.

Discover:

• What are the why’s, goals, needs, and/or pain points?

• How does the solution create better human experiences and business success?

• Who are the stakeholders and benefactors of the solution around which your vision is based?

• What are the solution’s business and individual use cases?

• What does the potential market look like (market analysis) and how is your solution differentiated from your competitors (competitor analysis)?

• How can you ensure that your solution will achieve the intended benefits for both people and business?

• When applicable, how can you ensure that the solution is highly delightful, usable, and sticky (see Chapters 7 and 8)?

• How complete will the solution be—design prototype, PoC/pilot/MVP, full solution?

Define:

• What data will be used? Does the data already exist? Do you need to generate any new data? If yes, by what mechanism (e.g., augmentation methods, IoT)? Are there any external data sources that would be beneficial (through purchase or that are publicly available)?

• Who owns each of the data sources and what steps are necessary to get access?

• What data pipeline (access, ingestion, ETL, processing, integration, storage), if any, is required?

• How will the data be prepared (cleaned, validated, transformed, labeled) for your solution?

• What platform or tools (e.g., Amazon Web Services [AWS] or Google Cloud Platform [GCP] cloud) will be used to train, validate, and optimize models or perform any other AI and machine learning tasks?

• What software programming languages, architecture, and technologies are required for your solution (tech stack)? This includes firmware and embedded software such as those involved in IoT, and edge and fog computing.

• What hardware, materials, and manufacturing are required for your solution?

• What are the software and hardware requirements for your solution (both functional and nonfunctional) on which all designs and working software/hardware will be based? Note that this should be in the context of Agile requirements for software and more waterfall-esque for hardware because, as they say: measure twice, cut once.

Design:

• What software and hardware designs are required for your solution? This includes visual designs, mechanical designs, and any other assets associated with designing a UX (e.g., information architecture, interaction and flow design, user journey, use cases) or physical object.

• What technical diagrams, charts, graphics, or schematics are needed?

Develop:

• What hypotheses do you have, and with what AI and machine learning software, algorithms, and techniques will you explore and experiment? Remember, this is scientific innovation.

• How will you turn all requirements and designs into a high-quality, successful reality?

• How will you ensure that you follow software and hardware development best practices?

• What Agile software development methodology will you use (e.g., Lean, Kanban, Scrum)?

• What Continuous Integration/Continuous Delivery (CI/CD), and general release mechanisms will you use?

• What version control technology (e.g., Git) and branching strategy will you use?

• How will you test everything to ensure maximum quality?

• How will you test usability and gauge other aspects of the UX?

• How will you measure success, what metrics will you use, and what tracking mechanisms must you build into the solution?

Deliver:

• How will you deploy your solution to a production environment so that it is being used by actual benefactors in the real world?

• How will you monitor the health of the solution and address any issues?

• How will you continue to learn from new data and make data-driven improvements to the solution over time?

• How will you ensure that the intended benefits of the solution are being realized by both people and your business (efficacy)?

• How will you monitor the efficacy of the solution and address any issues (e.g., stale models; aka model drift)?

• How will you scale the solution as needed, and ensure that all other non-functional requirements are being met?

Your solution strategy is all about making your AI vision a successful reality. This begins by performing discovery in order to guide the prioritization and definition of the best solution (prioritized roadmap), which then guides how you design, build, deliver, monitor, and improve the solution.

Note that some of the discovery questions given here were also included in our discussion of developing your AI vision. There is some overlap. The initial discovery and understanding from the higher-level vision phase will guide your strategy development and increase granularity in terms of discovery and definition.

From a product management perspective, a product vision and strategy is manifested as a prioritized product roadmap that consists of a plan to deliver a working solution either through continuous delivery (CD), product releases, or both. The product roadmap can be either time constrained based on product development team estimates or rough and not necessarily based on fixed time and effort estimates.

The latter is more appropriate with increasing levels of pioneering, innovation, and uncertainty (i.e., scientific innovation) and depends on your company’s degree of AI readiness and maturity. Statistics- and probability-based technologies are scientific, empirical, and nondeterministic by nature, and this certainly applies to using state-of-the-art and emerging AI techniques.

Developing an end-to-end, AI-based innovation strategy in the forms of a solution strategy and prioritized roadmap requires expertise across most functional areas of a business. This means that all AIPB expert categories are applicable as needed, with oversight and management as the responsibility of the manager group of experts (especially product managers given the prioritized roadmap output).

For the AIPB methodology strategy phase, and from the AIPB process categories, I recommend the following:

• Ideation and vision development

• Business and product strategy

• Roadmap prioritization

• Requirements elicitation

• Product design

I provide an example of creating an AIPB strategy in Chapter 14.

Now let’s move on to discussing the final three phases of the AIPB methodology, beginning with executing the strategy and actually building the solution. Given the target audience of the book, and that these three phases are more tactical in nature, we cover them at an overview level here, but the remainder of the book focuses on developing an AI vision and strategy, subjects that are better suited to the target audience.

The build phase centers on software, hardware, and analytics product development, and includes design, development, and testing at a high level. It assumes all discovery and definition work has been completed and a prioritized roadmap is available to guide the build phase process.

In reality, all phases discussed so far tend to be iterative, and as much as people try, some requirements, use cases, and other defining aspects are usually missed or not fully understood at the outset (Agile, not waterfall). This means that some amount of discovery and definition might continue during the build process. Also, and most important, I recommend an Agile approach where the Five Ds process is carried out on a more granular (feature-level), as-you-go basis, as opposed to performing discovery and definition for an entire roadmap at once (which is more waterfall). A competent product manager and product development team can facilitate this process, and Kanban is my recommended development methodology for this phase.

After you have developed a prioritized roadmap and Agile requirements, design, development, and testing work should follow. Requirements should include both functional and nonfunctional requirements. Functional requirements specify how the solution should work, look, and feel, whereas nonfunctional requirements specify solution requirements around scalability, reliability, and maintainability, for example.

Testing includes software, quality assurance (QA), and user acceptance testing (UAT). Note that any development not requiring designs can begin while design work is happening in parallel with other features (e.g., data preparation, exploratory analytics, predictive modeling).

All experts are involved in the build phase of AIPB—managers, designers, builders, testers, and scientists—with manager involvement centered mostly on leadership, oversight, and collaboration. Product-related roles such as product manager represent the primary management roles from the managers group of experts for the build phase, although other stakeholders in management will continue to participate in the build process through demos, feedback gathering, and status updates.

For the AIPB methodology build phase, and from the AIPB process categories, I recommend the product design and product development categories. Methods, techniques, and best practices associated with these two tend to evolve or change relatively often over time, so the experts involved should make decisions and provide guidance accordingly.

The output of the build phase is a testable solution, and that either represents a part or the entire solution. The output can be from one or multiple categories such as designs, software, hardware, models, and algorithms. The key point is that the output is in a testable state.

Testable design outputs include all relevant design assets such as wireframes, mockups, interactive prototypes, and flow diagrams. Software and hardware outputs include working functionality that can be executed and demonstrated to stakeholders. Analytics outputs can include insights, data visualizations, reports, descriptive analytics, predictive models, or other AI-powered software, models, and engines (e.g., recommendations, natural language processing).

As soon as each roadmap item is built in accordance with the requirements and is successfully tested for maximum quality, it is delivered, either continuously (continuous delivery) or as a release, which we discuss next.

Deliver

The AIPB deliver phase is about delivering high-quality, working solutions to a production environment; that is, an environment in which actual benefactors (business, users, customers) will take advantage of it. In the case of automation, the solution is deployed to automate a real process involving real-world data.

Deploying working solutions to production environments involves specialized experts and processes. Like the build phase, the result of the deployment process has both functional and nonfunctional requirements as well. Functionally, the solution must look and work as expected after it’s deployed. This is usually verified with certain production tests (e.g., smoke tests, QA, UAT). Nonfunctionally, the solution must successfully operate under the real-world conditions to which it will be subjected; for example, the solution must be able to scale as needed and be always available.

The experts involved in the deliver phase of AIPB include mostly builders and testers. For the AIPB methodology deliver phase, and from the AIPB process categories, I recommend the product development and product evaluation, validation, and optimization categories. As in the build phase, experts should employ the latest methods, techniques, and best practices.

The output of the deliver phase is a working solution. It will likely be minimalistic for new innovations, and represent an MVP, PoC, or pilot. I would recommend an Agile and Lean approach as a key strategy in order to ensure iterative validation, pivots, and success as quickly as possible. This means building an MVP first to test the riskiest assumptions, validate product-market fit, ensure that the solution actually accomplishes its goals, and better understand aspects of the UX such as usability and delight (concepts we discuss later in the book). You can continue to build on and improve the MVP with subsequent updates.

The deployed solution should have active health monitoring, logging, and tracking (e.g., tagging, event capture) in which to detect issues and perform analytics. Monitoring and analytics should provide data-driven insights around the health of the solution, whether the solution is providing the intended benefits (to both people and business!) and whether the solution is achieving intended KPIs and to what extent (e.g., success metrics, ROI), and ensuring that all nonfunctional needs are being met (e.g., scalability).

One specific and AI-related item to closely monitor is stale models and model drift. Markets, trends, fads, environments (e.g., economic), competitors, interests, and people change constantly. Data changes constantly as a result, which means that models trained and optimized to a certain performance level on yesterday’s data might not perform well on tomorrow’s data. AI-based solutions typically require ongoing retraining and improvement; or, put another way, continued data gathering, knowledge development, and ongoing learning. This is best facilitated by developing a data feedback loop in which new data generated by the solution is regularly fed back into the system for updated learning and performance improvement.

After you deploy a solution to a real-world environment and are continuously monitoring and analyzing it, as discussed, you need to direct your attention to optimization, the topic of the next section.

Optimize

The optimize phase is the final phase of the AIPB Methodology Component. After you have delivered an AI-based solution with appropriate monitoring and tracking, and you have determined that the solution is worth developing further (e.g., to achieve product-market fit), you should optimize it for better performance and user delight.

Health monitoring and logging should indicate whether any functional or nonfunctional issues need to be fixed or optimized. You should regularly address any signs of stale models and model drift with ongoing learning and performance optimization using a prebuilt data feedback loop. You should regularly analyze KPIs and metrics to determine the efficacy of the solution based on the intended vision and goals and also for discovering any patterns and insights indicating areas of further improvement and lift. You should regularly solicit, capture, and analyze user, customer, and stakeholder feedback in order to further improve and optimize the solution, as well.

Experimentation and testing are also powerful optimization tools that carry the added benefit of helping to determine cause and effect. Correlation does not imply causation, and predictive models are usually unable to uncover all causes leading to specific effects or outcomes. Experimentation techniques such as A/B and multivariate testing are very well suited to continued optimization through strategic experiments and gaining a deeper causal understanding as well. These techniques can also be used to compare the efficacy of different AI models, and generally using AI solutions as compared to the status quo.

Finally, AI and machine learning are rapidly evolving and advancing fields. This is true in all aspects, including research, software, algorithms, and hardware (e.g., deep learning optimized processors such as Graphics Processing Units [GPUs] and Tensor Processing Units [TPUs]). Optimization of AI and machine learning deployments is not only related to better model performance (e.g., predictive accuracy), but also to hardware and training cost reductions, increases in training speeds, reduced computational complexity and resources required, and faster and automated analytics.

The experts involved in the optimize phase of AIPB include everyone. The optimize phase is based on literally everything upstream of it, and any and all aspects can likely be optimized or at least improved. This includes potential involvement of all of the process categories and associated methods from the AIPB process categories. In fact, I’m a huge proponent of kaizen. It’s a Japanese word for improvement with a focus on continuous improvement that was made famous by the Lean manufacturing movement and Toyota’s renowned Toyota Production System (TPS).

The output of the optimize phase is a solution whose intended benefit and outcome performance is well understood and continuously improved in all possible ways; for example, goals, KPIs, lift, user delight, benefits, and performance. Put another way, the outputs are effective analytics and data-driven continuous optimizations.