So you have a problem you want to solve. An existing system you need to optimize. A new idea to create a business. Something you think your customers will love. A machine-learning–based solution that isn’t producing the value you’d hoped. Is an Intelligent System right for you? Sometimes yes. Sometimes no.
This chapter discusses when Intelligent Systems might be the right approach, and provides guidance on when other approaches might be better. It begins by describing the types of problems that can benefit from Intelligent Systems. It then discusses some of the properties required to make Intelligent Systems work.
Types of Problems That Need Intelligent Systems
It is always best to do things the easy way. If you can solve your problem without an Intelligent System, maybe you should. One key factor in knowing whether you’ll need an Intelligent System is how often you think you’ll need to update the system before you have it right. If the number is small, then an Intelligent System is probably not right.
For example, imagine implementing intelligence for part of a banking system. Your “intelligence” needs to update account balances as people withdraw money. The solution is pretty simple:
NewBalance = OldBalance - WithdrawalAmount
Something like that (and a bunch of error checking and logging and transaction management), not rocket science. Maybe you’ll have a bug. Maybe you’ll miss an error case (for example, if the balance goes negative). Then you might need to update the “intelligence” to get it right. Maybe a couple times? Or maybe you’re super sloppy and you need to update it a dozen times before you’re done.
Intelligent Systems are not for problems like this. They are for problems where you think you’re going to have to update the system much, much more. Thousands of times, tens of thousands of times, every hour for as long as the system exists. That kind of thing.
There are four situations that clearly require that level of iteration
:
- Big problems, that require a lot of work to solve.
- Open-ended problems, which continue to grow over time.
- Time-changing problems, where the right answer changes over time.
- Intrinsically hard problems, which push the boundaries of what we think is possible.
The rest of this section will explore these each in turn.
Big Problems
Some problems are big
. They have so many variables and conditions that need to be addressed that they can’t really be completed in a single shot
.
For example, there are more web pages than a single person could read in their lifetime—more than a hundred people could read. There are so many books, television programs, songs, video games, live event streams, tweets, news stories, and e-commerce products that it would take thousands of person-years just to experience them all.
These problems and others like them require massive scale. If you wanted to build a system to reason about one of these, and wanted to completely finish it before deploying a first version… Well, you’d probably go broke trying.
When you have a big problem that you don’t think you can finish in one go, an Intelligent System
might be a great way to get started, and an efficient way to make progress on achieving your vision, by giving users something they find valuable and something they are willing to help you improve.
Open-Ended Problems
Some problems are more than big. Some problems are open-ended
. That is, they don’t have a single fixed solution at all. They go on and on, requiring more work, without end. Web pages, books, television programs, songs, video games, live event streams—more and more of them are being created every day.
Trying to build a system to reason about and organize things that haven’t even been created yet is hard.
In these cases, a static solution—one where you build it, deploy it, and walk away—is unlikely to work. Instead, these situations require services that live over long periods of time and grow throughout their lifetimes.
If your problem has a bounded solution, an Intelligent System
might not be right. But if your problem is big and on-going, an Intelligent System might be the right solution.
Time-Changing Problems
Things change
. Sometimes the right answer today is wrong tomorrow. For example:
- Imagine a system for identifying human faces—and then facial tattoos become super popular.
- Imagine a system for predicting stock prices—and then an airplane crashes into a building.
- Imagine a system for moving spam email to a junk folder—and then a new genius-savant decides to get in the spam business and changes the game.
- Or Imagine a UX that users struggle to use—and then they begin to learn how to work with it.
One thing’s for certain—things are going to change.
Change means that the intelligence you implemented
yesterday—which was totally right for what was happening, which was making a lot of users happy, maybe even making your business a lot of money—might be totally wrong for what is going to happen tomorrow.
Addressing problems that change over time requires the ability to detect that something has changed and to adapt quickly enough to be meaningful.
If your domain changes slowly or in predictable ways, an Intelligent System might not be needed. On the other hand, if change in your domain is unpredictable, drastic, or frequent, an Intelligent System might be the right solution for you.
Intrinsically Hard Problems
Some problems are just hard. So hard that humans can’t quite figure out how to solve them. At least not all at once, not perfectly. Here are some examples of hard problems:
- Understanding human speech.
- Identifying objects in pictures.
- Predicting the weather more than a few minutes in the future (apparently).
- Competing with humans in complex, open-ended games.
- Understanding human expressions of emotion in text and video.
In these situations, machine learning has had great success, but this success has come on the back of years (or decades) of effort, gathering training data, understanding the problems, and developing intelligence. These types of systems are still improving and will continue to improve for the foreseeable future
.
There are many ways to make progress on such hard problems. One way is to close the loop between users and intelligence creation in a meaningful application using an Intelligent System as described in this book.
Situations When Intelligent Systems Work
In addition to having a problem that is difficult enough to need an Intelligent System—one that is big, open-ended, time-changing, intrinsically hard, or some combination—an Intelligent System needs a few more things to succeed:
- A problem where a partial solution is viable and interesting.
- A way to get data from usage of the system to improve intelligence.
- An ability to influence a meaningful objective.
- A problem that justifies the effort of building an Intelligent System.
The rest of this section explores these requirements in turn.
When a Partial System Is Viable and Interesting
Intelligent Systems are essentially always incomplete, incorrect in important ways, and likely to make all sorts of mistakes. It isn’t important that an Intelligent System is perfect. What matters is that an Intelligent System must be good enough to be interesting to users (and valuable to you).
An Intelligent System is viable when the value of the things it does right is (much) higher than the cost of the things it does wrong.
Consider an Intelligent System that makes “cheap” mistakes. Maybe the system is supposed to show a shopping list in an order that optimizes the user’s time in the grocery store. Head to the right, pick up eggs, then milk, then loop to the back of the store for the steak, then into the next aisle for pickles… and on and on. If this system gets a few things backwards it might waste time by sending users in the wrong direction, say 15–20 seconds per mistake. Irritating, but not the end of the world. If the mistakes aren’t too frequent—say just one every other shopping trip—it’s easy to imagine someone using the system while it is learning the layouts of all the grocery stores in the world and adapting to changes.
Consider another Intelligent System, one that makes “expensive” mistakes. Maybe it is controlling surgical robots, and mistakes put human life at risk. This system would have to meet a much higher quality bar before it is viable. That is, you’d need to produce a much better partial system
before deploying it to users. But even this system doesn’t need to be perfect to be viable. Remember—surgeons make mistakes, too. An Intelligent System doesn’t have to be perfect (and most never will be), it just has to be better than the other guy.
The point is that to be viable, an Intelligent System must provide a good deal for users (and for you) by producing more value with positive outcomes than it produces irritation (and cost) with negative outcomes.
When You Can Use Data from the System to Improve
Intelligent Systems work by closing the loop between usage and intelligence. When users use the system, the intelligence gets better; and when the intelligence gets better, the system produces more value for users
.
To make intelligence better with usage, you need to record the interactions between users and the systems in a careful way (which we will discuss in detail later). At a high level, this involves capturing what the user saw, what they did, and whether the outcome was positive or negative. And you need to observe many, many user interactions (the harder the problem, the more you’ll need). This type of data can be used to improve intelligence in many ways, including with machine learning.
When the System Can Interface with the Objective
The Intelligent System must be able to change things that influence the objective. This could be through automating a part of the system, or presenting information or options to a user that help them achieve their goals
. Intelligent Systems are most effective when the following conditions are met:
- The actions the Intelligent System can take directly affect the objective. When there is a positive outcome, it should be largely because the Intelligent System did well; and when there is a negative outcome, it should be largely because the Intelligent System did poorly. The more external factors that affect the outcome, the harder it is to make an effective Intelligent System.
- The actions the Intelligent System can take quickly affect the objective. Because the more time between the action and the outcome, the more chance there is for external factors to interfere.
- The actions the Intelligent System can take are in balance with the objectives. That is, the actions the Intelligent System can take are meaningful enough to the outcome that they are usually measurable.
In practice, figuring out where to put Intelligent Systems and how much of your problem to try to solve with them will be key challenges. Too big an objective and you won’t be able to connect the Intelligent System to it; too small and it won’t be worth all the work.
In fact, many large, complex systems have space for more than one Intelligent System inside them. For example, an operating system might have an Intelligent System to predict what to put into a RAM cache, one to predict what files to show the user when they search, one to recommend apps to users based on usage, and another to manage the tradeoffs between power and performance when the computer is not plugged in.
You can imagine using a single Intelligent System to manage an entire OS. But it probably wouldn’t work, because:
- There are too many things to control.
- The controls are too abstract from the goal.
- The types of feedback the system gets would not relate directly enough to the decisions the Intelligent System needs to make .
When it is Cost Effective
Intelligent Systems have different costs than other approaches to building computer systems and services. There are three main components of cost in most similar systems: The Intelligence, the Implementation, and the Orchestration. We will discuss these concepts in Parts 3, 4, and 5 of this book. But roughly Intelligence is the ways the system makes decisions about what is right to do and when, the “logic” of the system; Implementation is all the services, back-end systems, and client code that implements the system and interfaces intelligence outcomes with users; and Orchestration is the managing of the system throughout its life-cycle, such as making sure it is achieving its objectives, and dealing with mistakes.
The intelligence is usually cheaper in Intelligent Systems than other approaches
, for a couple of reasons:
- 1.In general, Intelligent Systems produce the complex logic of a system automatically (instead of using humans). When addressing a big, hard, or open-ended task, Intelligent Systems can lead to substantial savings on intelligence production.
- 2.With the proper intelligent experience, Intelligent Systems produce the training data (which machine learning requires) automatically as users interact with it. Gathering training data can be a major cost driver for intelligence creation, often requiring substantial human effort, and so getting data from users can drastically shift the economics of a problem.
The implementation of an Intelligent System is similar to or a bit more expensive than the implementation other service-based systems (such as a web service or a chat service). Most of the components are similar between an Intelligent System and a nonintelligent system, but Intelligent Systems have additional requirements related to orchestrating them over their lifecycles, including creating intelligence, changing intelligent experiences, managing mistakes, organizing intelligence, and so on. We will discuss all of these in detail later in the book.
The orchestration costs of an Intelligent System are similar but a bit more expensive compared to running a non-intelligent service (like a web service or a chat service). Intelligent Systems require ongoing work, including tuning the way the intelligence is exposed to users, growing the intelligence to be better and deal with new situations, and identifying and mitigating mistakes.
By the time you’ve finished this book you’ll be able to prepare a detailed analysis of the efficiency of using an Intelligent System for your application.
When You Aren’t Sure You Need an Intelligent System
Sometimes you have a place where users, data, and intelligence interact, but you aren’t sure you need all the elements of an Intelligent System
. This might be for any of these reasons:
- You aren’t sure your problem is actually hard, so you don’t want to build a whole bunch of fancy intelligence you might not need.
- You realize the problem is hard, but aren’t sure the extra work of building an Intelligent System will justify the effort for you.
- You want to take an incremental approach and solve problems as you encounter them, rather than stopping to build an Intelligent System.
That’s fine. Intelligent Systems (as described in this book) aren’t right for everyone—there are plenty of ways to build systems that solve hard problems. But if you do try to solve a big, open-ended, time-changing, or hard problem, you’ll eventually run into many of the challenges described in this book. Even if you choose not to build an Intelligent System as described here, this book will arm you to identify common issues quickly, understand what is going on, and will prepare you to respond with proven approaches.
Summary
Intelligent Systems are most useful when solving big, open-ended, time-changing, or intrinsically hard problems. These are problems where a single solution won’t work, and where the system needs to improve over time—months or years.
Intelligent Systems work well when:
- A partial solution is interesting to users and will support a viable business, so that you can start simply and grow the system’s capability over time.
- You can get data to improve the system from users as they use the system. Data is usually one of the main costs in building intelligence, so crafting a system that produces it automatically is very valuable.
- When the things the system can control affect the objectives: directly, quickly, and meaningfully. That is, the outcomes the system is getting can be tied to things the Intelligent System is doing.
- When other approaches are too expensive to scale.
When looking for places to use Intelligent Systems, break down your problem into pieces. Each piece should have a clear objective that is critical to overall success, feedback that is quick and direct, and controls (or user experiences your system can control) that directly impact the objective. Each such piece is a candidate for an Intelligent System.
And even if you choose not to use an Intelligent System at all, the patterns in this book about practical machine learning, interfacing intelligence with user experience, and more will benefit just about any machine learning endeavor.
For Thought
After reading this chapter, you should know:
- Whether your system could benefit from an Intelligent System.
- How to find potential Intelligent Systems in the world around you.
You should be able to answer questions like these:
- What is your favorite hobby that would benefit from an Intelligent System?
- What is an example of a common activity you do every day where an Intelligent System might not be right? Why not?
Consider your favorite piece of software:
- Identify three places the software could use an Intelligent System.
- Which one is the most likely to succeed? Which the least? Why?