So you have an Internet smart-toaster and you need to decide how long it should toast; or you have a break-time application and you need to decide when to give users a break; or you have a funny web page app and you need to decide what’s funny. We call the component of a system that makes these types of decisions the “intelligence.” The previous parts of this book helped you identify when you need intelligence
, how to connect it to users through intelligent experiences, how to implement it, and where it should live. This part of the book will help you create intelligence.
Intelligence maps from a context to a prediction about the context. For example, an intelligence might:
- Map from the usage history of a web site to an estimate of the usage in the next week.
- Map from an email message to the probability the email message contains a scam.
- Map from an image to an estimate of how many cucumbers are in the image.
This chapter explores the concepts of context and prediction in more detail. Later chapters will discuss how to create intelligence, how to evaluate it, how to organize it—and more.
An Example Intelligence
But first, let’s look at an example of intelligence in more detail. Imagine a pellet griller.
A what?
Well, a pellet griller
is sort of like a normal outdoor barbecue, but instead of having a big fire that you light and then wait to get to perfect temperature and then put your food on and flip and flip the food and hope the heat isn’t too hot or too cold on account of you waited too long or you didn’t wait long enough to let the fire get to the perfect temperature for cooking… A pellet griller has a bin full of little wooden pellets and it drops them into the flame one at a time as needed to keep the heat at a perfect temperature.
Incredible.
So the intelligence in a pellet griller needs to decide when to add pellets to the fire to keep temperature. Let’s break this down a little bit and explore the context of this intelligence and the predictions.
The pellet griller’s context might include this information:
- The current temperature in the grill.
- The temperature that was in the grill 1 minute ago.
- The number of wood pellets added in the past 5 minutes.
- The number of wood pellets added in the past 1 minute.
- The number of wood pellets added in the past 20 seconds.
- The air temperature outside the grill.
- The type of wood in the pellets.
- The time of day.And so on.
These are the properties that might be relevant to the task of keeping the grill at the perfect temperature. Some of them are obviously important to achieving success (like the current temperature in the grill), and some may or may not help (like the time of day). An intelligence doesn’t have to use everything from the context to make its decisions—but it can.
Based on the information in the context, the intelligence will make predictions. Show it a new context, and the intelligence will predict something about that context. The pellet griller’s intelligence might try to predict:
- If the temperature inside the grill will be hotter or colder one minute in the future.
- What the exact temperature inside the grill will be one minute in the future.
- The probability it should add a pellet to the fire right now to maintain the desired temperature.
Then the intelligent experience would use these predictions to automate the process of adding fuel to the fire. If the fire is going to be too cold one minute in the future—add some fuel. Easy!
Designing the right context and choosing the best thing to predict are important parts of creating effective Intelligent Systems—and getting them right usually requires some iteration and experimentation
.
Contexts
The context
includes all the computer processable information your intelligence might use to make its decisions. The intelligence is free to pick and choose which pieces of the context to use to make the best decisions. In that sense, the context is the palette of options the intelligence creator can choose from when creating intelligence.
In order to be useful, the context must be:
- Implemented in the intelligence runtime.
- Available to the intelligence creator to use in producing and evaluating intelligence .
Implemented at Runtime
To make a piece of information available for the intelligence, someone needs to do the work to hook the information into the intelligence runtime
.
For example, the pellet griller might be smarter
if it knew the temperature outside of the grill. But it takes work to know the temperature outside of the grill. Someone needs to bolt a temperature sensor onto each grill, test it, run some wires, write a driver, poll the sensor and copy its reading into the intelligence runtime every few seconds.
Common forms of context
include these:
- Information about what is going on in the system at the time: For example, what other programs are running, what is the current status of the things the system can control (are the lights on or off?), and what is the user seeing on their screen? Using this type of context is generally straightforward, but it does need to be implemented.
- Properties of the content the user is interacting with: For example, what genre is the song, what is its name, where did it come from, what words are on the web page, and what pixels are in the picture? This type of context usually requires processing the content to extract information from it or doing lookups to learn more about the content (such as looking up properties about a song in an external database).
- History of user interactions: How has the user has interacted with the Intelligent System in the past? To use these interactions, the system needs to monitor usage, aggregate it, and persist it over time.
- Properties about the user: These include age, interests, gender. To use these properties , the system needs to gather the information from the user and store it.
- Any relevant sensor readings: Which require the sensors to be implemented into the hardware and sensor readings to be provided to the runtime .
Deciding what to put into the context requires balancing the value of the information with the cost of implementation
.
One best practice is to start by creating a context that includes everything that is cheap to include—the information that is already near the intelligence runtime
as part of building the system in the first place, such as information about what is going on in the system, and properties of the content the user is interacting with.
This might be enough to get going. But expect to continually augment the context as you push to better and better intelligence.
It can also be useful to add some speculative things to the context. For example, maybe it would help to know how long the grill has been on. Maybe near the beginning of a grilling session the metal of the grill is cold and it takes more fuel to heat the thing up; maybe after one hour of grilling everything is so hot that you don’t need as much fuel. Or maybe it doesn’t matter.
Including information like this in the context can help your users show you what matters. If the variable is relevant to getting good outcomes, you’ll see it in the telemetry and add it to the intelligence. If not, you can always remove
it from the context later.
Available for Intelligence Creation
To create effective intelligence
from context
, someone needs to do the work to get the information into the intelligence creator’s hands—probably in the form of telemetry.
Say you’ve shipped tens of thousands of pellet grillers. Your intelligence is running all over the world. Pellets are being added to fires in Florida, in Norway, in Zimbabwe. Grill temperatures are going up and grill temperatures are going down. How are you going to use all of this information to create better intelligence?
Someone is going to have to collect all the grilling contexts and get it to you. And the contexts need to be connected to the outcomes. For example, imagine a grilling session taking place in someplace-USA. If the pellet griller’s intelligence runtime knows:
- The current temperature in the grill is 290 degrees.
- The temperature that was in the grill 1 minute ago was 292 degrees.
- The number of pellets added in the past 5 minutes was 7.
- The number of pellets added in the past 1 minute was 2.
- The number of pellets added in the past 20 seconds was 0.
- The air temperature outside of the grill is 82 degrees.
- And the outcome: The temperature in the grill gets 3 degrees hotter over the following minute.
And you collect information like this from tens of thousands of grilling sessions, and get it all back to your intelligence creators—you are going to be able to make some fantastic grilling intelligence
.
Data for creating intelligence doesn’t have to come from your users, but, as we’ve discussed in chapters on getting data from experience and telemetry, an Intelligent System works best when the data does come from actual usage.
The data used to create intelligence must be very similar to the data that is present in the intelligence runtime. If they are out of sync, the intelligence will behave differently for you and for your customers—which could be a big problem
.
Things Intelligence Can Predict
Intelligence can make predictions
about the contexts it encounters. These predictions are usually:
- Classifications of the context into a small set of possibilities or outcomes.
- Estimations of probabilities about the context or future outcomes.
- Regressions that predict numbers from the context.
- Rankings which indicate which entities are most relevant to the context.
- Hybrids and combinations of these.
This section will explore these concepts in more detail.
Classifications
A classification is a statement from a small set of possibilities. It could be a statement about the context directly, or it could be a prediction
of an outcome that will occur based on the context. For example:
- Based on the context, classify the grill as:
- Too hot
- Too cold
- Just right
- Based on the context, classify the movie as:
- A horror flick
- A romantic comedy
- An adventure
- A documentary
- Based on the context, classify the picture as:
- A cow
- A red balloon
- Neither a cow or a red balloon—something else
Classifications are commonly used when there are a small number of choices: two, five, a dozen.
Classifications are problematic when:
- There are many possible choices—when you have hundreds or thousands of possibilities. In these situations you might need to break up the problem into multiple sub-problems or change the question the intelligence is trying to answer.
- You need to know how certain the prediction is—for example, if you want to take an action when the intelligence is really certain. In this case, consider probability estimates instead of classifications.
Probability Estimates
Probability estimations predict the probability
the context is of a certain type or that there will be a particular outcome. Compared to classifications, a probability estimation is less definitive, but more precise. A classification would say “it is a cat”; a probability estimation would say “it is 75% likely to be a cat.”
Other examples of probability estimates include these:
- The web page is 20% likely to be about politics, 15% likely to be about shopping, 10% likely to be a scam, and so on.
- The user is 7% likely to click accept if we offer to reformat their hard drive.
- There is a 99% chance it will rain next week.
Probability estimation are commonly used with one or more thresholds. For example:
if(predictedProbability > 90%)
{
IntelligentExperience->AutomateAnAction();
}
else if(predictedProbability > 50%)
{
IntelligentExperience->PromptTheUser();
}
else
{
// Do nothing...
}
In this sense, probabilities contain more information
than classifications. You can turn a probability into a classification using a threshold, and you can vary the threshold over time to tune your intelligence.
Most machine-learning algorithms
create probabilities (or scores, which are similar to probabilities) internally as part of their models, so it is very common for Intelligent Systems to use probability estimates instead of classifications.
Probability estimations are problematic when:
- As with classifications, probabilities don’t work well when there are many possible outcomes.
- You need to react to small changes: Slight changes in the context can cause probabilities to jitter. It is common to smooth the output of multiple sequential probabilities to reduce jitter (but this adds latency). It is also good practice to quantize probabilities unless you really, really need the detail.
Also note that probabilities
usually aren’t actually probabilities. They are more like directional indicators. Higher values are more likely; lower values are less likely. An intelligence might predict 90%, but that doesn’t mean the outcome will happen 9 out of 10 times—unless you carefully calibrate your intelligence. Be careful when interpreting probabilities.
Regressions
Regressions
are numerical estimates about a context, for example:
- The picture contains 6 cows.
- The manufacturing process will have 11 errors this week.
- The house will sell for 743 dollars per square foot.
Regressions allow you to have more detail in the answers you get from your intelligence. For example, consider an intelligence for an auto-pilot for a boat.
- A classification might say, “The correct direction is right.”
- A probability might say, “The probability you should turn right is 75%.”
- A regression might say, “You need to turn 130 degrees right.”
These convey very different information. All three say “right,” but the regression also conveys that you have a long way to go – you better start spinning that wheel!
Regressions are problematic when:
- You need to react to small changes: Slight changes in the context can cause regressions to jitter. It is common to smooth the output of multiple sequential regressions to reduce jitter (but this adds latency).
- You need to get training data from users: It is much easier to know “in this context, the user turned right” than to know “in this context the user is going to turn 114 degrees right.”
Classifications can be used to simulate regressions. For example, you could try to predict classifications with the following possibilities:
- “Turn 0 - 10 degrees right.”
- “Turn 11 - 45 degrees right.”
- “Turn 46 - 90 degrees right.”
- And so on.
This quantizes the regression
and may be simpler to train and to use.
Rankings
Rankings
are used to find the items most relevant to the current context:
- Which songs will the user want to listen to next?
- Which web pages are most relevant to the current one?
- Which pictures will the user want to include in the digital scrap-book they are making?
Ranking have been successfully used with very large numbers of items, like every web page on the Internet, every movie in a digital media service, or every product in an e-commerce store
.
Rankings are commonly used when there are many possible relevant entities and you need to find the top few.
Rankings can be thought of using probabilities
. Take each item, find the probability it is relevant to the current context, and “rank” the items in order of the probability estimates (but actual ranking algorithms are more complex than this).
Hybrids and Combinations
Most intelligences produce classifications, probability estimations, regressions, or rankings. But combinations
and composite answers are possible.
For example, you might need to know where the face is in an image. You could have one regression that predicts the X location of the face and another that predicts the Y location, but these outputs are highly correlated—the right Y answer depends on which X you select, and vice versa. It might be better to have a single regression with two simultaneous outputs, the X location of the face and the Y location.
Summary
Intelligence is the part of the system that understands the contexts users encounter and makes predictions about the contexts and their outcomes.
Context is all the information available to the intelligence to make its determinations. An intelligence doesn’t have to use all the parts of the context, but it can.
In order to be used as part of the context, information needs to be hooked into the intelligence runtime, and it needs to be available to the intelligence creators.
The intelligence runtime and the information available to intelligence creators must be exactly the same. Any differences could lead to hard-to-find problems.
Intelligence can give many forms of predictions, including:
- Classifications, which map contexts to a small number of states.
- Probability estimations, which predict the probability a context is in a particular state or a particular outcome will occur.
- Regressions, which estimate a number from contexts.
- Rankings, which find content that is relevant to the current context.
- Some hybrid or combination that combines one or more of these.
For Thought…
After reading this chapter, you should:
- Be able to describe a context as used in intelligence, including how to enable it in an intelligence runtime and in training.
- Understand the types of predictions an intelligence can make and when the various options are strong or weak.
You should be able to answer questions like these:
- Choose the Intelligent Systems you like best and come up with 20 things that might be part of its context.
- Which of the 20 things you used in your last answer would be the hardest to use as context in an Intelligent System? Why?
- Among the Intelligent Systems you interact with, find an example of intelligence that does classification, an example of regression, and an example of ranking.