So far, we have seen how we can leverage Core ML and, in general, machine learning (ML) to better understand the physical world we live in (perceptual tasks). From the perspective of designing user interfaces, this allows us to reduce the friction between the user and the system. For example, if you are able to identify the user from a picture of their face, you can remove the steps required for authentication, as demonstrated with Apple's Face ID feature which is available on iPhone X. With Core ML, we have the potential to have devices better serve us rather than us serving them. This adheres to a rule stated by developer Eric Raymond that a computer should never ask the user for any information that it can auto detect, copy, or deduce.

We can take this idea even further; given sufficient amounts of data, we can anticipate what the user is trying to do and assist them in achieving their tasks. This is the premise of this chapter. Largely inspired and influenced by Google's AutoDraw AI experiment, we will implement an application that will attempt to guess what the user is trying to draw and provide pre-drawn drawings that the user can subtitute with (image search).

In this chapter, we'll explore this idea by looking at how we can try to predict what the user is trying to draw, and find suggestions for them to substitute it with. We will be exploring two techniques. The first is using a convolutional neural network (CNN), which we are becoming familiar with, to make the prediction, and then look at how we can apply a context-based similarity sorting strategy to better align the suggestions with what the user is trying to sketch. In the next chapter, we will continue our exploration by looking at how we can use a recurrent neural network (RNN) for the same task.

By the end of this chapter, you will have: 

• Applied CNNs to the task of sketch recognition 
• Gained further experience preparing input for a model 
• Learned how feature maps can be extracted from CNNs and used to measure how similar two images are

There is a lot to cover, so let's get started by building a simple drawing application.