These types of machine learning models are used to predict the outcome based upon the data presented to it. The instructions provided are explicit and detailed, or at least should be, which is what has garnered the label supervised. We are basically learning a function which maps an input to an output based upon input and output pairs. This function is inferred from training data which is called labeled, in that it specifically tells the function what it expects. In supervised learning, there is always an input and corresponding output (or more correctly, a desired output value). More formally, this type of algorithm uses a technique known as inductive bias to accomplish this, which basically means that there are a set of assumptions which the algorithm will use to predict the outputs given inputs it may or may not have previously seen.
In supervised learning we typically have access to a set of X features (X1, X2, X3, ... Xx), measured on observations, and a response Y, also measured on those same n observations. We then try and predict Y using X1, X2, X3, ... Xn.
Models such as Support Vector Machines (SVM), linear regression, Naive Bayes, and tree-based methods are just a few examples of supervised learning.
Next, let's briefly discuss a few things which we need to concern ourselves with when it comes to supervised learning. They are, in no particular order:
- Bias-variance trade-off
- Amount of training data
- Input space dimensionality
- Incorrect output values
- Data heterogeneity