Linear regression, which is also called simple linear regression, defines the relationship between two variables using a straight line. During linear regression, our aim is to draw a line closest to the data by finding the slope and intercept that define the line. The equation for simple linear regression is generally given as follows:

X is a single feature, Y is a target, and a and b are the intercept and slope respectively. The question is, how do we choose a and b? The answer is to choose the line that minimizes the error function, u. This error function is also known as loss or cost function, which is the sum of the square (to ignore the positive and negative cancelation) of the difference of the vertical distance between the line and the data point.

This calculation is called the Ordinary Least Squares (OLS). Note that explaining every aspect of regression is beyond the scope of this book, and we suggest you explore the Further reading section to broaden your knowledge about the subject.