Due to the fact that there are several parameters to tune for an algorithm, exhausting all algorithms and fine-tuning each one can be extremely time-consuming and computationally expensive. We should instead shortlist one to three algorithms to start with using the general guidelines that follow (note we herein focus on classification, but the theory transcends in regression and there is usually a counterpart algorithm in regression).

There are several things we need to be clear about before shortlisting potential algorithms, as described in the following:

• Size of the training dataset
• Dimensionality of the dataset
• Whether the data is linearly separable
• Whether features are independent
• Tolerance and trade-off of bias and variance
• Whether online learning is required