We can interpret ML models as functions that take different types of parameters. Such functions provide outputs for given inputs based on the values of these parameters. Developers can configure the behavior of ML models for solving problems by adjusting model parameters. Training a ML model can usually be treated as a process of searching the best combination of its parameters. We can split the ML model's parameters into two types. The first type consists of parameters internal to the model, and we can estimate their values from the training (input) data. The second type consists of parameters external to the model, and we cannot estimate their values from training data. Parameters that are external to the model are usually called hyperparameters.

Internal parameters have the following characteristics:

• They are necessary for making predictions.
• They define the quality of the model on the given problem.
• We can learn them from training data.
• Usually, they are a part of the model.

If the model contains a fixed number of internal parameters, it is called parametric. Otherwise, we can classify it as non-parametric.

Examples of internal parameters are as follows:

• Weights of artificial neural networks (ANNs)
• Support vector values for SVM models
• Polynomial coefficients for linear regression or logistic regression

On the other hand, hyperparameters have the following characteristics:

• They are used to configure algorithms that estimate model parameters.
• The practitioner usually specifies them.
• Their estimation is often based on using heuristics.
• They are specific to a concrete modeling problem.

It is hard to know the best values for a model's hyperparameters for a specific problem. Also, practitioners usually need to perform additional research on how to tune required hyperparameters so that a model or a training algorithm behaves in the best way. Practitioners use rules of thumb, copying values from similar projects, as well as special techniques such as grid search for hyperparameter estimation.

Examples of hyperparameters are as follows:

• C and sigma parameters used in the SVM algorithm for a classification quality configuration
• The learning rate parameter that is used in the neural network training process to configure algorithm convergence
• The k value that is used in the KNN algorithm to configure the number of neighbors