The Shogun library contains all the necessary classes for the grid search approach. We start by defining the base model. In the Shogun library, we can use the CKernelRidgeRegression class for this purpose. This class implements the polynomial regression model based on the SVM algorithm, and it uses kernels for precise model specialization. Therefore, we can use the polynomial kernel to simulate the polynomial regression model. Also, this kernel type has a configurable hyperparameter called the polynomial degree. The following code sample shows how to create the kernel object:

auto kernel = some<CPolyKernel>(/*cache size*/ 256, /*degree*/ 15);
kernel->init(x, x);

Notice that the kernel object requires our training data for initialization. Despite the fact that data normalization routines are usually incorporated into the algorithm pipelines in the Shogun library, the kernel object requires additional configuration, so we add the normalization object to the kernel object:

auto kernel_normaiizer = some<CSqrtDiagKernelNormalizer>();
kernel->set_normalizer(kernel_normaiizer);

The CKernelRidgeRegression class already has an implementation of L2 (Ridge) regularization. The following code sample shows us how to configure the initial values for the regularization coefficient:

float64_t tau_regularization = 0.00000001;
float64_t tau_regularization_max = 0.000001;
auto model = some<CKernelRidgeRegression>(tau_regularization, kernel, y);

Next, we define the cross-validation object for the grid search. There are several data splitting strategies in the Shogun library that we can use. We have selected the CStratifiedCrossValidationSplitting class here, which implements the same size folds (the data blocks) for splitting. In the following code snippet, we're dividing our dataset into five folds:

auto splitting_strategy = some<CStratifiedCrossValidationSplitting>(y, 5);

We chose MSE as a performance metric. The CMeanSquaredError class implements it:

auto evaluation_criterium = some<CMeanSquaredError>();  

Then, we create the CCrossValidation object and initialize it with the instances of the splitting strategy object and the performance metric object:

auto cross_validation = some<CCrossValidation>(
   model, x, y, splitting_strategy, evaluation_criterium);
cross_validation->set_autolock(false);
cross_validation->set_num_runs(1);

We disabled autolock for the CCrossValidation object because this class does not support this option. This option can speed up the training process in cases where the model supports it. We also only configured one number of runs for the cross-validation process. We did this for demonstration purposes, but for real-life projects, it makes sense to run cross-validation several times.

To define a parameter grid, we use the CModelSelectionParameters class. An object of this class implements a node of a tree that contains a predefined range of values for one hyperparameter. The following code shows how to make a tree of such nodes, which will be an analog for the parameter grid:

auto params_root = some<CModelSelectionParameters>();
auto param_tau = some<CModelSelectionParameters>("tau");
params_root->append_child(param_tau);
param_tau->build_values(tau_regularization, tau_regularization_max,
                        ERangeType::R_LINEAR, tau_regularization_max);
auto param_kernel = some<CModelSelectionParameters>("kernel", kernel);
auto param_kernel_degree = some<CModelSelectionParameters>("degree");
param_kernel_degree->build_values(5, 15, ERangeType::R_LINEAR, 1);
param_kernel->append_child(param_kernel_degree);
params_root->append_child(param_kernel);

Notice that we used the build_values() method to generate a range of values. This method takes the minimum and the maximum values as arguments. After we've configured the cross-validation and the parameter grid (the tree, in our case) objects, we can initialize and run the grid search algorithm. The CGridSearchModelSelection class implements the grid search algorithm:

auto model_selection =
     some<CGridSearchModelSelection>(cross_validation, params_root);
auto best_parameters = model_selection->select_model(/*print_state*/
                                                     true);
best_parameters->apply_to_machine(model);

After instantiating the CGridSearchModelSelection object, we used the select_model() method to search for the best parameter values. Then, we applied them to the model with the apply_to_machine() method. The Shogun library does not guarantee that the model will be in a trained state with the best parameters after model selection. It will in a trained state with the last ones. Therefore, we need to retrain the model with the whole dataset to get better performance:

if (!model->train(x)) {
  std::cerr << "training failed
";
}

After the final training process, the model is ready to be evaluated.