The Shogun library can save model parameters in different file formats such as ASCII, JSON, XML, and HDF5. This library can't load model architectures from a file and is only able to save and load the weights of the exact model. But there is an exception for neural networks: the Shogun library can load a network structure from a JSON file. An example of this functionality is shown in the following example.

As in the previous example, we start by generating the training data:

const int32_t n = 1000;
SGMatrix<float64_t> x_values(1, n);
SGVector<float64_t> y_values(n);

std::random_device rd;
std::mt19937 re(rd());
std::uniform_real_distribution<double> dist(-1.5, 1.5);

// generate data
for (int32_t i = 0; i < n; ++i) {
    x_values.set_element(i, 0, i);
    
    auto y_val = func(i) + dist(re);
    y_values.set_element(y_val, i);
}

auto x = some<CDenseFeatures<float64_t>>(x_values);
auto y = some<CRegressionLabels>(y_values);

// rescale
auto x_scaler = some<CRescaleFeatures>();
x_scaler->fit(x);
x_scaler->transform(x, true);

We filled the x object of the CDenseFeatures type with the predictor variable values and the y object of the CRegressionLabels type with the target variable values. The linear dependence is the same as what we used in the previous example. We also rescaled the x values with the object of the CRescaleFeatures type.

To show how the serialization API in the Shogun library works, we will use the CLinearRidgeRegression and CNeuralNetwork models.

The following code sample shows how to train and serialize the CLinearRidgeRegression model:

void TrainAndSaveLRR(Some<CDenseFeatures<float64_t>> x,
                     Some<CRegressionLabels> y) {
    float64_t tau_regularization = 0.0001;
    auto model =
    some<CLinearRidgeRegression>(tau_regularization, nullptr, nullptr);
    model->set_labels(y);
    if (!model->train(x)) {
        std::cerr << "training failed
";
    }
    
    auto file = some<CSerializableHdf5File>("shogun-lr.dat", 'w');
    if (!model->save_serializable(file)) {
        std::cerr << "Failed to save the model
";
    }
}

Here, we saved the model parameters in the HDF5 file format with an object of the CSerializableHdf5File type. For other file formats, we can find the corresponding types in the library. All serializable models in the Shogun library have the save_serializable and load_serializable functions for saving and loading model parameters, respectively. These functions take a serialization file object. In this case, this object was of the CSerializableHdf5File type.

The following code shows how to train and save the parameters of a neural network object:

void TrainAndSaveNET(Some<CDenseFeatures<float64_t>> x,
                     Some<CRegressionLabels> y) {
    auto dimensions = x->get_num_features();
    auto layers = some<CNeuralLayers>();
    layers = wrap(layers->input(dimensions));
    layers = wrap(layers->linear(1));
    auto all_layers = layers->done();
    
    auto network = some<CNeuralNetwork>(all_layers);
    // configure network parameters
    ...

    network->set_labels(y);
    if (network->train(x)) {
        auto file = some<CSerializableHdf5File>("shogun-net.dat", 'w');
        if (!network->save_serializable(file)) {
            std::cerr << "Failed to save the model
";
        }
    } else {
        std::cerr << "Failed to train the network
";
    }
}

Here, we can see that neural network serialization is similar to serializing the linear regression model.

To test our serialized model, we will generate a new set of test data. The following code shows how we do this:

SGMatrix<float64_t> new_x_values(1, 5);
std::cout << "Target values : 
";
for (index_t i = 0; i < 5; ++i) {
    new_x_values.set_element(static_cast<double>(i), 0, i);
    std::cout << func(i) << std::endl;
}

auto new_x = some<CDenseFeatures<float64_t>>(new_x_values);
x_scaler->transform(new_x, true);

Notice that we reused the x_scaler object. But as we mentioned previously, it should be serialized and loaded too if we plan to stop and relaunch our application after training.

The following code shows the deserialization process:

void LoadAndPredictLRR(Some<CDenseFeatures<float64_t>> x) {
    auto file = some<CSerializableHdf5File>("shogun-lr.dat", 'r');
    auto model = some<CLinearRidgeRegression>();
    if (model->load_serializable(file)) {
        auto y_predict = model->apply_regression(x);
        std::cout << "LR predicted values: 
" << y_predict->to_string() << 
            << std::endl;
    }
}
...
LoadAndPredictLRR(new_x);

Here, the new CLinearRidgeRegression object was created and the load_serializable method was used to load its parameter.

As we mentioned previously, there is a particular function that's used to load a neural network structure from JSON files or strings. The problem is that we can't export this structure as a file with the library API, so we should make it by ourselves or write our custom exporter. However, this functionality allows us to define neural network architectures without programming in a declarative style. This can be useful for experiments because we don't need to recompile a whole application. It also allows us to deploy a new architecture to production without program updates, but note that we need to take care of preserving the input and output network tensor dimensions.

To load the neural network from the JSON-formatted string in the Shogun library, we can use an object of the CNeuralNetworkFileReader type. The following code shows how to use it:

Some<CNeuralNetwork> NETFromJson() {
    CNeuralNetworkFileReader reader;
    const char* net_str =
    "{"
        ""optimization_method": "NNOM_GRADIENT_DESCENT","
        ""max_num_epochs": 1000,"
        ""gd_mini_batch_size": 0,"
        ""gd_learning_rate": 0.01,"
        ""gd_momentum": 0.9,"
        
        ""layers":"
        "{"
            ""input1":"
            "{"
                ""type": "NeuralInputLayer","
                ""num_neurons": 1,"
                ""start_index": 0"
                "},"
            ""linear1":"
            "{"
                ""type": "NeuralLinearLayer","
                ""num_neurons": 1,"
                ""inputs": ["input1"]"
                "}"
            "}"
        "}";
    auto network = wrap(reader.read_string(net_str));
    
    return network;
}

Here, we defined the neural network architecture with a JSON string. This is the same neural network architecture that we used for training. Then, the read_string method of the CNeuralNetworkFileReader object was used to load and create an object of the CNeuralNetwork type.

The following code shows how to use the NETFromJson function to create a network object from the JSON string and initialize it with the serialized parameters:

void LoadAndPredictNET(Some<CDenseFeatures<float64_t>> x) {
    auto file = some<CSerializableHdf5File>("shogun-net.dat", 'r');
    
    auto network = NETFromJson();
    
    if (network->load_serializable(file)) {
        auto new_x = some<CDenseFeatures<float64_t>>(x);
        auto y_predict = network->apply_regression(new_x);
        std::cout << "Network predicted values: 
"
        << y_predict->to_string() << std::endl;
    }
}

The newly created neural network object is of the CNeuralNetwork type. We used the load_serializable method of the new neural network object to load the previously serialized parameters. It's essential to preserve the same architecture of ML model objects that are used for serialization and deserialization as a different architecture can lead to runtime errors when deserialization is performed.

In this section, we looked at how to use the Shogun library API for serialization. This library doesn't provide any functions that can be used to export ML model architectures, but it can load them from a JSON string. ML model parameters can be serialized into various file formats. In the next section, we will delve into the Shark-ML library's serialization API.