In this chapter, we learned how to save and load model parameters in different ML frameworks. We saw that all the frameworks we used in the Shogun, Shark-ML, Dlib, and PyTorch libraries have an API for model parameter serialization. Usually, these are quite simple functions that work with model objects and some input and output streams. Also, we discussed another type of serialization API that can be used to save and load the overall model architecture. At the time of writing, the frameworks we used don't fully support such functionality. The Shogun toolkit can load neural network architectures from the JSON descriptions, but can't export them. The Dlib library can export neural networks in XML format but can't load them. The PyTorch C++ API lacks a model architecture that supports exporting, but it can load and evaluate model architectures that have been exported from the Python API with its TorchScript functionality. However, the PyTorch library does provide access to the Caffe2 library API, which allows us to load and evaluate models saved in ONNX format from C++.
We briefly looked at ONNX format and realized that it is quite a popular format for sharing models among different ML frameworks. It supports almost all operations and objects that are used to serialize complex neural network models effectively. At the time of writing, it is supported by all popular ML frameworks, such as TensorFlow, PyTorch, MXNet, and others. Also, Microsoft provides the ONNX runtime implementation, which allows us to run the ONNX model's inference without us having to depend on any other frameworks.
And at the end of this chapter, we developed a C++ application that can be used to run the inference of the ResNet-50 model, which was trained with the MXNet framework and exported in ONNX format. This application was made with the Caffe2 C++ API in order to load the model and evaluate it on the loaded image for classification.
In the next chapter, we will discuss how to deploy ML models developed with C++ libraries to mobile devices and server instances.