This section will take a look at the ML ecosystem in Go, first discussing the essentials we want from a library, and then assessing each of the main Go ML libraries in turn.

Go's ML ecosystem has historically been quite limited. The language was introduced in 2009, well before the DL revolution that has brought many new programmers into the fold. You might assume that Go has seen the growth in libraries and tools that other languages have. History, instead, determined that many of the higher-level APIs for the mathematical operations that underpin our networks have appeared as Python libraries (or have complete Python bindings). There are numerous well-known examples of this, including PyTorch, Keras, TensorFlow, Theano, and Caffe (you get the idea).

Unfortunately, these libraries have either zero or incomplete bindings for Go. For example, TensorFlow can do inference (Is this a cat or not?), but not training (What is a cat anyway? Oh, okay, I'll take a look at these examples and figure it out). While this takes advantage of Go's strengths when it comes to deployment (compilation down to single binary, compiler speed, and low memory footprint), from a developer's perspective, you're then forced to work across two languages (Python for training your model and Go for running it).

Issues you may face, beyond the cognitive hit of switching syntax when designing, implementing, or troubleshooting, extend to environment and configuration problems. These problems include questions such as: Is my Go environment configured properly? Is my Python 2 binary symlinked to Python or is it Python 3? Is TensorFlow GPU working properly? If our interest is in designing the best model and getting it trained and deployed in the minimum amount of time, none of the Python or Go bindings libraries are suitable.

It is important, at this point, to ask: so, what do we want out of a DL library in Go? Quite simply, we want as much unnecessary complication abstracted away as possible while preserving flexibility and control over our model and how it is trained.

What does this mean in practice? The following list outlines the answers to this query:

• We do not want to interface with Basic Linear Algebra Subprograms (BLAS) directly to construct basic operations such as multiplication and addition.
• We do not want to define a tensor type and associated function(s) each time we implement a new network.
• We do not want to write an implementation of Stochastic Gradient Descent (SGD) from scratch each time we want to train our network.

The following are some of the things that will be covered in this book:

• Automatic or symbolic differentiation: Our DNN is trying to learn some function. It iteratively solves the problem of what is the function that will take an input image and output the label cat? by calculating the gradient (the gradient descent optimizations) with respect to the loss (how wrong is our function?). This allows us to understand whether to change the weights in our network and by how much, with the specific mode of differentiation breaking up the calculation of these gradients (effectively using the chain rule), giving us the performance we need to be able to train deep networks with millions of parameters.
• Numerical stabilization functions: This is essential for DL, as we will explore in later sections of this book. A primary example of this is Batch Normalization or BatchNorm, as the attendant function is often called. It aims to put our data on the same scale to increase training speed, and it reduces the possibility of maximum values cascading through the layers and causing gradient explosion (something we will discuss in greater detail in Chapter 2, What is a Neural Network and How Do I Train One?).
• Activation functions: These are mathematical operations that introduce nonlinearities into the various layers in our neural network and help to determine which neurons in a layer will be activated, passing their values down to the next layer in the network. Examples include Sigmoid, Rectified Linear Unit (ReLU), and Softmax. These will be considered in greater detail in Chapter 2, What is a Neural Network and How Do I Train One?
• Gradient descent optimizations: We will also cover these extensively in Chapter 2, What is a Neural Network and How Do I Train One? But, as the primary optimization method used in DNNs, we consider this a core function necessary for any library to have DL as its purpose.
• CUDA support: Nvidia's drivers allow us to offload the fundamental operations involved in our DNNs to our GPU. GPUs are really great for parallel workloads involving matrix transformations (indeed, this was their original purpose: computing the world-geometry of games) and can reduce the time it takes to train your model by an order of magnitude or more. Suffice to say, CUDA support is essential for modern DNN implementations and is therefore available in the major Python libraries described previously.
• Deployment tools: As we will cover in Chapter 9, Building a Deep Learning Pipeline, deployment of a model for training or inference is often overlooked in discussions about DL. With neural network architectures growing more complex, and with the availability of vast amounts of data, training your network on, say, AWS GPUs, or deploying your trained model to other systems (for example, a recommendation system integrated into a news website) is a critical step. You will improve your training time and extend the amount of computing that can be used. This means being able to experiment with more complex models too. Ideally, we would want a library that makes it easy to integrate with existing tools or has tools of its own.

Now that we've got a reasonable set of requirements for our ideal library, let's take a look at a number of the popular options out there in the community. The following list is by no means exhaustive; however, it covers most of the major ML-related Go projects on GitHub, from the most narrow to the most general.