With all that done, let's walk through the the feed forward function, line by line.
First, recall from the section, Emulating a neural network, that we can define a neural network as follows:
func affine(weights [][]float64, inputs []float64) []float64 {
return activation(matVecMul(weights, inputs))
}
We do the first matrix multiplication as part of calculating the first hidden layer: hidden := m.do(func() (tensor.Tensor, error) { return nn.hidden.MatVecMul(a)) }). MatVecMul is used because we're multiplying a matrix by a vector.
Then we perform the second part of calculating a layer: act0 := m.do(func() (tensor.Tensor, error) { return hidden.Apply(sigmoid, tensor.UseUnsafe()) }). Once again, the tensor.UseUnsafe() function option is used to tell the function to not allocate a new tensor. Voila! We've successfully calculated the first layer.
The same two steps are repeated for the final layer, and we get a one-hot-ish vector. Do note that for the first step, I used tensor.MatVecMul(nn.final, act0) instead of nn.final.MatVecMul(act0). This was done to show that both functions are indeed the same, and they just take different types (the method takes a concrete type while the package function takes an abstract data type). They are otherwise identical in function.
Is there a way to abstract the function into a function like affine so that you could just call a single function and not repeat yourself?
Before we return the result, we need to perform a check to see if anything in the prece-ing steps have errored. Think about what are the errors that could happen. They would, in my experience, predominantly be shape related errors. In this specific project, a shape error should be considered a failure, so we return a nil result and the error.
The reason why we would have to check for errors at this point is because we are about to use pred. If pred is nil (which it would be if an error had occurred earlier), trying to access the .Data() function would cause a panic.
Anyway, after the check, we call the .Data() method, which returns the raw data as a flat slice. It's an interface{} type though, so we would have to convert it back to a []float64 before inspecting the data further. Because the result is a vector, it is no different in data layout from a []float64, so we can directly call argmax on it.
argmax simply returns the index of the greatest value in the slice. It's defined thus:
func affine(weights [][]float64, inputs []float64) []float64 {
return activation(matVecMul(weights, inputs))
}
And thus, we have managed to write a feed forward function for our neural network.