Finally, let's build a neural network! We'll be building a simple three-layer neural network with one hidden layer. A three-layer neural network has two weight matrices, so we can define the neural network as such:

type NN struct {
  hidden, final *tensor.Dense
  b0, b1 float64
}

hidden represents the weight matrix between the input layer and hidden layer, while final represents the weight matrix between the hidden layer and the final layer.

This is a graphical representation of our *NN data structure:

The input layer is the slice of 784 float64 which is then fed forward (that is, a matrix multiplication followed by an activation function) to form the hidden layer. The hidden layer is then fed forward to form the final layer. The final layer is a vector of ten float64, which is exactly the one-hot encoding that we discussed earlier. You can think of them as pseud-probabilities,  because the values don't exactly sum up to 1.

A key thing to note: b0 and b1 are bias values for the hidden layer and the final layer, respectively. They are not actually used mainly due to the mess;  it's quite difficult to get the correct differentiation. A challenge for the reader is to later incorporate the use of b0 and b1.

And to create a new neural network, we have the New function:

func New(input, hidden, output int) (retVal *NN) {
  r := make([]float64, hidden*input)
  r2 := make([]float64, hidden*output)
  fillRandom(r, float64(len(r)))
  fillRandom(r2, float64(len(r2)))
  hiddenT := tensor.New(tensor.WithShape(hidden, input), tensor.WithBacking(r))
  finalT := tensor.New(tensor.WithShape(output, hidden), tensor.WithBacking(r2))
  return &NN{
    hidden: hiddenT,
    final: finalT,
  }
}

The fillRandom function fills a []float64 with random values. In our case, we fill it up from random values drawn from a uniform distribution. Here, we use the distuv package from Gonum:

func fillRandom(a []float64, v float64) {
  dist := distuv.Uniform{
    Min: -1 / math.Sqrt(v),
    Max: 1 / math.Sqrt(v),
  }
  for i := range a {
    a[i] = dist.Rand()
  }
}

After the slices r and r2 have been filled, the tensors hiddenT and finalT are created, and the *NN is returned.