Our hidden layer will start with 32 neurons. At this point, we can't know how many neurons are necessary. This is really a hyperparameter and can be explored and tuned later. Identifying an appropriate network architecture for a given problem is an open problem in the field of deep learning.

Since each of these 32 neurons in the hidden layer will output their activation to the output layer, the shape of the hidden layer will be (10, 32).