In this chapter, we covered how to build a simple neural network and how to inspect your graph, as well as many of the commonly used activation functions. We then covered the basics of how a neural network is trained via backpropagation and gradient descent. Finally, we discussed some of the different options for gradient descent algorithms and optimizations for your neural network.

The next chapter will cover building a practical feedforward neural network and autoencoders, as well as Restricted Boltzmann Machines (RBMs).