These are the models that most fascinate me. From their wide range of applications to their brilliant origin—everything seems marvelous to me. Note that there is no such a thing as an all mighty model and neural nets are not it. They can be distinguished by being very flexible models that can perform both unsupervised and supervised learning.
Even though it's a very powerful method, it's certainly not all powerful. Neural nets are able to capture linear and non-linear relations. Yet, everything will depend on how you design the networks (researcher's ability and experience), how complex is the problem at hand, and how many observations do you have.
Computational constraints can also be a bottleneck. As powerful as they are known to be, neural networks are not remembered as computationally inexpensive methods. It's not hard to find problems that can't be efficiently handled by consumer grade computers. Cloud computing is usually the best and cheaper way out of this problem. Nonetheless, if you are dealing with sensible data, you might consider transforming it to ensure anonymity first, or not working with clouds at all.
This section will only briefly discuss the nuts and bolts of neural networks. More details on that are about to be delivered in Chapter 8, Neural Networks and Deep Learning. There you might find a practical guide to the keras package, which enables us to use TensorFlow directly from R. Here you may find a very condensed practical guide to h2o.
The very first time I ever saw a neural network working was when I was learning how to play Super Mario Bros. Surprisingly enough, the AI got to know shortcuts that someone hardly noticed before. The second time I saw it, an artist was using neural networks to craft figures.
Neural networks are a universe within themselves. There is much to talk about but let's stick with this: with some degree of simplicity, neural nets imitate how neurons transmit/interpret information by arranging nodes into three kinds of layers (input, hidden, and output layers).
There are many hyperparameters to take into consideration when you are about to train a neural net. If you are new to neural networks, you may only learn quite a few things from this section but don't worry, you will see a lot more of it in Chapter 8, Neural Networks and Deep Learning.
By chaining enough hidden layers with a sufficient number of hidden nodes, neural nets can (slowly) approximate any sort of linear and nonlinear functions. Here is a representation of a neural network that I have designed to predict Brazilian live cattle prices for a forecast horizon of 60 business days:
The preceding diagram displays a fully connected feedforward network—there are various sorts of network architectures available—trained with the neuralnet package. The blue circles represent bias nodes. In the left, you can see a couple of black circles stacked. These are the input nodes, information flows from them to the rest of the network. Information flow is marked by the arrows.
Nodes are providing the next ones with numeric information that is multiplied by a number. These multipliers are called weights and are shown by Figure 6.14 right above the arrow connecting a node to some other. Weights are hyperparameters that learning algorithms such as the Adam optimization and backpropagation will seek to improve.
Once a node gets all of the balanced numeric information coming from other nodes (and sometimes itself), it will sum all of them and apply a given function (activation function—another important decision to make). This functions' results will land on another node that will repeat the process until the output node, the lone single node displayed in the extreme right of Figure 6.14. Output nodes are the ones giving the output.
There is still much to talk about with the nuts and bolts of neural nets in general, but for the time being, I will keep such details for Chapter 8, Neural Networks and Deep Learning. Let's see how the h2o package can be used to train neural network models.