Linear functions

A possible extension to the step function might be to use a linear function, as shown in the following code:

func linear(x){
return 0.5 * x
}

This is still very simple and, if we were to chart it out, it would look something like the following diagram:

However, this function is still not very useful. If we were to look at the gradient, we'll see that, when we differentiate this function, all we get is a straight line equal to the value of a. This means it suffers the same problem as the step function; that is to say, we won't see much improvement from backpropagation.

In addition, if we were to stack several layers of this, you'll find that really all we get is not too different from having just one layer. This isn't useful if we are trying to build models with multiple layers, especially with non-linear relationships.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.220.139.168