Learning rate parameter

The learning rate is a parameter that determines the speed of learning. More formally, it determines how much we are adjusting the weights of our network with respect to the loss gradient. If it is too low, we travel slower down our slope. Even though we desire to have a low learning rate, it could mean that we'll be taking a long time to reach convergence. The learning rate also affects how quickly our model can converge tolocal minima (best accuracy).

When dealing with neurons, it determines the acquisition time (the amount of time it takes for a response to a new experience to be learned) for neurons with weights being used for training.

Effect of various learning rates on convergence (image credit: cs231n)
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.21.46.92