The neural network performance, as you must have observed by now, depends on a lot on hyperparameters. Thus, it becomes important that one gains an understanding as to how these parameters affect the network. Common examples of hyperparameters are learning rate, regularizers, regularizing coefficient, dimensions of hidden layers, initial weight values, and even the optimizer selected to optimize weights and biases.