LM101-068: How to Design Automatic Learning Rate Selection for Gradient Descent Type Machine Learning Algorithms

Summary: Simple mathematical formulas are presented that ensure convergence of a generated sequence of parameter vectors which are updated using an iterative algorithm consisting of adding a stepsize number multiplied by a search direction vector to the current parameter values and repeating this process. These formulas may be used as the basis for the design of artificially intelligent smart automatic learning rate selection algorithms. Please visit:  www.learningmachines101.com