Concept
AdaGrad
AdaGrad (Adaptive Gradient) is a gradient descent variant that gives each parameter its own learning rate, scaled down in proportion to how much that parameter has been updated in the past. Like plain SGD, except the global step size is divided per-parameter by the square…
The rest of “AdaGrad” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→