Concept
Stochastic gradient descent
Stochastic gradient descent (SGD) is gradient descent that estimates the gradient from a small random sample of the training data rather than the whole set. Full-batch gradient descent computes the loss gradient over every example before taking one step; SGD computes it over…
The rest of “Stochastic gradient descent” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→