Concept
Next-token prediction
Next-token prediction is the training objective behind most modern language models: given a sequence of tokens, predict the probability distribution over the next token, then repeat. The model outputs a vector of logits the size of the vocabulary, a softmax turns that into…
The rest of “Next-token prediction” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→