Concept
Vocabulary token
A vocabulary token is one entry in the fixed set of symbols a language model can read and emit. The model's embedding table has exactly one row per token, and its output layer produces one logit per token, so the vocabulary size is a hard structural parameter (often 30k to…
The rest of “Vocabulary token” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→