Concept
Positional encodings
Positional encodings are vectors added to token embeddings so a model knows where each token sits in a sequence. Recurrent networks get order for free because they process tokens one at a time.
The rest of “Positional encodings” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→