Concept
Position embeddings
Position embeddings are learned or fixed vectors added to token representations so a Transformer knows the order of its inputs. Self-attention is permutation-invariant: it computes the same output regardless of how you shuffle the input tokens, because attention weights…
The rest of “Position embeddings” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→