Concept
Full attention
Full attention is the standard self-attention mechanism from the original Transformer, where every token attends to every other token in the sequence. For each query position, you compute a similarity score against all key positions, softmax those scores, and take a weighted…
The rest of “Full attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→