Concept
Self-attention
Self-attention is a mechanism where every position in a sequence computes a weighted combination of all other positions, letting each token gather context directly from the whole sequence. Unlike a recurrent layer, which passes information step by step and forgets over…
The rest of “Self-attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→