Concept
Sliding-window attention
Sliding-window attention restricts each token to attend only to a fixed number of recent tokens, rather than the entire preceding sequence. Standard self-attention is O(n²) in sequence length because every token computes scores against every other token.
The rest of “Sliding-window attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→