Concept
Block-causal attention
Block-causal attention is a masking scheme that treats a sequence as an ordered set of blocks, applying full bidirectional attention within each block and causal attention across blocks. It sits between the two familiar extremes: a standard decoder mask forbids any token…
The rest of “Block-causal attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→