Concept
Bidirectional intra-block attention
Bidirectional intra-block attention relaxes the causal masking of a decoder-only transformer for tokens that belong to the same predefined block, while keeping attention causal between blocks. Standard autoregressive models mask every position so it sees only earlier tokens;…
The rest of “Bidirectional intra-block attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→