Concept
PrefixLM
PrefixLM (Prefix Language Model) is an attention masking scheme that mixes bidirectional and causal attention within a single sequence. Tokens in the prefix (the input or instruction) attend to each other in both directions, like an encoder.
The rest of “PrefixLM” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→