Concept
Joint self-attention
Joint self-attention is a self-attention operation computed over the concatenated token sequences of multiple streams at once, rather than running attention separately per stream. Like the self-attention inside a standard transformer block, except the query/key/value…
The rest of “Joint self-attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→