Concept
Token concatenation
Token concatenation is the practice of joining sequences of tokens from different inputs into one long sequence before feeding them to a transformer, so that self-attention operates over all of them jointly. It is the alternative to cross-attention, where one stream stays…
The rest of “Token concatenation” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→