Concept
Multi-head attention
Multi-head attention runs several attention operations in parallel, each with its own learned projections, then concatenates their outputs. It generalizes the single scaled dot-product attention introduced in "Attention Is All You Need," which computes `softmax(QK^T /…
The rest of “Multi-head attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→