Concept
Scaled dot-product attention
Scaled dot-product attention is the core operation introduced in "Attention Is All You Need." It computes a weighted average of value vectors, where the weights come from how well each query matches each key. Think of it like a soft dictionary lookup, except instead of exact…
The rest of “Scaled dot-product attention” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→