Concept
Unembedding matrix
The unembedding matrix is the final linear layer of a transformer language model that maps a hidden state vector back into vocabulary space, producing one logit per token. It is the transpose-shaped counterpart to the embedding matrix at the input: where embedding turns a…
The rest of “Unembedding matrix” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→