Concept
Prefilling
Prefilling is the first phase of transformer inference, where the model processes the entire input prompt in a single forward pass to populate the KV cache before generating any output tokens. It contrasts with the decode phase, which produces tokens one at a time, each…
The rest of “Prefilling” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→