Concept
Distillation
Distillation trains a smaller or faster "student" model to reproduce the behavior of a larger, slower "teacher," transferring learned capability without copying the teacher's architecture or cost. ) fits the student to the teacher's soft output probabilities rather than hard…
The rest of “Distillation” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→