Concept
Logits distillation
Logits distillation is a form of knowledge distillation where a student model is trained to match the raw output scores (logits) of a teacher model, rather than the hard labels. It is like standard supervised training, except your target is the teacher's full pre-softmax…
The rest of “Logits distillation” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→