Concept
Log-probability gap
The log-probability gap is the difference between two models' log-probabilities for the same token: log p_teacher(token) - log p_student(token). It measures how much more (or less) one model favors a specific next token given the same prefix.
The rest of “Log-probability gap” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→