Concept
Reference model
A reference model is a frozen copy of a policy that you keep around to measure how far your trained model has drifted from its starting point. In preference-tuning methods like RLHF and DPO, it is almost always the supervised fine-tuned (SFT) model you begin with, held fixed…
The rest of “Reference model” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→