Concept
Continual pre-training
Continual pre-training (sometimes "continued pre-training" or "domain-adaptive pre-training") means taking an already-pretrained base model and running more of the same next-token objective on a new corpus, before any instruction tuning. Think of it like the original…
The rest of “Continual pre-training” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→