Stream-R1: Reliability-Perplexity Aware Reward Distillation for Streaming Video Generation
Bin Wu, Mengqi Huang, Shaojin Wu, Weinan Jia, Yuxin Wang, Zhendong Mao, Yongdong Zhang
arXiv:2605.03849 · 129▲ · cs.CV
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
Stream-R1 modifies distribution matching distillation (DMD) for streaming autoregressive video generation by reweighting the distillation loss with a pretrained video reward model at two levels: a per-rollout scalar weight (exponential of the reward score) and a per-pixel spatiotemporal weight derived from backpropagating the reward model's gradient. It replaces the uniform weighting that treats every rollout, frame, and pixel as equally reliable supervision with reward-guided weighting that concentrates optimization on regions where refinement helps most.
Why it matters
For anyone distilling a slow multi-step video diffusion teacher into a fast few-step student, this adds no inference cost and no architecture change to the student while improving quality across visual, motion, and text-alignment axes. The temporal weighting reduces quality drift in long autoregressive rollouts, which is the main failure mode when generating minute-plus videos frame by frame.
Practical takeaway
If you already train reward-guided distilled video models (this builds directly on the Reward Forcing framework with Wan2.1 as student/teacher), you can extract a spatial and temporal saliency map from a single extra backward pass through your reward model and use it to reweight the DMD loss, rather than applying the reward as a single scalar.
Key result
On VBench short video (5s, 832x480), the 4-step Stream-R1 reached a Total score of 84.40 versus 84.13 for the Reward Forcing baseline and 84.26 for its own multi-step teacher Wan2.1, running at 23.1 FPS. Gains over the baseline are small (+0.27 Total) and the comparison is mainly against one closely related baseline; the more convincing evidence is the widening quality gap over Reward Forcing at 120s and 180s durations.
Jargon
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.