Issue #1
Sent 2026-05-11
Mean Mode Screaming: Mean--Variance Split Residuals for 1000-Layer Diffusion Transformers
On a matched 400-layer DiT on ImageNet 256x256, MV-Split reaches FID 2.60 / IS 185.5 at 50k steps versus LayerScale's 2.90 / 165.5, while the un-stabilized baseline diverges before the first checkpoint. The 1000-layer result (FID 2.77) is presented only as a scale-validation run on a separate training pipeline, not a matched comparison, so it is not directly comparable to the 400-layer numbers.
arXiv:2605.06169 · 238▲ · cs.LG, cs.CV
- Vision
- Training
- Theory
The paper diagnoses a training collapse in very deep Diffusion Transformers where token representations homogenize toward their sequence mean, which the author calls Mean Mode Screaming, and traces it to a gradient decomposition where a mean-coherent component grows as O(T) once tokens align. It proposes MV-Split Residuals, a modified Post-Norm residual merge that applies separate learnable gains to the centered (token-varying) part of the residual branch versus a leaky replacement of the trunk mean.
Full breakdown →Stream-R1: Reliability-Perplexity Aware Reward Distillation for Streaming Video Generation
On VBench short video (5s, 832x480), the 4-step Stream-R1 reached a Total score of 84.40 versus 84.13 for the Reward Forcing baseline and 84.26 for its own multi-step teacher Wan2.1, running at 23.1 FPS. Gains over the baseline are small (+0.27 Total) and the comparison is mainly against one closely related baseline; the more convincing evidence is the widening quality gap over Reward Forcing at 120s and 180s durations.
arXiv:2605.03849 · 129▲ · cs.CV
- Vision
- Efficiency
- Training
Stream-R1 modifies distribution matching distillation (DMD) for streaming autoregressive video generation by reweighting the distillation loss with a pretrained video reward model at two levels: a per-rollout scalar weight (exponential of the reward score) and a per-pixel spatiotemporal weight derived from backpropagating the reward model's gradient. It replaces the uniform weighting that treats every rollout, frame, and pixel as equally reliable supervision with reward-guided weighting that concentrates optimization on regions where refinement helps most.
Full breakdown →RLDX-1 Technical Report
On ALLEX humanoid tasks RLDX-1 reached 86.8% success versus roughly 40% for both pi-0.5 and GR00T N1.6; on the memory-dependent Object-in-Box Selection task it hit 91.7% versus ~30% for both baselines. Caveats: real-robot evals use small trial counts (e.g. 24 trials on Pot-to-Cup), and the baselines were trained for matched steps rather than exhaustively tuned.
arXiv:2605.03269 · 126▲ · cs.RO, cs.AI, cs.LG
- Robotics
- Multimodal
- Inference
RLDX-1 is a Vision-Language-Action model for robotic manipulation built on the Multi-Stream Action Transformer (MSAT), which extends the Multi-Modal Diffusion Transformer to route each input modality (vision-language cognition, proprioception, actions, and physical signals like torque and tactile) through a dedicated stream coupled by joint self-attention. It adds a motion module for temporal dynamics, an explicit memory queue for long-horizon tasks, a synthetic data pipeline using video generation with motion-consistency filtering, and inference optimizations (CUDA Graph capture plus custom fused kernels).
Full breakdown →Stream-T1: Test-Time Scaling for Streaming Video Generation
On 30s video generation (128 MovieGen prompts, evaluated with VBench-Long and VideoAlign), Stream-T1 beats its LongLive baseline on nearly all metrics, with the largest gains on the human-preference VideoAlign scores; note these are near the noise floor (motion quality went from -0.002 to 0.226, which the paper reports as a large percentage but off a tiny base). Gains on standard VBench consistency metrics are under 1 percent. It also outperforms Best-of-N and beam-search baselines applied to the same model.
arXiv:2605.04461 · 109▲ · cs.CV
- Vision
- Inference
Stream-T1 is a test-time scaling framework built on top of a streaming (chunk-by-chunk autoregressive) video diffusion model (LongLive). It combines three inference-time tricks: initializing each chunk's noise by spherical interpolation from the previous chunk's noise, beam search pruning of chunk candidates using a combined image-reward plus video-reward score, and reward-guided routing of evicted KV cache entries into discard, EMA-merge, or append pathways.
Full breakdown →OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents
OpenSearch-VL-30B-A3B raises average Pass@1 from 47.8 to 61.6 across seven benchmarks versus its Qwen3-VL-30B-A3B agentic baseline, with the largest single-benchmark gain of +24.5 on MMSearch. Correctness is judged by GPT-4o rather than exact match, and the strongest proprietary comparisons are mostly direct-reasoning models rather than equivalently tooled agents.
arXiv:2605.05185 · 106▲ · cs.CV
- Multimodal
- Agents
- RL
OpenSearch-VL is a fully open recipe (data, code, models) for training multimodal search agents that interleave reasoning with tool calls over images and web search. It combines a Wikipedia-graph-based data synthesis pipeline, a tool environment (text/image search, OCR, crop, sharpen, super-resolution, perspective correction), and a modified GRPO reinforcement learning algorithm with fatal-aware token masking and one-sided advantage clamping to salvage useful reasoning from trajectories that hit cascading tool failures.
Full breakdown →Flow-OPD: On-Policy Distillation for Flow Matching Models
On SD-3.5-Medium, Flow-OPD (Merge variant) raised GenEval from 0.63 to 0.93 and OCR accuracy from 0.59 to 0.94, versus GRPO-Mix which reached only 0.73 GenEval and 0.83 OCR, roughly a 10-point average gain over vanilla multi-reward GRPO across four benchmarks. Caveat: baselines are the authors' own Flow-GRPO reruns, and the reported teacher-surpassing effect is supported mainly by qualitative edge cases rather than aggregate metrics.
arXiv:2605.08063 · 102▲ · cs.CV, cs.AI
- Vision
- RL
- Training
Flow-OPD is a post-training method for text-to-image flow matching models that replaces multi-reward reinforcement learning with on-policy distillation from multiple single-task teacher models. It trains domain-expert teachers via single-reward GRPO, then distills them into one student by routing prompts to the relevant teacher and matching velocity fields, showing that the reverse-KL divergence between student and teacher policies reduces analytically to a weighted MSE loss on the vector fields.
Full breakdown →Continuous Latent Diffusion Language Model
The authors trained CoLa DLM against strictly matched ~2B-parameter autoregressive (LLaMA-style) and LLaDA baselines with identical data, tokenizer, and optimization, and report favorable scaling curves up to about 2000 EFLOPs across 8 benchmarks (LAMBADA, MMLU, SIQA, SQuAD, HellaSwag, etc.). The paper claims strong scaling behavior rather than a single clean win number, and comparisons are limited to models the authors trained themselves at ~2B scale.
arXiv:2605.06548 · 85▲ · cs.CL, cs.AI, cs.CV
- LLMs
- Theory
- Multimodal
CoLa DLM is a text generation model that replaces left-to-right autoregression with a three-part pipeline: a Text VAE maps text to continuous latent variables, a block-causal Diffusion Transformer (DiT) learns a semantic prior over those latents via Flow Matching, and a conditional decoder turns generated latents back into text. The key design choice is that diffusion transports a latent prior (global semantics) rather than recovering token-level observations, separating semantic planning from token realization.
Full breakdown →OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories
46.0% on BrowseComp versus 43.4% for Tongyi DeepResearch (trained with full CPT plus SFT plus RL), plus 58.1% BrowseComp-ZH, 34.6% HLE, and 78.0% xbench, all as reported by the authors comparing against baselines taken from other papers' technical reports and leaderboards rather than head-to-head reruns.
arXiv:2605.04036 · 72▲ · cs.AI, cs.CL
- Agents
- LLMs
- Data
The paper trains a 30B search agent (OpenSeeker-v2, based on Qwen3-30B-A3B-Thinking) using only supervised fine-tuning on synthetic ReAct trajectories, skipping the usual continual pre-training and reinforcement learning stages. The main contribution is three tweaks to the data synthesis pipeline: expanding the knowledge graph size used to generate questions, enlarging the tool set, and filtering out any trajectory solvable in too few tool-call steps to enforce a difficulty floor.
Full breakdown →Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.