LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling
Jian Yang, Shawn Guo, Wei Zhang, Tianyu Zheng, Yaxin Du, Haau-Sing Li, Jiajun Wu, Yue Song, Yan Xing, Qingsong Cai, Zelong Huang, Chuan Hao, Ran Tao, Xianglong Liu, Wayne Xin Zhao, Mingjie Tang, Weifeng Lv, Ming Zhou, Bryan Dai
arXiv:2606.18023 · 209▲ · cs.LG, cs.AI
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
The paper studies how many times to loop a Parallel Loop Transformer (PLT), a looped-Transformer variant that reuses shared blocks with cross-loop position offsets (CLP) and shared-KV gated sliding-window attention to keep latency and KV-cache memory flat regardless of loop count. It trains LoopCoder-v2, a 7B coder, from scratch on 18T tokens at loop counts of 1, 2, 3, and 4, and uses per-loop diagnostics (hidden-state dynamics, attention routing, output-distribution shift) to explain why performance peaks at two loops.
Why it matters
For anyone deploying looped models to add test-time compute without extra parameters, PLT means an extra loop is nearly free in latency and memory, but this work shows more loops are not better: two loops is the sweet spot and three or more actively hurts. It gives a cheap diagnostic (watch when effective rank stops rising) to pick loop count instead of running expensive full benchmark sweeps.
Practical takeaway
If you experiment with looped/recurrent-depth models, set the loop count to 2 rather than assuming monotonic gains, and track the effective-rank trajectory across loops: if it is still rising another loop may help, if it has begun falling extra loops just add fixed offset cost. Also note latent looping and explicit chain-of-thought stack super-additively, so combine both rather than choosing one.
Key result
On SWE-bench Verified, the two-loop model scored 64.4 vs 43.0 for the non-looped baseline, while the three-loop model regressed to 27.6 and four-loop to 22.4 (all same 7B model, matched training and tuning). Caveat: these are the authors' own from-scratch models and the headline comparison is against their own R=1 baseline, not an independent equal-compute system.
Jargon
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.