Toward Generalist Autonomous Research via Hypothesis-Tree Refinement
Jiajie Jin, Yuyang Hu, Kai Qiu, Qi Dai, Chong Luo, Guanting Dong, Xiaoxi Li, Tong Zhao, Xiaolong Ma, Gongrui Zhang, Zhirong Wu, Bei Liu, Zhengyuan Yang, Linjie Li, Lijuan Wang, Hongjin Qian, Yutao Zhu, Zhicheng Dou
arXiv:2606.11926 · 126▲ · cs.CL, cs.AI
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
Arbor is a framework for autonomous research (which the authors call Autonomous Optimization) where an LLM agent iteratively improves a code artifact against a fixed objective without step-level human supervision. Its core mechanism is Hypothesis Tree Refinement (HTR): a persistent tree whose nodes bind a hypothesis, a git-worktree artifact version, evaluation evidence, and a distilled insight, managed by a long-lived coordinator that dispatches short-lived executors and only promotes changes that pass a held-out test evaluator.
Why it matters
For anyone building long-running coding agents, the practical lesson is that separating exploration (a dev evaluator) from admission (a held-out test evaluator with a merge gate) prevents the agent from silently overfitting to its own feedback signal. The tree-structured state also isolates each experiment in its own git worktree, so competing attempts stay auditable and reversible rather than piling up in one context window.
Practical takeaway
If you run agents like Codex or Claude Code on iterative optimization tasks, add a strict dev/test split and refuse to accept a change unless it improves the held-out set, because the paper shows dev-score gains routinely fail to transfer. The system is open source, so you can inspect the coordinator/executor loop directly.
Key result
Across six real research tasks (model training, harness engineering, data synthesis) under a 48-hour budget and identical task interface, Arbor got the best held-out result on all six and roughly 2.5x the average relative held-out gain of Codex and Claude Code. On MLE-Bench Lite it reached 86.36% Any Medal with GPT-5.5, though this is a small task suite and baseline numbers are taken from an external leaderboard rather than re-run.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.