Code as Agent Harness
Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li, Lingjie Chen, Yanjun Zhao, Ke Yang, Bingxuan Li, Cheng Qian, Gaotang Li, Xiao Lin, Zhichen Zeng, Ruizhong Qiu, Sirui Chen, Yifan Sun, Xiyuan Yang, Ruida Wang, Rui Pan, Chenyuan Yang, Dylan Zhang, Liri Fang, Zikun Cui, Yang Cao, Pan Chen, Dorothy Sun, Ren Chen, Mahesh Srinivasan, Nipun Mathur, Yinglong Xia, Hong Li, Hong Yan, Pan Lu, Lingming Zhang, Tong Zhang, Hanghang Tong, Jingrui He
arXiv:2605.18747 · 223▲ · cs.CL, cs.AI
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
A survey that frames code as the 'harness' for LLM agents: the executable, inspectable, and stateful layer through which agents reason, act, model their environment, and coordinate. It organizes existing work into three layers (harness interface, harness mechanisms like planning/memory/tool-use/feedback control, and multi-agent scaling) and catalogs methods across coding assistants, GUI/OS automation, embodied agents, and scientific discovery.
Why it matters
For engineers building agent systems, it reframes the reliability bottleneck away from raw model quality toward the surrounding software: sandboxes, validators, execution loops, memory, and permission boundaries. It gives a vocabulary and taxonomy for thinking about why agents fail on long-horizon tasks (state loss, unverifiable actions, silent execution errors) rather than treating the LLM as the whole system.
Practical takeaway
Use this as a reading map, not a method. If you are designing an agent loop, the paper's distinction between model capabilities, system-provided infrastructure, and agent-created code artifacts (tests, temporary tools, execution traces) is a useful checklist, and the linked GitHub repo aggregates the cited systems (Voyager, SWE-bench, Code-as-Policies, etc.).
Key result
This is a survey with no experiments or benchmark numbers of its own; it summarizes representative systems (PAL, Program-of-Thoughts, SayCan, Voyager, SWE-bench, and others) rather than reporting a headline quantitative result. Note the arXiv ID and 2026 dates suggest a preprint whose cited-system performance claims should be checked against the original papers.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.