EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, Yang Yang
arXiv:2607.02440 · 45▲ · cs.AI, cs.CL
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
EvoPolicyGym is a benchmark that measures how well a coding agent (a language model plus its tool-using harness) can iteratively rewrite an executable decision policy for a reinforcement learning environment, using feedback from sandboxed rollouts under a fixed budget of 128 episodes. Agents edit a Python policy that exposes reset and act(obs), submit it for scored train rollouts, and get judged on held-out return of the checkpoint the server picks via hidden validation.
Why it matters
For anyone building agentic coding tools, this separates 'can the agent write a patch' from 'can the agent convert limited feedback into a policy that generalizes.' It gives trajectory-level diagnostics (when improvements happen, whether the agent invents new control structure versus tuning constants) rather than a single pass/fail, which is useful if you care about how an agent spends a bounded interaction budget. This is primarily a research and evaluation contribution, not something that changes a production workflow today.
Practical takeaway
Watch for evaluation setups that distinguish structural synthesis (building perception, memory, planning code) from parametric tuning (adjusting gains and thresholds), since the paper shows weak agents can churn structure endlessly without ever landing on the right mechanism. If you evaluate agents, budgeting interactions and hiding validation splits is a cheap way to expose overfitting to visible feedback.
Key result
On the 16-task Core16 suite (Gym/Box2D, MuJoCo, MiniGrid, robotics/driving) under a shared 128-episode budget, GPT-5.5 (via the Codex harness) reached the top aggregate rank score of 0.891 with top-two placement on all 16 environments, versus 0.750 for Claude Opus 4.7 and 0.109 for a uniform random policy. Caveat: scores are rank-based aggregates across heterogeneous reward scales rather than absolute returns, each cell is a single run (no repeated seeds reported), and the tested models and harnesses are fictional-sounding future versions, so treat the specific numbers as illustrative of the protocol rather than a settled ranking.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.