EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments
Jundong Xu, Qingchuan Li, Jiaying Wu, Yihuai Lan, Shuyue Stella Li, Huichi Zhou, Bowen Jiang, Lei Wang, Jun Wang, Anh Tuan Luu, Caiming Xiong, Hae Won Park, Bryan Hooi, Zhiyuan Hu
arXiv:2606.13681 · 142▲ · cs.CL
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
EvoArena is a benchmark suite that turns static agent benchmarks (terminal workflows, software repos, user preferences) into version chains where the same environment evolves across releases, and the agent must adapt to changes while preserving still-valid prior behavior. The paper also proposes EvoMem, a git-like memory layer that logs non-additive memory updates as append-only patches (storing before/after state, rationale, and triggering evidence) and retrieves relevant patches alongside the latest memory at inference.
Why it matters
Most memory-based agents collapse memory to a single latest state, so a rule updated for a new API version quietly erases behavior that still applies to older versions or rollbacks. This benchmark quantifies how badly agents degrade under that kind of persistent environment drift, and the patch approach is a concrete integration pattern (wraps an existing memory system rather than replacing it) for practitioners building agents that must survive version changes.
Practical takeaway
Watch for a shift from single-state memory stores toward versioned, evidence-tracked memory in agent frameworks; you can try wrapping an existing memory system (they demonstrate integrations with A-Mem, OpenHands, Terminus2, Memento-Skill) with an append-only patch log to preserve overwritten states rather than consolidating everything into the latest snapshot.
Key result
Current agents average only 39.6% accuracy across the three EvoArena domains. Adding EvoMem gives a modest average step-accuracy gain of 1.5% and a 3.7% chain-accuracy gain on EvoArena, plus 6.1% on GAIA and 4.8% on LoCoMo; note the step gains are small and per-model, so the headline value is the benchmark exposing the failure mode more than the memory fix eliminating it.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.