Issue #3
Sent 2026-05-25
HRM-Text: Efficient Pretraining Beyond Scaling
The 1B model scored 60.7% MMLU, 84.5% GSM8K, and 82.2% DROP while using roughly 96-432x less compute and 100-900x fewer tokens (40B unique) than 2-7B open models like Llama3.2 3B and Qwen. Caveats: it is an existence proof from one checkpoint capped at 1B scale, it skips raw-text base pretraining so the comparison to base models is not apples-to-apples, factual coverage (MMLU) lags larger models, and a contamination test flagged DROP as marginally inflated at n=13.
arXiv:2605.20613 · 321▲ · cs.CL
- LLMs
- Efficiency
- Training
HRM-Text replaces the standard decoder-only Transformer with a Hierarchical Recurrent Model that splits computation into a slow strategic (H) module and a fast execution (L) module, stabilized for language via two tricks the authors call MagicNorm (a hybrid Pre/Post normalization scheme) and a warmup schedule for truncated backpropagation through time. Instead of pretraining on raw web text, it trains a 1B-parameter model from scratch only on instruction-response pairs, computing loss just on the response (a task-completion objective) with a PrefixLM attention mask that lets instruction tokens attend bidirectionally.
Full breakdown →SkillOpt: Executive Strategy for Self-Evolving Agent Skills
Across 52 (model, benchmark, harness) cells, SkillOpt was best or tied-best on all 52. The headline case (GPT-5.5, direct chat) rose from 58.8 to 82.3 average over six benchmarks (+23.5 over no skill), and +5.4 over an oracle that picks the best of six competing baselines per cell; note the largest single gains (SpreadsheetBench 41.8 to 80.7) come from procedural tasks where the zero-shot baseline was weak, and all experiments use the same optimizer/target model family so results may not generalize to arbitrary setups.
arXiv:2605.23904 · 250▲ · cs.AI, cs.CL
- Agents
- LLMs
- Training
SkillOpt is a text-space optimizer that treats an agent's natural-language skill document as a trainable object, borrowing deep-learning controls like batches, a textual learning rate (bounded number of edits per step), a held-out validation gate, and epoch-wise slow/meta updates. A frontier optimizer model reads execution trajectories, proposes structured add/delete/replace edits to the skill, and only accepts edits that improve validation performance, exporting a compact best_skill.md (300 to 2,000 tokens) while the target model and harness stay frozen.
Full breakdown →Code as Agent Harness
This is a survey with no experiments or benchmark numbers of its own; it summarizes representative systems (PAL, Program-of-Thoughts, SayCan, Voyager, SWE-bench, and others) rather than reporting a headline quantitative result. Note the arXiv ID and 2026 dates suggest a preprint whose cited-system performance claims should be checked against the original papers.
arXiv:2605.18747 · 223▲ · cs.CL, cs.AI
- Agents
- LLMs
- Robotics
A survey that frames code as the 'harness' for LLM agents: the executable, inspectable, and stateful layer through which agents reason, act, model their environment, and coordinate. It organizes existing work into three layers (harness interface, harness mechanisms like planning/memory/tool-use/feedback control, and multi-agent scaling) and catalogs methods across coding assistants, GUI/OS automation, embodied agents, and scientific discovery.
Full breakdown →DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards
On seven math benchmarks with Qwen3-8B-Base, DelTA raised the average score from 25.14 (strongest same-scale baseline, SAPO) to 28.40, a 3.26-point gain; on Qwen3-14B-Base it improved from 37.29 (FIPO) to 39.91. Gains were consistent but modest across all benchmarks, and were reproduced on Olmo3-7B-Base (+3.79 avg) and code generation (47.7 to 49.5 weighted avg).
arXiv:2605.21467 · 207▲ · cs.LG, cs.CL
- RL
- LLMs
- Training
The paper reframes RLVR (reinforcement learning from verifiable rewards) policy updates as an implicit linear discriminator over token-gradient vectors, arguing that standard sequence-level methods like DAPO let shared high-frequency tokens (formatting, entities) dominate the update direction. It proposes DelTA, which computes per-token coefficients based on how well each token's gradient distinguishes positive-advantage from negative-advantage responses, then uses those coefficients to reweight the DAPO surrogate loss.
Full breakdown →TransitLM: A Large-Scale Dataset and Benchmark for Map-Free Transit Route Generation
The 4B jointly-trained model reached 71.0% Route Exact Match and 97% connectivity on Optimal Route Generation (10K samples, 4 cities), versus 40.2% Exact Match for the best general-purpose LLM (Gemini-3.1-Pro) even when that baseline was given an easier boarding/alighting-only output format. In GPS-only input (all text removed), the domain model held at 70.4% Exact Match while general LLMs collapsed to under 1%, though the benchmark measures structural overlap against a single production routing engine's outputs, not real-trip validation.
arXiv:2605.22355 · 179▲ · cs.CL, cs.AI, cs.LG
- LLMs
- Data
- Training
TransitLM is a dataset of over 13 million transit route planning records (from Amap logs across four Chinese cities, 120,845 stations) plus a three-task benchmark, released as a continual pre-training corpus and supervised fine-tuning data. The authors validate it by continually pre-training then fine-tuning Qwen3 models (0.6B to 4B) that generate complete transit routes end-to-end from origin-destination GPS coordinates, with each station ID registered as a dedicated vocabulary token so routes cannot be hallucinated via character composition.
Full breakdown →Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?
Across all 27 models the mean Prejudice Rate is 51.3% (over half of correct ratings lack grounded cues) and mean Holistic-Grounding Rate is only 10.4%, with the best model (Gemini 3 Flash) reaching just 33.5%. Closed and open models are close on rating (delta -5.6%) and verbal reasoning (delta -3.6%) but diverge sharply on cue retrieval (delta -26.6%); note Task 2 reasoning quality is scored by an AI-as-Judge (GPT-4o-mini), a self-acknowledged reliability caveat.
arXiv:2605.22109 · 171▲ · cs.AI, cs.CV, cs.CY
- Multimodal
- LLMs
- Vision
The paper introduces Grounded Personality Reasoning (GPR), a task that forces multimodal LLMs to justify each Big Five (OCEAN) personality rating with cited, timestamped behavioral evidence, and releases MM-OCEAN, a benchmark of 1,104 videos and 5,320 cue-grounding multiple-choice questions built by a multi-agent annotation pipeline with human verification. It benchmarks 27 MLLMs across three tiers (rating, open-ended reasoning, structured cue grounding) plus four failure-mode metrics that separate 'right answer' from 'right reasons'.
Full breakdown →Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation
On the extreme NOIZEUS 0dB setting, Mega-ASR reaches 19.80 WER vs. 23.97 for Qwen3-ASR (its own base model), a 17.4% relative reduction; averaged across CHiME-4, VOiCES, and NOIZEUS it gets 6.70 vs. 7.93. Note most gains are measured against the same backbone it was fine-tuned from, and much of the training/eval data is synthetically simulated.
arXiv:2605.19833 · 137▲ · cs.SD, cs.AI, cs.CL, cs.MM, eess.AS
- RL
- Training
- Data
Mega-ASR fine-tunes Qwen3-ASR-1.7B to handle heavily degraded real-world audio using two techniques: Acoustic-to-Semantic Progressive Supervised Fine-Tuning (a WER-graded curriculum that trains the encoder, then the LLM, then jointly) and Dual-Granularity WER-Gated Policy Optimization (a reinforcement learning reward that switches between token-level and sentence-level scoring based on WER). It is trained on Voices-in-the-Wild-2M, a 2.4M-clip synthetic dataset built by simulating 7 atomic acoustic effects and 54 compound scenarios at the spectrogram level.
Full breakdown →SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution
On Terminal-Bench 2.0 (avg@5 accuracy) online evolution improved GPT-5.2 by 2.6 points, GPT-5.4 mini by 1.1, and GPT-5.5 by 0.9 over a no-skills baseline; the largest offline transfer gain was 7.8 points for GPT-5.2 from a library built on historical Terminal-Bench Pro trajectories. Gains are modest and uneven (skills sometimes regress on harder splits and on GPT-5.5), and the authors attribute the small averages to a heavy-tailed effect where skills help a few tasks a lot and hurt others.
arXiv:2605.18401 · 130▲ · cs.CL, cs.AI
- Agents
- LLMs
- RL
SkillsVote is a lifecycle framework for managing libraries of Agent Skills (structured folders of instructions, scripts, and applicability conditions used by LLM coding agents). It does three things: performs agentic search over skill folders to pick relevant skills before a task, decomposes execution traces into skill-linked subtasks after a task, and gates library updates so only successful, reusable discoveries edit or create skills.
Full breakdown →Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.