Issue #9
Sent 2026-07-06
Orca: The World is in Your Mind
On real-robot out-of-distribution tasks, Orca-4B scored 32.4 overall rule-based vs 29.4 for pi-0.5 (a VLA pretrained on large-scale robot data) and 10.5 for a Qwen3.5 baseline, with action experts trained from scratch on only 200 trajectories per task across 5 tasks. Caveat: the eval covers just 5 short tasks and success rates are very low (6% binary success for Orca), so the comparison shows partial-progress advantages more than reliable task completion.
arXiv:2606.30534 · 235▲ · cs.CV
- Multimodal
- Robotics
- Vision
Orca is an encoder-decoder model that learns a shared 'world latent space' from video and language by predicting the latent representation of adjacent frames (unconscious learning) and event-conditioned future states described by text (conscious learning), plus a standard VQA loss. After pretraining, the backbone (built on a Qwen VLM) is frozen and small task-specific decoders are trained for text, image prediction, and robot action generation to test whether the shared latent transfers.
Full breakdown →Program-as-Weights: A Programming Paradigm for Fuzzy Functions
On FuzzyBench (exact-match on a verified test set of held-out fuzzy-task specs), a 0.6B interpreter running PAW programs scored 73.78% versus 68.70% for directly prompting Qwen3-32B, at roughly 50x less inference memory (~1.2GB vs ~60GB). Caveats: both training and test data are synthetic (generated by gpt-5.2, which itself ceilings at 96%), all tasks are single-step, and on some non-benchmark datasets in the same table (SMS, IMDB) PAW trails the larger prompted baselines.
arXiv:2607.02512 · 106▲ · cs.LG, cs.AI, cs.CL
- LLMs
- Efficiency
- Inference
The paper introduces Program-as-Weights (PAW), a system that compiles a natural-language description of a fuzzy task (things like log alerting, JSON repair, or intent classification that resist clean rule-based code) into a small neural artifact. A 4B compiler reads the spec plus an auto-generated pseudo-program of paraphrase-and-examples, then emits a per-function LoRA adapter that gets hot-attached to a frozen 0.6B interpreter model, which runs the task locally.
Full breakdown →Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent
The paper claims Agents-A1 (35B) matches or beats 1T-parameter models (Kimi-K2.6, DeepSeek-V4) on SEAL-0, IFBench, HiPhO, FrontierScience-Olympiad, and MolBench-Bind, with strong results on SciCode, HLE, and BrowseComp. Caveat: the summary figure and text report relative comparisons on selected science and search benchmarks rather than head-to-head numbers across a broad general-purpose suite, so the parity claim is scoped to these curated domains.
arXiv:2606.30616 · 90▲ · cs.CL
- Agents
- LLMs
- Training
A recipe for training a 35B mixture-of-experts agent (Agents-A1) to handle long-horizon agentic tasks by scaling trajectory length rather than parameters. The method has three stages: full-domain supervised fine-tuning on ~100K long trajectories (average 45K tokens), training domain-specific teacher models, then consolidating them into one student via domain-routed on-policy distillation with salient vocabulary alignment (SVA), which aligns student and teacher on the teacher's top-k token support instead of just the sampled token.
Full breakdown →BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding
On Qwen3-4B at temperature 1, BlockPilot reached 4.20x speedup and acceptance length 5.92 averaged across Math, Code, and Chat benchmarks, versus 3.80x for the best fixed-block baseline DFlash(16); gains over the strongest fixed-block variant are consistent but modest (roughly 0.1 to 0.4x speedup) across models and temperatures.
arXiv:2606.31315 · 75▲ · cs.CL
- LLMs
- Inference
- Efficiency
BlockPilot is a lightweight predictor that picks a per-sample block size for diffusion-based speculative decoding, replacing the fixed block size inherited from training. It uses the target model's last-token predictive distribution after prefilling as input to a two-layer MLP classifier that selects the block size from a small local range around the training value.
Full breakdown →AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents
In the fixed-difficulty (A0) matrix, enabling the triggered strategic skill layer moved wins from 3/10 (no scaffold) to 6/10, but at N=10 per cell this is directional only (Fisher exact p is about 0.37, overlapping confidence intervals), so it is not statistically significant. The stronger evidence is operational: their bounded baseline won 3/10 versus 0/5 for two open-source transcript-accumulating StS2 agents on the same backbone, character, and difficulty, while using far fewer tokens, but those competitors are untuned community projects rather than matched baselines.
arXiv:2607.02255 · 55▲ · cs.AI, cs.CL
- Agents
- LLMs
- Efficiency
The paper introduces AgenticSTS, an LLM agent for the deck-building game Slay the Spire 2 that replaces the usual growing chat transcript with a bounded memory contract: every decision prompt is rebuilt fresh from five typed slots (protocol, state schema, game rules, episodic summaries, and triggered strategic skills) via typed retrieval, so no raw cross-decision history is appended. It also ships a testbed of 298 tagged trajectories, frozen memory snapshots, and analysis scripts so individual memory layers can be ablated in isolation.
Full breakdown →EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
On the 16-task Core16 suite (Gym/Box2D, MuJoCo, MiniGrid, robotics/driving) under a shared 128-episode budget, GPT-5.5 (via the Codex harness) reached the top aggregate rank score of 0.891 with top-two placement on all 16 environments, versus 0.750 for Claude Opus 4.7 and 0.109 for a uniform random policy. Caveat: scores are rank-based aggregates across heterogeneous reward scales rather than absolute returns, each cell is a single run (no repeated seeds reported), and the tested models and harnesses are fictional-sounding future versions, so treat the specific numbers as illustrative of the protocol rather than a settled ranking.
arXiv:2607.02440 · 45▲ · cs.AI, cs.CL
- Agents
- RL
- LLMs
EvoPolicyGym is a benchmark that measures how well a coding agent (a language model plus its tool-using harness) can iteratively rewrite an executable decision policy for a reinforcement learning environment, using feedback from sandboxed rollouts under a fixed budget of 128 episodes. Agents edit a Python policy that exposes reset and act(obs), submit it for scored train rollouts, and get judged on held-out return of the checkpoint the server picks via hidden validation.
Full breakdown →Beyond IID: How General Are Tabular Foundation Models, Really?
Foundation model error degrades sharply above roughly 100,000 rows: mean normalized TFM error rises from 0.183 (tiny, 100 to 1k rows) to 0.528 on large datasets, a statistically significant jump (Mann-Whitney, rank-biserial r between 0.68 and 0.79). On the full 142-dataset leaderboard, tuned+ensembled RealMLP leads at 1282 Elo versus 1224 for zero-shot TabPFN-2.6, with XGBoost calibrated to 1000 Elo.
arXiv:2606.30410 · 42▲ · cs.LG, cs.AI
- Data
- Training
The paper introduces BeyondArena, a benchmark of 142 manually curated tabular datasets that spans IID, temporal, and grouped (non-IID) prediction tasks across sample sizes from 100 to 1 million rows, plus DataFoundry, a Python framework for reproducible dataset curation. It evaluates 11 models including three tabular foundation models (TabPFN-2.6, TabICLv2, TabDPT) against gradient-boosted trees and MLPs using in-context learning for the foundation models and tuning plus ensembling for the traditional ones.
Full breakdown →Morphing into Hybrid Attention Models
On Qwen3-1.7B NIAH retrieval, FlashMorph matches or beats the strongest baseline (HALO) on the hardest NIAH-Single-3 setting (94.4 at 128K, 73.2 at 256K vs HALO's 67.4 and 52.8) while using 20M layer-selection tokens vs HALO's 234M and PostNAS's 50B. Caveat: at the largest 30B-A3B MoE scale its recall-intensive numbers trail or roughly match uniform interleaving, so the advantage is clearest at smaller scales.
arXiv:2606.30562 · 42▲ · cs.CL
- LLMs
- Efficiency
- Inference
FlashMorph is a method for choosing which layers of a pretrained Transformer to keep as full attention when converting the rest to linear attention (a hybrid model). Instead of fixed placement rules or scoring layers one at a time, it attaches a frozen linear-attention branch to each layer, adds a learnable scalar gate per layer, and jointly optimizes all gates on synthetic passkey-retrieval data with a regularizer that pushes layers toward linear attention, then keeps the top-K layers by gate value.
Full breakdown →Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.