Issue #2
Sent 2026-05-18
CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence
The best system, Gemini-3.1-Pro-Preview, reached only 76.0 SAA overall while scoring 86.1 on answer correctness, and the best open-source model (Qwen3-VL-235B) hit just 22.5 SAA. Note answer and relevance scores come from an LLM judge (Qwen3-VL-235B), and recall uses a lenient IoU 0.5 threshold, so the absolute numbers depend on those judge and threshold choices.
arXiv:2605.12882 · 274▲ · cs.CL, cs.CV
- Multimodal
- Vision
- Safety
CiteVQA is a document VQA benchmark that requires models to return element-level bounding-box citations alongside each answer, then scores both jointly with a metric called Strict Attributed Accuracy (SAA), which only credits a sample when the answer is correct AND the cited region matches ground truth. Ground-truth citations are generated by an automated pipeline (document parsing plus MLLM agents plus masking-based ablation to find crucial evidence) and validated by expert review across 1,897 questions from 711 multi-page PDFs.
Full breakdown →MinT: Managed Infrastructure for Training and Serving Millions of LLMs
Adapter-only handoff cut the measured training-to-serving handoff step by 18.3x on a 4B dense model (252 MiB adapter and 0.036s load vs 8.06 GB checkpoint and 71.8s materialize) and 2.85x on a 30B MoE model. The million-adapter claim is a built-and-audited catalog artifact (1,000,000 adapters, 256 sampled read-OK), not a claim of simultaneous GPU residency, and warm serving held 100% of a 5s TTFT SLO only through 2 requests/sec (dropping to 72% at 4 rps) on a single 64-adapter TP=4 engine.
arXiv:2605.13779 · 223▲ · cs.LG, cs.AI, cs.DC
- LLMs
- Inference
- Efficiency
MinT is a managed infrastructure system for LoRA post-training and serving that keeps expensive base models resident on GPUs and moves only exported LoRA adapter revisions (not merged full checkpoints) through training, export, serving, and rollback. It provides a Tinker-compatible service API on top of vLLM samplers and PEFT/Megatron trainers, with a policy-record abstraction that tracks each adapter's base version, rank, checkpoint, and cache tier.
Full breakdown →SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
The 2B NEO-unify interface reaches 31.56 PSNR / 0.85 SSIM on MS-COCO 2017 reconstruction after 90K steps, versus 31.56 PSNR / 0.93 SSIM for the FLUX.1-dev VAE (comparable PSNR, lower SSIM). Caveat: this is the smaller predecessor model at a 32x compression ratio, and most head-to-head benchmark tables for the full U1 models are described qualitatively rather than shown here.
arXiv:2605.12500 · 194▲ · cs.CV
- Multimodal
- Vision
- Training
SenseNova-U1 is a multimodal model that handles both understanding (image/text comprehension) and image generation in a single architecture that operates directly on raw pixels and text, dropping the usual pretrained vision encoder and VAE. It uses a Mixture-of-Transformers (MoT) backbone where a shared attention path processes both streams, training understanding with next-token prediction and generation with pixel-space flow matching. Two variants ship: a dense 8B and a 30B mixture-of-experts model with about 3B active parameters.
Full breakdown →Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling
On IMO-ProofBench, SU-01 scored 57.6% with direct generation and 70.2% with test-time scaling, versus 23.1% for the similar-size Qwen3.6-35B-A3B baseline. Caveat: the headline competition results (35/35 gold-level points on IMO 2025 and USAMO 2026) were graded by human experts only in the TTS condition and rest on a handful of problems, and those TTS runs sustain 100K+ token trajectories, so the compute cost per problem is high.
arXiv:2605.13301 · 165▲ · cs.AI, cs.CL
- LLMs
- RL
- Inference
The paper describes a training recipe that turns a 30B-A3B post-trained reasoning model into an olympiad-level math and physics solver, using three stages: supervised fine-tuning ordered by reverse-perplexity curriculum, a two-stage RL pipeline (verifiable-reward RL followed by proof-quality RL with a generative reward model, self-refinement, and experience replay), and a test-time solve-verify-refine loop. The resulting model, SU-01, is trained on 338K sub-8K-token SFT trajectories plus 200 RL steps.
Full breakdown →PhysBrain 1.0 Technical Report
On real-world Franka manipulation, PhysBrain 1.0 raised average single-object grasping success from 47.1% to 63.3% and long-horizon task success from 31.0% to 45.0% versus a pi_0.5 baseline, over 50 trials per category. Note the paper reports SOTA across many benchmarks (ERQA, PhysBench, SimplerEnv, LIBERO, RoboCasa) but provides limited head-to-head detail, and the arxiv metadata dates are anomalous.
arXiv:2605.15298 · 145▲ · cs.RO, cs.AI, cs.CL, cs.CV
- Robotics
- Multimodal
- Data
PhysBrain 1.0 is a vision-language-action (VLA) pipeline that converts large-scale egocentric human video into structured physical supervision. A data engine parses clips into JSON scene records (objects, spatial dynamics, action execution, depth relations), renders those into natural-language question-answer pairs to train a base VLM, then adapts that model to robot control with a design meant to preserve general multimodal ability.
Full breakdown →MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
On the PetFinder dataset, TabPFN-2.5 with full trimodal target-aware representations reached 88.0% AUC versus 84.9% for the same frozen trimodal setup and 83.7% for frozen structured-plus-image, a roughly 3 point gain; the paper reports the target-aware advantage holds across all learners, encoder sizes, and PCA dimensions but notes the curation pipeline selects for datasets where tuning helps, so absolute rankings are biased.
arXiv:2605.10616 · 142▲ · cs.LG, cs.CL, cs.CV
- Multimodal
- Data
- Training
MulTaBench is a benchmark of 40 datasets (20 image-tabular, 20 text-tabular) for multimodal tabular learning, curated by an automated pipeline that only keeps datasets where each modality adds independent predictive signal and where task-specific tuning of the encoder beats frozen off-the-shelf embeddings. The curation compares four conditions (unimodal, joint frozen, joint target-aware) across five tabular learners, using LoRA finetuning of the top 3 layers of e5 (text) and DINO-v3 (image) encoders as a preprocessing step to produce Target-Aware Representations.
Full breakdown →Self-Distilled Agentic Reinforcement Learning
On Qwen2.5-3B, SDAR improved ALFWorld success rate by +9.4% (84.4 vs 75.0), Search-QA by +7.0%, and WebShop accuracy by +4.7% over GRPO, with similar gains on 7B. Caveat: WebShop is evaluated on only 128 fixed validation tasks, training runs are 150 steps, and gains over the stronger hybrid baselines (Skill-SD, RLSD) are smaller and inconsistent on some subtasks.
arXiv:2605.15155 · 116▲ · cs.LG, cs.AI, cs.CL
- RL
- Agents
- LLMs
SDAR trains multi-turn LLM agents by combining GRPO reinforcement learning with an auxiliary on-policy self-distillation loss, where a teacher branch is the same policy given privileged training-only context (retrieved skills). The distillation signal is filtered through a per-token sigmoid gate that strengthens supervision on tokens the teacher endorses (positive teacher-student log-probability gap) and softly attenuates tokens where the teacher disagrees.
Full breakdown →Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
In frame-wise 2-step generation on Wan2.1-1.3B, Causal Forcing++ scores 84.14 VBench Total (vs 84.04 for 4-step chunk-wise Causal Forcing) and 6.661 VisionReward (vs 6.326), while cutting first-frame latency 50% (0.27s vs 0.60s) and Stage 2 training cost about 4x (2,900 vs 11,600 A800 GPU-hours). Gains over the prior SOTA baseline are small in absolute terms; the main win is efficiency and latency, and evaluation uses only 100 prompts for VisionReward and dynamic-degree metrics.
arXiv:2605.15141 · 96▲ · cs.CV
- Vision
- Efficiency
- Training
The paper distills bidirectional video diffusion models into few-step autoregressive students for real-time interactive video generation, replacing the expensive causal ODE initialization step of prior work (Causal Forcing) with causal consistency distillation (causal CD). Causal CD gets its training signal from a single online teacher ODE step between adjacent timesteps on real videos, avoiding the need to precompute and store full PF-ODE trajectories.
Full breakdown →Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.