Inference

Yafu Li, Runzhe Zhan, Haoran Zhang, Shunkai Zhang, Yizhuo Li, Zhilin Wang, Jiacheng Chen, Futing Wang, Xuyang Hu, Yuchen Fan, Bangjie Xu, Yucheng Su, Xinmiao Han, Chenxi Li, Haodi Lei, Yufeng Zhao, Zejin Lin, Qianjia Cheng, Tong Zhu, Xiaoye Qu, Ganqu Cui, Peng Ye, Yun Luo, Zhouchen Lin, Yu Qiao, Bowen Zhou, Ning Ding, Yu Cheng

The paper describes a training recipe that turns a 30B-A3B post-trained reasoning model into an olympiad-level math and physics solver, using three stages: supervised fine-tuning ordered by reverse-perplexity curriculum, a two-stage RL pipeline (verifiable-reward RL followed by proof-quality RL with a generative reward model, self-refinement, and experience replay), and a test-time solve-verify-refine loop. The resulting model, SU-01, is trained on 338K sub-8K-token SFT trajectories plus 200 RL steps.

Zhifei Xie, Zihang Liu, Ze An, Xiaobin Hu, Yue Liao, Ziyang Ma, Dongchao Yang, Mingbao Lin, Deheng Ye, Shuicheng Yan, Chunyan Miao

The paper builds Audio-Interaction, a streaming audio language model (fine-tuned from Qwen2.5-Omni-3B) that consumes audio in 400ms chunks and predicts a per-chunk control token (<silent> or <response>) to decide whether to speak or keep listening. It is trained via a framework called SoundFlow using a 2.6M-item synthetic streaming corpus (StreamAudio-2M) and served with a FIFO-scheduled asynchronous inference scheme that decouples encoding from decoding.

Hao Zhang, Yiming Hu, Yong Wang, Mingqiao Mo, Xin Xiao, Xiangxiang Chu

BlockPilot is a lightweight predictor that picks a per-sample block size for diffusion-based speculative decoding, replacing the fixed block size inherited from training. It uses the target model's last-token predictive distribution after prefilling as input to a two-layer MLP classifier that selects the block size from a small local range around the training value.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou

The paper introduces chain-of-thought prompting: instead of giving a large language model few-shot examples that map questions directly to answers, each exemplar includes the intermediate reasoning steps that lead to the answer. The model then generates its own step-by-step reasoning before producing a final answer, with no finetuning involved.

Liliana Hotsko, Yinxi Li, Yuntian Deng, Pengyu Nie

Code2LoRA is a hypernetwork that reads a code repository (compressed into embeddings by a frozen Qwen3-Embedding model) and generates a repository-specific LoRA adapter for a frozen code LLM in a single forward pass, so repo knowledge lives in parameters instead of the prompt. It has two variants: Static, which maps one repository snapshot to an adapter, and Evo, which uses a GRU to update a hidden state per commit diff so the adapter tracks an evolving codebase.

DreamX Team, Yancheng Bai, Rui Chen, Xiangxiang Chu, Rujing Dang, Hao Dou, Bingjie Gao, Qiwen Gu, Siyu Hong, Jiachen Lei, Geng Li, Jifan Li, Ruimin Lin, Qingfeng Shi, Bingze Song, Lei Sun, Jing Tang, Ruitian Tian, Jun Wang, Jiahong Wu, Pengfei Zhang, Shen Zhang, Jiashu Zhu

DreamX-World 1.0 is an interactive text/image-to-video world model built by fine-tuning Wan2.2 to support camera navigation, revisiting earlier scenes, and prompted multi-object events across photorealistic, game, and stylized domains. It combines a camera-conditioning method called E-PRoPE (projective positional encoding applied to spatially downsampled tokens), geometry-based memory retrieval, DMD distillation into a few-step autoregressive generator, and RL alignment, with serving optimizations to hit real-time streaming.

Jisang Han, Seonghu Jeon, Jaewoo Jung, René Zurbrügg, Honggyu An, Tifanny Portela, Marco Hutter, Marc Pollefeys, Seungryong Kim, Sunghwan Hong

GAM (Geometric Action Model) is a robot manipulation policy that repurposes a pretrained geometric foundation model (a transformer that infers 3D depth/geometry from images) by splitting it at an intermediate layer, using the shallow layers as an observation encoder and inserting a causal transformer that predicts future latent tokens conditioned on language, proprioception, and action history. The remaining deep layers of the same backbone then decode those predicted tokens into both future depth maps and robot action chunks in a single forward pass.

Siyin Wang, Junhao Shi, Senyu Fei, Zhaoyang Fu, Li Ji, Jingjing Gong, Xipeng Qiu

In-Context World Modeling (ICWM) is a training and inference method for Vision-Language-Action (VLA) robot policies that prepends a short sequence of self-generated random exploratory movements (recorded as start-image, action, end-image triplets) as context before the task query. The Transformer backbone infers the system configuration (camera viewpoint, kinematics) implicitly from this context in a single forward pass, without any parameter updates or task demonstrations at test time.

Lorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang, Lukas Cavigelli

KVarN is a calibration-free KV cache quantizer that combines a Hadamard rotation in the channel dimension with an iterative Sinkhorn-style variance normalization across both the token and channel axes of the K and V matrices before round-to-nearest quantization. It targets the specific error accumulation that happens when the cache is quantized on-the-fly during long autoregressive decoding, rather than the parallel prefill setting most prior methods were tuned for.

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei, Yangzhou Liu, Zhiqi Li, Yunze Man, Guo Chen, Andrew Tao, Guilin Liu, Jan Kautz, Lei Zhang, Zhiding Yu

LocateAnything is a vision-language model for object detection and grounding that uses Parallel Box Decoding (PBD), which predicts all four coordinates of a bounding box in a single forward step instead of generating them token by token. It treats each box (or point) as an atomic block with bidirectional intra-block attention, trained jointly with a standard next-token prediction objective and offering three inference modes (Fast, Slow, Hybrid).

Jian Yang, Shawn Guo, Wei Zhang, Tianyu Zheng, Yaxin Du, Haau-Sing Li, Jiajun Wu, Yue Song, Yan Xing, Qingsong Cai, Zelong Huang, Chuan Hao, Ran Tao, Xianglong Liu, Wayne Xin Zhao, Mingjie Tang, Weifeng Lv, Ming Zhou, Bryan Dai

The paper studies how many times to loop a Parallel Loop Transformer (PLT), a looped-Transformer variant that reuses shared blocks with cross-loop position offsets (CLP) and shared-KV gated sliding-window attention to keep latency and KV-cache memory flat regardless of loop count. It trains LoopCoder-v2, a 7B coder, from scratch on 18T tokens at loop counts of 1, 2, 3, and 4, and uses per-loop diagnostics (hidden-state dynamics, attention routing, output-distribution shift) to explain why performance peaks at two loops.

Jiacheng Chen, Xinyu Zhang, Shunkai Zhang, Yanmohan Wang, Lin Li, Tiancheng Qin, Qin Wang, Zhengmao Zhu, Tianle Li, Jingyang Li, Zehan Li, Binyang Jiang, Jin Zhu, Han Ding, Fei Yu, Chenyu Du, Zijian Song, Jiayuan Song, Zhi Zhang, Yunan Huang, Weiyu Cheng, Pengyu Zhao, Yu Cheng

MaxProof is a test-time scaling framework for competition math proofs that treats one merged model as generator, verifier, refiner, and ranker, then runs an evolution-style population search (dual PATCH/REWRITE refinement, conservative min-score fitness, pairwise tournament final selection). The underlying MiniMax-M3 model is trained through three specialist stages (proof generation via CISPO RL under a four-layer generative verifier, error-finding verification, and critique-conditioned repair) before being merged into a single release model.

Mind Lab, :, Song Cao, Vic Cao, Andrew Chen, Kaijie Chen, Cleon Cheng, Steven Chiang, Kaixuan Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Hongquan Gu, Aaron Guan, Nolan Ho, Mutian Hong, Hailee Hou, Peixuan Hua, Charles Huang, Miles Jiang, Nora Jiang, Yuyi Jiang, Qiuyu Jin, Fancy Kong, Andrew Lei, Kyrie Lei, Alexy Li, Lucian Li, Ray Li, Theo Li, Zhihui Li, Jiayi Lin, Kairus Liu, Kieran Liu, Logan Liu, Xiang Liu, Irvine Lu, Maeve Luo, Runze Lv, Pony Ma, Verity Niu, Anson Qiu, Vincent Wang, Rio Yang, Maxwell Yao, Carrie Ye, Regis Ye, Wenlin Ye, Josh Ying, Danney Zeng, Yuhan Zhan, Anya Zhang, Di Zhang, Ruijia Zhang, Sueky Zhang, Ya Zhang, Wei Zhao, Ada Zhou, Changhai Zhou, Yuhua Zhou, Xinyue Zhu, Murphy Zhuang

MinT is a managed infrastructure system for LoRA post-training and serving that keeps expensive base models resident on GPUs and moves only exported LoRA adapter revisions (not merged full checkpoints) through training, export, serving, and rollback. It provides a Tinker-compatible service API on top of vLLM samplers and PEFT/Megatron trainers, with a policy-record abstraction that tracks each adapter's base version, rank, checkpoint, and cache tier.

Disen Lan, Jianbin Zheng, Yuxi Ren, Xin Xia, Xuanda Wang, Xuefeng Xiao, Xipeng Qiu, Yu Cheng

FlashMorph is a method for choosing which layers of a pretrained Transformer to keep as full attention when converting the rest to linear attention (a hybrid model). Instead of fixed placement rules or scoring layers one at a time, it attaches a frozen linear-attention branch to each layer, adds a learnable scalar gate per layer, and jointly optimizes all gates on synthetic passkey-retrieval data with a regularizer that pushes layers toward linear attention, then keeps the top-K layers by gate value.

Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng

The paper introduces Program-as-Weights (PAW), a system that compiles a natural-language description of a fuzzy task (things like log alerting, JSON repair, or intent classification that resist clean rule-based code) into a small neural artifact. A 4B compiler reads the spec plus an auto-generated pseudo-program of paraphrase-and-examples, then emits a per-function LoRA adapter that gets hot-attached to a frozen 0.6B interpreter model, which runs the task locally.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela

This paper introduces Retrieval-Augmented Generation (RAG), a fine-tuning recipe that pairs a pre-trained seq2seq generator (BART-large) with a neural retriever (Dense Passage Retriever) that pulls passages from a dense vector index of Wikipedia. It presents two variants: RAG-Sequence, which conditions the whole output on one retrieved document, and RAG-Token, which can draw a different document per generated token, with both retriever and generator trained end-to-end using only input/output pairs.

Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, Beomjun Kim, Byungjun Yoon, Changsung Jang, Daewon Choi, Dongsu Han, Donguk Lee, Heeseung Kwon, Hojin Jeon, Jaehyun Kang, Jaekyoung Bae, Jihyuk Lee, Jimin Lee, John Won, Joonwoo Ahn, Junhyeong Park, Junyoung Sung, Kyungmin Lee, Minseong Han, Minsung Yoon, Sejune Joo, Seonil Son, Seungcheol Park, Seunggeun Cho, Seungjun Moon, Seungku Kim, Yonghoon Dong, Yongjin Cho, Youngchan Kim, Chang Hwan Kim, Dohyeon Kim, Heecheol Kim, Heewon Lee, Hensen Ahn, Hyungkyu Ryu, Hyunsoo Choi, Hyunsoo Shin, Jaeheon Jung, Jaewoo Kim, Jinwook Kim, Joochul Chang, Joonsoo Kim, Junghun Park, Jungwoo Park, Junho Cho, Junhyeok Park, Junwon Lee, Kangwook Lee, Kwanghoon Kim, Kyoungwhan Choe, Manoj Bhadu, Nayoung Oh, Sangjun Kim, Sangwoo Kim, Seunghoon Shim, Seunghyun Kim, Seungjun Lee, Seungyup Ka, Sungryol Yang, Wook Jung, Yashu Shukla, Yeonjae Lee, Yeonwoo Bae, Jinwoo Shin

RLDX-1 is a Vision-Language-Action model for robotic manipulation built on the Multi-Stream Action Transformer (MSAT), which extends the Multi-Modal Diffusion Transformer to route each input modality (vision-language cognition, proprioception, actions, and physical signals like torque and tactile) through a dedicated stream coupled by joint self-attention. It adds a motion module for temporal dynamics, an explicit memory queue for long-horizon tasks, a synthetic data pipeline using video generation with motion-consistency filtering, and inference optimizations (CUDA Graph capture plus custom fused kernels).

Yijing Tu, Shaojin Wu, Mengqi Huang, Wenchuan Wang, Yuxin Wang, Chunxiao Liu, Zhendong Mao

Stream-T1 is a test-time scaling framework built on top of a streaming (chunk-by-chunk autoregressive) video diffusion model (LongLive). It combines three inference-time tricks: initializing each chunk's noise by spherical interpolation from the previous chunk's noise, beam search pruning of chunk candidates using a combined image-reward plus video-reward score, and reward-guided routing of evicted KV cache entries into discard, EMA-merge, or append pathways.

Lianghua Huang, Zhi-Fan Wu, Wei Wang, Yupeng Shi, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuzheng Wang, Zoubin Bi

Wan-Streamer is a single Transformer that handles text, audio, and video as both input and output, using block-causal attention plus conditional flow matching to generate synchronized speech and video responses in a streaming, full-duplex manner. It replaces the usual cascade of separate VAD, ASR, LLM, TTS, and avatar-rendering modules with one end-to-end model built around causal VAEs, causal encoders/decoders, and a thinker-performer split at inference time.

Songhao Wu, Zhongxin Chen, Yuxuan Liu, Heng Cui, Cong Li, Rui Yan

The paper introduces EmbedFilter, a linear post-processing transformation that improves text embeddings extracted zero-shot from LLMs. It works by taking the SVD of the LLM's unembedding matrix and projecting embeddings onto only the mid-range ('bulk') singular vectors, discarding the 'edge spectrum' (largest and smallest singular values) that the authors show encodes high-frequency, semantically empty tokens.