Multimodal

Zhifei Xie, Zihang Liu, Ze An, Xiaobin Hu, Yue Liao, Ziyang Ma, Dongchao Yang, Mingbao Lin, Deheng Ye, Shuicheng Yan, Chunyan Miao

The paper builds Audio-Interaction, a streaming audio language model (fine-tuned from Qwen2.5-Omni-3B) that consumes audio in 400ms chunks and predicts a per-chunk control token (<silent> or <response>) to decide whether to speak or keep listening. It is trained via a framework called SoundFlow using a 2.6M-item synthetic streaming corpus (StreamAudio-2M) and served with a FIFO-scheduled asynchronous inference scheme that decouples encoding from decoding.

Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang, Jiahao Kong, Weijun Zeng, Jutao Xiao, Jie Yang, Wentao Zhang, Bin Wang, Conghui He

CiteVQA is a document VQA benchmark that requires models to return element-level bounding-box citations alongside each answer, then scores both jointly with a metric called Strict Attributed Accuracy (SAA), which only credits a sample when the answer is correct AND the cited region matches ground truth. Ground-truth citations are generated by an automated pipeline (document parsing plus MLLM agents plus masking-based ablation to find crucial evidence) and validated by expert review across 1,897 questions from 711 multi-page PDFs.

Hongcan Guo, Qinyu Zhao, Yian Zhao, Shen Nie, Rui Zhu, Qiushan Guo, Feng Wang, Tao Yang, Hengshuang Zhao, Guoqiang Wei, Yan Zeng

CoLa DLM is a text generation model that replaces left-to-right autoregression with a three-part pipeline: a Text VAE maps text to continuous latent variables, a block-causal Diffusion Transformer (DiT) learns a semantic prior over those latents via Flow Matching, and a conditional decoder turns generated latents back into text. The key design choice is that diffusion transports a latent prior (global semantics) rather than recovering token-level observations, separating semantic planning from token realization.

NVIDIA, :, Aditi, Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, Aarti Basant, Mukesh Beladiya, Mohammad Qazim Bhat, Zaid Pervaiz Bhat, Dan Blick, Vanni Brighella, Han Cai, Tiffany Cai, Eric Cameracci, Jiaxin Cao, Yulong Cao, Mark Carlson, Carlos Casanova, Ting-Yun Chang, Yan Chang, Yu-Wei Chao, Prithvijit Chattopadhyay, Roshan Chaudhari, Chieh-Yun Chen, Junyu Chen, Ke Chen, Qizhi Chen, Wenkai Chen, Xiaotong Chen, Yu Chen, An-Chieh Cheng, Click Cheng, Xiu Chia, Jeana Choi, Chaeyeon Chung, Wenyan Cong, Yin Cui, Magdalena Dadela, Nalin Dadhich, Wenliang Dai, Joyjit Daw, Alperen Degirmenci, Rodrigo Vieira Del Monte, Robert Denomme, Sameer Dharur, Marco Di Lucca, Ke Ding, Wenhao Ding, Yifan Ding, Yuzhu Dong, Nicole Drumheller, Yilun Du, Aigul Dzhumamuratova, Aleksandr Efitorov, Hamid Eghbalzadeh, Naomi Eigbe, Imad El Hanafi, Hassan Eslami, Benedikt Falk, Jiaojiao Fan, Jim Fan, Amol Fasale, Sergiy Fefilatyev, Liang Feng, Francesco Ferroni, Sanja Fidler, Xiao Fu, Vikram Fugro, Prashant Gaikwad, TJ Galda, Katelyn Gao, Yihuai Gao, Wenhang Ge, Sreyan Ghosh, Arushi Goel, Vivek Goel, Akash Gokul, Rama Govindaraju, Jinwei Gu, Miguel Guerrero, Elfie Guo, Aryaman Gupta, Siddharth Gururani, Hugo Hadfield, Song Han, Ankur Handa, Zekun Hao, Mohammad Harrim, Ali Hassani, Nathan Hayes-Roth, Yufan He, Chris Helvig, Cyrus Hogg, Madison Huang, Michael Huang, Sophia Huang, Yufan Huang, Jacob Huffman, DeLesley Hutchins, Suneel Indupuru, Boris Ivanovic, Arihant Jain, Joel Jang, Ryan Ji, Yanan Jian, Dongfu Jiang, Jingyi Jin, Atharva Joshi, Nikhilesh Joshi, Pranjali Joshi, Andy Ju, Jaehun Jung, Weiwei Kang, Scott Kassekert, Jan Kautz, Ashna Khetan, Julia Kiczka, Slawek Kierat, Gwanghyun Kim, Kuno Kim, Sunny Kim, Kezhi Kong, Xin Kong, Zhifeng Kong, Tomasz Kornuta, Egor Krivov, Hui Kuang, Saurav Kumar, Chia-Wen Kuo, George Kurian, Wojciech Kutak, JF Lafleche, Himangshu Lahkar, Omar Laymoun, Jayjun Lee, Sanggil Lee, Gabriele Leone, Boyi Li, Freya Li, Jiajun Li, Jinfeng Li, Ling Li, Pengcheng Li, Shangru Li, Tingle Li, Xiaolong Li, Xuan Li, Zhaoshuo Li, Zhiqi Li, Hao Liang, Maosheng Liao, Chen-Hsuan Lin, Tsung-Yi Lin, Ming-Yu Liu, Sifei Liu, Zihan Liu, Hai Loc Lu, Xiangyu Lu, Alice Luo, Ruipu Luo, Wenjie Luo, Jiangran Lyu, Martin Ding Ma, Nic Ma, Qianli Ma, Dawid Majchrowski, Louis Marcoux, Miguel Martin, Qing Miao, Ashkan Mirzaei, Shreyas Misra, Kaichun Mo, Durra Mohsin, Hyejin Moon, Pawel Morkisz, Saeid Motiian, Kirill Motkov, Seungjun Nah, Yashraj Narang, Deepak Narayanan, Thabang Ngazimbi, Julian Ouyang, Shubham Pachori, David Page, Yatian Pang, Sehwi Park, Mahesh Patekar, Mostofa Patwary, Marco Pavone, Trung Pham, Wei Ping, Soha Pouya, Shrimai Prabhumoye, Varun Praveen, Delin Qu, Hesam Rabeti, Morteza Ramezanali, Marilyn Reeb, Xuanchi Ren, Kristen Rumley, Wojciech Rymer, Jun Saito, Yeongho Seol, John Shao, Piyush Shekdar, Tianwei Shen, Humphrey Shi, Min Shi, Stella Shi, Kevin Shih, Mohammad Shoeybi, Mateusz Sieniawski, Shuran Song, Alexander Sotelo, Amir Sotoodeh, Sunil Srinivasa, Vignesh Srinivasakumar, Bartosz Stefaniak, Rahul Heinrich Steiger, Shangkun Sun, Jiaxiang Tang, Shitao Tang, Yangyang Tang, Yue Tang, Tolou Tavakkoli, Kayley Ting, Krzysztof Tomala, Wei-Cheng Tseng, Jibin Varghese, Sergei Vasilev, Thomas Volk, Raju Wagwani, Roger Waleffe, Andrew Z. Wang, Boxiang Wang, Haoxiang Wang, Qiao Wang, Shihao Wang, Shijie Wang, Ting-Chun Wang, Yan Wang, Yu Wang, Rohit Watve, David Wehr, Fangyin Wei, Xinshuo Weng, Jay Zhangjie Wu, Kedi Wu, Hongchi Xia, Summer Xiao, Tianjun Xiao, Kevin Xie, Daguang Xu, Jiashu Xu, Mengyao Xu, Ruqing Xu, Xingqian Xu, Yao Xu, Dinghao Yang, Dong Yang, Hans Yang, Xiaodong Yang, Xuning Yang, Yichu Yang, Yurong You, Zhiding Yu, Hao Yuan, Simon Yuen, Xiaohui Zeng, Pengcuo Zeren, Cindy Zha, Haotian Zhang, Jenny Zhang, Jing Zhang, Liangkai Zhang, Paris Zhang, Shun Zhang, Xuanmeng Zhang, Zhizheng Zhang, Ann Zhao, Yilin Zhao, Yuliya Zhautouskaya, Charles Zhou, Fengzhe Zhou, Shilin Zhu, Yuke Zhu, Dima Zhylko, Artur Zolkowski

Cosmos 3 is a family of omnimodal models (4B, 16B, 64B params) that jointly process and generate language, image, video, audio, and action within a single Mixture-of-Transformers architecture. It uses a dual-tower design where one parameter set handles autoregressive reasoning (next-token prediction) and a separate set handles diffusion-based generation of pixels/audio/actions, with the two towers sharing a joint attention operation and both initialized from a pretrained Qwen3-VL.

Nan Chen, Yiyang Cai, Rongchang Xie, Junwen Pan, Cheng Chen, Weinan Jia, Zhuowei Chen, Wen Zhou, Zhenbang Sun, Wenhan Luo

DomainShuttle is a subject-driven text-to-video system built on the Wan2.1/2.2 DiT video models that adds three components: Domain-MoT (separate transformer branches for reference images versus video plus a domain-aware AdaLN modulation), Video-Reference DualRoPE (placing reference tokens in a separate positional encoding space from video tokens), and a Cross-Pair Consistent Loss that aligns two different reference sets of the same subject. The goal is to preserve a reference subject's intrinsic features while still allowing style, domain, and semantic changes driven by the text prompt (cross-domain generation), rather than only copying the reference (in-domain).

Yifu Yuan, Yaoting Huang, Xianze Yao, Yutong Li, Shuoheng Zhang, Linqi Han, Pengyi Li, Jiangeng Sun, Wenting Jia, Zhao Zhang, Yuhao Liu, Ruihao Liao, Yucheng Hu, Qiyu Wu, Yuxiao Li, Zibin Dong, Fei Ni, Yan Zheng, Shuyang Gu, Yi Ma, Hongyao Tang, Han Hu, Jianye Hao

Embodied-R1.5 is an 8B-parameter vision-language model fine-tuned to unify three robotics-relevant capabilities (spatial reasoning, task planning/correction, and pointing/trajectory generation) in one model, trained via supervised fine-tuning followed by reinforcement learning with verifiable rewards on 15B tokens of mostly auto-generated data. It also ships a Planner-Grounder-Corrector loop where a single model instance plays all three roles for long-horizon tasks, and can be extended into a robot action policy (VLA) by attaching a flow-matching action head.

Jisang Han, Seonghu Jeon, Jaewoo Jung, René Zurbrügg, Honggyu An, Tifanny Portela, Marco Hutter, Marc Pollefeys, Seungryong Kim, Sunghwan Hong

GAM (Geometric Action Model) is a robot manipulation policy that repurposes a pretrained geometric foundation model (a transformer that infers 3D depth/geometry from images) by splitting it at an intermediate layer, using the shallow layers as an observation encoder and inserting a causal transformer that predicts future latent tokens conditioned on language, proprioception, and action history. The remaining deep layers of the same backbone then decode those predicted tokens into both future depth maps and robot action chunks in a single forward pass.

Siyin Wang, Junhao Shi, Senyu Fei, Zhaoyang Fu, Li Ji, Jingjing Gong, Xipeng Qiu

In-Context World Modeling (ICWM) is a training and inference method for Vision-Language-Action (VLA) robot policies that prepends a short sequence of self-generated random exploratory movements (recorded as start-image, action, end-image triplets) as context before the task query. The Transformer backbone infers the system configuration (camera viewpoint, kinematics) implicitly from this context in a single forward pass, without any parameter updates or task demonstrations at test time.

Kwai Keye Team, Bin Wen, Changyi Liu, Chengru Song, Chongling Rao, Guowang Zhang, Han Li, Haonan Fan, Hengrui Ju, Jiankang Chen, Jiapeng Chen, Jiawei Yuan, Kaixuan Yang, Kaiyu Jiang, Kun Gai, Lingzhi Zhou, Na Nie, Sen Na, Tianke Zhang, Tingting Gao, Xuanyu Zheng, Yulong Chen, Fan Yang, Haixuan Gao, Lele Yang, Mingqiao Liu, Muxi Diao, Qi Zhang, Qile Su, Wei Chen, Wentao Hong, Xingyu Lu, Yancheng Long, Yankai Yang, Yingxin Li, Yiyang Fan, Yu Xia, Yuzhe Chen, Ziliang Lai, Chuan Yi, Haonan Jia, Tianming Liang, Weixin Xu, Xiaoxiao Ma, Yang Tian, Yufei Han, Feng Han, Hang Li, Jing Wang, Jinghui Jia, Junmin Chen, Junyu Shi, Ruilin Zhang

A technical report on Kwai Keye-VL-2.0-30B-A3B, an open-source Mixture-of-Experts multimodal model (30B total, 3B active) that adapts DeepSeek Sparse Attention (DSA) to a GQA-based backbone so it can process 256K-token contexts, aimed at hour-long video understanding. It also describes a post-training recipe combining Cross-Modal Multi-Teacher On-Policy Distillation (MOPD) with Context-RL and Video-RL to add agent, code, and search skills without wiping out base reasoning.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever

This is the CLIP paper. It trains an image encoder and a text encoder jointly on 400 million (image, text) pairs scraped from the internet, using a contrastive objective that predicts which caption goes with which image within a batch rather than predicting exact caption words. After pre-training, you build a classifier for any dataset by feeding the class names (as text prompts) through the text encoder, so the model classifies images it was never explicitly trained to label.

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei, Yangzhou Liu, Zhiqi Li, Yunze Man, Guo Chen, Andrew Tao, Guilin Liu, Jan Kautz, Lei Zhang, Zhiding Yu

LocateAnything is a vision-language model for object detection and grounding that uses Parallel Box Decoding (PBD), which predicts all four coordinates of a bounding box in a single forward step instead of generating them token by token. It treats each box (or point) as an atomic block with bidirectional intra-block attention, trained jointly with a standard next-token prediction objective and offering three inference modes (Fast, Slow, Hybrid).

Alan Arazi, Eilam Shapira, Shoham Grunblat, Mor Ventura, Elad Hoffer, Gioia Blayer, David Holzmüller, Lennart Purucker, Gaël Varoquaux, Frank Hutter, Roi Reichart

MulTaBench is a benchmark of 40 datasets (20 image-tabular, 20 text-tabular) for multimodal tabular learning, curated by an automated pipeline that only keeps datasets where each modality adds independent predictive signal and where task-specific tuning of the encoder beats frozen off-the-shelf embeddings. The curation compares four conditions (unimodal, joint frozen, joint target-aware) across five tabular learners, using LoRA finetuning of the top 3 layers of e5 (text) and DINO-v3 (image) encoders as a preprocessing step to produce Target-Aware Representations.

Jiwen Liu, Shujuan Li, Zhixue Fang, Xiaohan Li, Yan Zhou, Zijie Meng, Zhimin Zhang, Yawen Luo, Guoxin Zhang, Yu-Shen Liu, Pengfei Wan

OmniDirector is a video generation framework that clones camera motion from a reference video by rendering the extracted camera poses as a 'camera grid' (a video of grid lines moving through an empty 3D room), then feeding that grid into a Multi-Modal Diffusion Transformer via token concatenation. It also uses a hierarchical prompt expansion agent built on Qwen3-VL to describe camera motion (split into inter-shot and intra-shot descriptions) and fuse it with the reference image and user prompt at inference time.

Shuang Chen, Kaituo Feng, Hangting Chen, Wenxuan Huang, Dasen Dai, Quanxin Shou, Yunlong Lin, Xiangyu Yue, Shenghua Gao, Tianyu Pang

OpenSearch-VL is a fully open recipe (data, code, models) for training multimodal search agents that interleave reasoning with tool calls over images and web search. It combines a Wikipedia-graph-based data synthesis pipeline, a tool environment (text/image search, OCR, crop, sharpen, super-resolution, perspective correction), and a modified GRPO reinforcement learning algorithm with fatal-aware token masking and one-sided advantage clamping to salvage useful reasoning from trajectories that hit cascading tool failures.

Yihao Wang, Yuheng Ji, Mingyu Cao, Yanqing Shen, Runze Xiao, Huaihai Lyu, Senwei Xie, Euan Liu, Klara Tian, Tianfeng Long, Yichi Zhang, Zhengliang Cai, Ruike Chen, Jifan Zhao, Ruochuan Shi, Zihan Tang, Jing Lyu, Wenxing Tan, Ningbo Zhang, Yangtao Hu, Yuming Gao, Xiansheng Chen, Junkai Zhao, Congsheng Xu, Boan Zhu, Ziqi Wang, Yupu Feng, Qiongqiong Zhang, Yingli Zhao, Yulong Ao, Shaoxuan Xie, You Liu, Guocai Yao, Leiduo Zhang, Xiaodan Liu, Yunyan Zhang, Yance Jiao, Xinyan Yang, Jiaxing Wei, Xu Liu, Tengfei Pan, Shaokai Nie, Chunlei Men, Sen Cui, Xiaojie Jin, Hongyang Li, Jianlan Luo, Yao Mu, Yunchao Wei, Jun Yan, Hang Zhao, Xiaolong Zheng, Jiaming Li, Yonghua Lin, Tiejun Huang, Zhongyuan Wang, Pengwei Wang

Orca is an encoder-decoder model that learns a shared 'world latent space' from video and language by predicting the latent representation of adjacent frames (unconscious learning) and event-conditioned future states described by text (conscious learning), plus a standard VQA loss. After pretraining, the backbone (built on a Qwen VLM) is frozen and small task-specific decoders are trained for text, image prediction, and robot action generation to test whether the shared latent transfers.

Yueyi Sun, Yuhao Wang, Jason Li, Ye Tian, Tao Zhang, Jacky Mai, Yihan Wang, Haochen Wang, Jinbin Bai, Ling Yang, Yunhai Tong

PerceptionDLM is a multimodal diffusion language model that captions multiple image regions at once inside a single denoising pass, instead of the region-by-region autoregressive decoding used by prior models. It combines a SigLIP-2 vision encoder and an 8B LLaDA diffusion backbone with region prompting (learnable per-region embeddings), RoI-aligned feature replay, and a structured attention mask that isolates each region's tokens while sharing global context.

Caixin Kang, Tianyu Yan, Sitong Gong, Mingfang Zhang, Liangyang Ouyang, Ruicong Liu, Bo Zheng, Huchuan Lu, Kaipeng Zhang, Yoichi Sato, Yifei Huang

The paper introduces Grounded Personality Reasoning (GPR), a task that forces multimodal LLMs to justify each Big Five (OCEAN) personality rating with cited, timestamped behavioral evidence, and releases MM-OCEAN, a benchmark of 1,104 videos and 5,320 cue-grounding multiple-choice questions built by a multi-agent annotation pipeline with human verification. It benchmarks 27 MLLMs across three tiers (rating, open-ended reasoning, structured cue grounding) plus four failure-mode metrics that separate 'right answer' from 'right reasons'.

Shijie Lian, Bin Yu, Xiaopeng Lin, Changti Wu, Hang Yuan, Xiaolin Hu, Zhaolong Shen, Yuzhuo Miao, Haishan Liu, Yuxuan Tian, Yukun Shi, Cong Huang, Kai Chen

PhysBrain 1.0 is a vision-language-action (VLA) pipeline that converts large-scale egocentric human video into structured physical supervision. A data engine parses clips into JSON scene records (objects, spatial dynamics, action execution, depth relations), renders those into natural-language question-answer pairs to train a base VLM, then adapts that model to robot control with a design meant to preserve general multimodal ability.

Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, Xuhong Huang, Pei Lin, Junyang Lin, Dayiheng Liu, Shuai Bai, Jingren Zhou, Jiazhao Zhang, Haoqi Yuan, Gengze Zhou, Hang Yin, Ye Wang, Yiyang Huang, Zixing Lei, Wujian Peng, Delin Chen, Yingming Zheng, Jingyang Fan, Xianwei Zhuang, Xin Zhou, Haoyang Li, Anzhe Chen, Tong Zhang, Xuejing Liu, Yuchong Sun, Ruizhe Chen, Zhaohai Li, Chenxu Lü, Zhibo Yang, Tao Yu, Xionghui Chen

Qwen-VLA is a single vision-language-action model that puts a DiT-based flow-matching action decoder on top of the Qwen3.5-4B multimodal backbone, so one model handles robot manipulation, navigation, and trajectory prediction. It uses embodiment-aware text prompts (describing the robot platform, arm config, control frequency, and prediction horizon) instead of separate output heads per robot, and is trained in four stages: text-to-action decoder pretraining without images, multimodal continued pretraining, supervised fine-tuning, and PPO reinforcement learning.

Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, Beomjun Kim, Byungjun Yoon, Changsung Jang, Daewon Choi, Dongsu Han, Donguk Lee, Heeseung Kwon, Hojin Jeon, Jaehyun Kang, Jaekyoung Bae, Jihyuk Lee, Jimin Lee, John Won, Joonwoo Ahn, Junhyeong Park, Junyoung Sung, Kyungmin Lee, Minseong Han, Minsung Yoon, Sejune Joo, Seonil Son, Seungcheol Park, Seunggeun Cho, Seungjun Moon, Seungku Kim, Yonghoon Dong, Yongjin Cho, Youngchan Kim, Chang Hwan Kim, Dohyeon Kim, Heecheol Kim, Heewon Lee, Hensen Ahn, Hyungkyu Ryu, Hyunsoo Choi, Hyunsoo Shin, Jaeheon Jung, Jaewoo Kim, Jinwook Kim, Joochul Chang, Joonsoo Kim, Junghun Park, Jungwoo Park, Junho Cho, Junhyeok Park, Junwon Lee, Kangwook Lee, Kwanghoon Kim, Kyoungwhan Choe, Manoj Bhadu, Nayoung Oh, Sangjun Kim, Sangwoo Kim, Seunghoon Shim, Seunghyun Kim, Seungjun Lee, Seungyup Ka, Sungryol Yang, Wook Jung, Yashu Shukla, Yeonjae Lee, Yeonwoo Bae, Jinwoo Shin

RLDX-1 is a Vision-Language-Action model for robotic manipulation built on the Multi-Stream Action Transformer (MSAT), which extends the Multi-Modal Diffusion Transformer to route each input modality (vision-language cognition, proprioception, actions, and physical signals like torque and tactile) through a dedicated stream coupled by joint self-attention. It adds a motion module for temporal dynamics, an explicit memory queue for long-horizon tasks, a synthetic data pipeline using video generation with motion-consistency filtering, and inference optimizations (CUDA Graph capture plus custom fused kernels).

Haiwen Diao, Penghao Wu, Hanming Deng, Jiahao Wang, Shihao Bai, Silei Wu, Weichen Fan, Wenjie Ye, Wenwen Tong, Xiangyu Fan, Yan Li, Yubo Wang, Zhijie Cao, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Yuwei Niu, Yue Zhu, Bo Liu, Chengguang Lv, Haojia Yu, Haozhe Xie, Hongli Wang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jingcheng Ni, Junxiang Xu, Kaihuan Liang, Lianqiang Shi, Linjun Dai, Linyan Wang, Oscar Qian, Peng Gao, Pengfei Liu, Qingping Sun, Rui Shen, Ruisi Wang, Shengnan Ma, Shuang Yang, Siyi Xie, Siying Li, Tianbo Zhong, Xiangli Kong, Xuanke Shi, Yang Gao, Yongqiang Yao, Yves Wang, Zhengqi Bai, Zhengyu Lin, Zixin Yin, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, Dahua Lin

SenseNova-U1 is a multimodal model that handles both understanding (image/text comprehension) and image generation in a single architecture that operates directly on raw pixels and text, dropping the usual pretrained vision encoder and VAE. It uses a Mixture-of-Transformers (MoT) backbone where a shared attention path processes both streams, training understanding with next-token prediction and generation with pixel-space flow matching. Two variants ship: a dense 8B and a 30B mixture-of-experts model with about 3B active parameters.

Seokju Cho, Ryo Hachiuma, Abhishek Badki, Hang Su, Byung-Kwan Lee, Chan Hee Song, Sifei Liu, Subhashree Radhakrishnan, Seungryong Kim, Yu-Chiang Frank Wang, Min-Hung Chen

SpatialClaw is a training-free agent framework that gives a vision-language model a persistent Python kernel (preloaded with input frames, perception tools like SAM3 and Depth Anything 3, and libraries like NumPy/SciPy) and lets it write and run one code cell per step, inspecting intermediate results before writing the next cell. This 'code as the action interface' approach replaces two prior designs: single-pass code (commit to a full program before seeing any output) and structured JSON/XML tool-calls (fixed API menu).

Lianghua Huang, Zhi-Fan Wu, Wei Wang, Yupeng Shi, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuzheng Wang, Zoubin Bi

Wan-Streamer is a single Transformer that handles text, audio, and video as both input and output, using block-causal attention plus conditional flow matching to generate synchronized speech and video responses in a streaming, full-duplex manner. It replaces the usual cascade of separate VAD, ASR, LLM, TTS, and avatar-rendering modules with one end-to-end model built around causal VAEs, causal encoders/decoders, and a thinker-performer split at inference time.

Kaining Ying, Hengrui Hu, Siyu Ren, Jiamu Li, Fengjiao Chen, Ziwen Wang, Xuezhi Cao, Xunliang Cai, Henghui Ding

WBench is a benchmark for evaluating interactive video world models (systems that generate the next video frame conditioned on observation history plus a user action). It defines 289 test cases with 1,058 multi-turn interaction turns across five dimensions (video quality, setting adherence, interaction adherence, consistency, physics compliance), scored by 22 automatic sub-metrics that combine specialist vision models (MegaSaM pose estimation, SAM2, Depth Anything 3, DINOv2) with VLM-based grading.