SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
Haiwen Diao, Penghao Wu, Hanming Deng, Jiahao Wang, Shihao Bai, Silei Wu, Weichen Fan, Wenjie Ye, Wenwen Tong, Xiangyu Fan, Yan Li, Yubo Wang, Zhijie Cao, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Yuwei Niu, Yue Zhu, Bo Liu, Chengguang Lv, Haojia Yu, Haozhe Xie, Hongli Wang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jingcheng Ni, Junxiang Xu, Kaihuan Liang, Lianqiang Shi, Linjun Dai, Linyan Wang, Oscar Qian, Peng Gao, Pengfei Liu, Qingping Sun, Rui Shen, Ruisi Wang, Shengnan Ma, Shuang Yang, Siyi Xie, Siying Li, Tianbo Zhong, Xiangli Kong, Xuanke Shi, Yang Gao, Yongqiang Yao, Yves Wang, Zhengqi Bai, Zhengyu Lin, Zixin Yin, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, Dahua Lin
arXiv:2605.12500 · 194▲ · cs.CV
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
SenseNova-U1 is a multimodal model that handles both understanding (image/text comprehension) and image generation in a single architecture that operates directly on raw pixels and text, dropping the usual pretrained vision encoder and VAE. It uses a Mixture-of-Transformers (MoT) backbone where a shared attention path processes both streams, training understanding with next-token prediction and generation with pixel-space flow matching. Two variants ship: a dense 8B and a 30B mixture-of-experts model with about 3B active parameters.
Why it matters
The dominant approach today stitches together separate encoders, tokenizers, and diffusion heads for perception versus generation, which fragments training pipelines and representation spaces. A single end-to-end model that learns from pixels directly simplifies system design and, per the paper, reaches roughly VAE-level reconstruction quality without a pretrained VAE, though this remains a research artifact rather than something you would deploy today.
Practical takeaway
Watch for encoder-free unified models as an alternative to modular VLM-plus-diffusion stacks. Code, weights, and a demo are released, so you can try the 8B or A3B variants directly, and note the disaggregated serving design (LightLLM for understanding, LightX2V for generation) if you want to run understanding and generation on separate GPU groups.
Key result
The 2B NEO-unify interface reaches 31.56 PSNR / 0.85 SSIM on MS-COCO 2017 reconstruction after 90K steps, versus 31.56 PSNR / 0.93 SSIM for the FLUX.1-dev VAE (comparable PSNR, lower SSIM). Caveat: this is the smaller predecessor model at a 32x compression ratio, and most head-to-head benchmark tables for the full U1 models are described qualitatively rather than shown here.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.