PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models
Yueyi Sun, Yuhao Wang, Jason Li, Ye Tian, Tao Zhang, Jacky Mai, Yihan Wang, Haochen Wang, Jinbin Bai, Ling Yang, Yunhai Tong
arXiv:2606.19534 · 64▲ · cs.CV, cs.AI, cs.CL
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
PerceptionDLM is a multimodal diffusion language model that captions multiple image regions at once inside a single denoising pass, instead of the region-by-region autoregressive decoding used by prior models. It combines a SigLIP-2 vision encoder and an 8B LLaDA diffusion backbone with region prompting (learnable per-region embeddings), RoI-aligned feature replay, and a structured attention mask that isolates each region's tokens while sharing global context.
Why it matters
For dense region-captioning workloads, autoregressive models scale latency linearly with the number of regions because each region is decoded separately. A diffusion model that emits several region captions in parallel keeps per-image latency roughly flat as region count grows, which changes the cost profile for tasks like describing every masked object in an image.
Practical takeaway
If you run multi-region captioning at scale, watch diffusion VLMs as a way to trade a small quality hit for lower latency. Code, models, and the ParaDLC-Bench benchmark are released, so you can try parallel region decoding and compare it against AR baselines like GAR-8B directly.
Key result
On the authors' own ParaDLC-Bench, PerceptionDLM hits 62.4% average accuracy versus roughly 35% for diffusion baselines (LLaDA-V) and about 69% for AR region models (GAR, DAM), while cutting total benchmark inference time to 276s vs 479s for GAR. Caveat: accuracy is still below the AR models, the benchmark is new and self-constructed, and the 3.44x throughput speedup is measured under a fixed 4-masks-per-image workload.
Jargon
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.