Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation
Zhifei Xie, Kaiyu Pang, Haobin Zhang, Deheng Ye, Xiaobin Hu, Shuicheng Yan, Chunyan Miao
arXiv:2605.19833 · 137▲ · cs.SD, cs.AI, cs.CL, cs.MM, eess.AS
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
Mega-ASR fine-tunes Qwen3-ASR-1.7B to handle heavily degraded real-world audio using two techniques: Acoustic-to-Semantic Progressive Supervised Fine-Tuning (a WER-graded curriculum that trains the encoder, then the LLM, then jointly) and Dual-Granularity WER-Gated Policy Optimization (a reinforcement learning reward that switches between token-level and sentence-level scoring based on WER). It is trained on Voices-in-the-Wild-2M, a 2.4M-clip synthetic dataset built by simulating 7 atomic acoustic effects and 54 compound scenarios at the spectrogram level.
Why it matters
Standard ASR models collapse into empty outputs or hallucinations when audio is badly degraded (far-field, reverb, dropout stacked together), which is common in classrooms, vehicles, and phone recordings. This work targets that failure regime specifically and ships a plug-and-play routing setup so the noise-robust weights only activate on degraded audio, leaving clean-speech accuracy intact.
Practical takeaway
If you deploy ASR in adverse acoustic conditions, watch for the released Voices-in-the-Wild dataset and benchmark to stress-test your own models against compound degradations rather than single-factor noise. The routing trick (a LoRA binary classifier deciding whether to use robust vs. base weights) is a reusable pattern to add robustness without regressing clean-audio performance.
Key result
On the extreme NOIZEUS 0dB setting, Mega-ASR reaches 19.80 WER vs. 23.97 for Qwen3-ASR (its own base model), a 17.4% relative reduction; averaged across CHiME-4, VOiCES, and NOIZEUS it gets 6.70 vs. 7.93. Note most gains are measured against the same backbone it was fine-tuned from, and much of the training/eval data is synthetically simulated.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.