Introspective Diffusion Language Models

Demo

Abstract

Diffusion language models (DLMs) offer a compelling promise: parallel token generation could break the sequential bottleneck of autoregressive (AR) decoding. Yet in practice, DLMs consistently lag behind AR models in quality.

We argue that this gap stems from a fundamental failure of introspective consistency: AR models agree with what they generate, whereas DLMs often do not. We formalize this via the introspective acceptance rate, which quantifies whether a model internally accepts its previously generated tokens. Through this lens, we uncover a key structural advantage of AR models: causal masking combined with logit shifting implicitly enforces introspective consistency during training.

Motivated by this insight, we introduce the Introspective Diffusion Language Model (I-DLM), a new paradigm that preserves the introspective consistency of AR training while retaining diffusion-style parallelism. I-DLM uses a novel introspective strided decoding (ISD) algorithm, which enables the model to verify previously generated tokens while advancing new ones in the same forward pass. This yields a new quality–efficiency frontier unavailable to either AR or prior diffusion models.

Empirically, I-DLM-8B is the first DLM to match the quality of its same-scale AR counterpart while surpassing all prior DLMs in both quality and practical serving efficiency across 15 benchmarks. It attains 72.5 on AIME-24 and 45.1 on LiveCodeBench-v6, outperforming LLaDA-2.1-mini (16B) by more than +29 and +14 points respectively — despite using half the parameters. At high concurrency, I-DLM delivers 2.9–4.1x higher throughput than prior DLMs. With gated LoRA, ISD enables lossless (bit-for-bit identical) acceleration that outperforms speculative decoding baselines at scale.

Why Introspective Consistency?

We identify three fundamental bottlenecks in current DLMs:

(1) Low introspective consistency. DLMs cannot reliably agree with their own generations. SDAR achieves only 0.699 acceptance rate vs. I-DLM's 0.984.

(2) Compute inefficiency. Parallel decoding costs far more FLOPs per token than it saves. At TPF ~2.5, TiDAR needs ~7.8x overhead vs. I-DLM's ~2.5x.

(3) Infrastructure incompatibility. Multi-step denoising is poorly aligned with AR serving stacks. SDAR's throughput barely scales with TPF (slope=84 vs. I-DLM's 549).

The I-DLM Method

Comparison of decoding paradigms. I-DLM uses strict causal attention with adaptive stride and is a drop-in replacement within AR serving infrastructure. ISD produces a quality-guaranteed token together with draft tokens via introspective strided decoding.

Experimental Results

I-DLM is the first DLM to match the quality of its same-scale AR counterpart while surpassing all prior DLMs in both quality and practical serving efficiency across 15 benchmarks. I-DLM-8B outperforms LLaDA-2.1-mini (16B) by +29.2 on AIME-24 and +14.7 on LiveCodeBench-v6 despite using half the parameters.

Throughput–latency tradeoff across batch sizes. I-DLM consistently outperforms prior DLMs at moderate to high concurrency, delivering 2.9–4.1x higher throughput at C=64. It also outperforms speculative decoding methods (EAGLE-3, DFlash) starting from C≥16.

End-to-End Quality (Table 2)

Accuracy (%) on 15 benchmarks. I-DLM uses ISD (N=4, sampling). Blue = best non-AR under 30B. Bold = best non-AR under 100B.

Extended Comparison (Table 3)

Benchmarks commonly reported across diffusion LLM methods. "---" = not reported.

Citation

@article{yu2025introspective,
  title={Introspective Diffusion Language Models},
  author={Yu, Yifan and Jian, Yuqing and Wang, Junxiong and Zhou, Zhongzhu
          and Zhuang, Donglin and Fang, Xinyu and Yanamandra, Sri
          and Wu, Xiaoxia and Wu, Qingyang and Song, Shuaiwen Leon
          and Dao, Tri and Athiwaratkun, Ben and Zou, James
          and Lai, Fan and Xu, Chenfeng},
  journal={arXiv preprint},
  year={2025}
}