Illustrative image: a neural network diagram — D3D is a new way to make large language models generate text faster. This is not a screenshot of D3D. Image: Wikimedia Commons / Cburnett, CC BY-SA 3.0.
TL;DR — D3D speculative decoding explained in one line: researchers at Fudan University, Nanjing University and Huawei built a diffusion-based draft model that works independently of the model it assists, letting drafting and verification run at the same time — and report Qwen3 inference up to 7.48× faster, with no change to the answers.
📌 KEY HIGHLIGHTS
- The result: average decoding speedups of 5.00× with Qwen3-8B and 5.71× with Qwen3-32B across seven benchmarks, peaking at 7.48× on GSM8K maths problems.
- The trick: a standalone diffusion drafter that drafts the next block while the target model verifies the current one — the two stages no longer have to take turns.
- The engineering: blockwise denoising plus auxiliary autoregressive supervision, and layer pruning that removes 5 of 36 layers while keeping over 99% of the acceptance length.
- Open and free: the code is on GitHub under the MIT licence, with six draft-model checkpoints on Hugging Face — but the asynchronous mode needs two GPUs.
- The catch: this is a preprint with author-reported numbers; independent replication is still pending.
D3D speculative decoding explained: what happened?
On 4 November 2026, researchers Baoyou Chen, Ruiqiao Mei, Guancong Lin, Tao Song and Siyu Zhu of Fudan University, Weihao Yuan of Nanjing University and Qun Liu of Huawei posted a paper to arXiv titled "D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding" (arXiv:2611.06425). It tackles one of the most practical problems in running large language models: generation is slow because a model normally produces one token at a time, reading its full weights from memory for every token.
Speculative decoding is the standard workaround. A small, fast "draft" model guesses several tokens ahead, and the big target model checks them all in a single pass, keeping the ones it agrees with. D3D is the second speculative-decoding paper in a week on this blog: SpecPilot speculative decoding used an autonomous AI loop to tune the technique, while D3D redesigns the drafter itself.
The problem D3D attacks is a waiting game. Earlier diffusion-based drafters predict a whole block of tokens in parallel, but each new block has to wait for updated hidden states from the target model — so drafting and verification still run one after the other, and much of the speed advantage evaporates. D3D removes that dependency, and the paper claims the first working demonstration of asynchronous, lossless speculative decoding with a standalone diffusion drafter.
How D3D's diffusion drafter works
D3D combines two ideas. First, blockwise denoising: instead of predicting tokens one by one, the draft model fills in a whole block of 16 tokens at once, starting from noise and refining it — the diffusion part. Second, auxiliary autoregressive supervision: during training, the drafter is also taught to predict tokens in the usual left-to-right way, which keeps its guesses accurate. At a block size of 16, D3D averages 7.88 accepted tokens per block with Qwen3-8B as the target and 7.73 with Qwen3-32B.
Because the drafter no longer needs the target's hidden states, it can work ahead. While the target model verifies one block, D3D prepares continuations for several possible verification outcomes in a single packed pass on a second GPU — then the target's verdict picks the matching continuation for the next round. That is asynchronous speculative decoding, and it is why the method needs two GPUs to show its full benefit.
There is also a cost-cutting step. D3D progressively prunes draft-model layers according to how much each one affects acceptance length. Removing five of 36 layers retains over 99% of the mean acceptance length across three benchmarks with Qwen3-8B — a smaller, cheaper drafter that barely loses any accuracy.
Illustrative image: a GPU package with high-bandwidth memory — D3D's asynchronous mode runs the draft model on a second GPU while the target verifies. Image: Wikimedia Commons / C. Spille (pcgameshardware.de), CC BY-SA 4.0.
The numbers: 5× on average, 7.48× at best
The paper's headline figures come from single-request, greedy decoding across seven benchmarks — GSM8K, MATH-500, AIME25, HumanEval, MBPP, LCB-v6 and MT-Bench. D3D averages 5.00× decoding speedup with Qwen3-8B and 5.71× with Qwen3-32B, against 4.03× and 3.67× for DFlash, the previous diffusion-drafting method. The best single result is 7.48× on GSM8K with Qwen3-32B.
The open-source repository adds a second set of numbers: throughput on its SGLang hybrid backend, using the smaller D3D-1.7B drafter with Qwen3-8B at concurrency 1. Those figures are lower than the paper's headline averages, which is worth knowing before you quote the 5× number.
| Benchmark | Tokens/second | Speedup vs standard decoding |
|---|---|---|
| GSM8K | 658.10 | 4.876× |
| MATH-500 | 572.66 | 4.243× |
| HumanEval | 479.15 | 3.550× |
| MBPP | 428.39 | 3.174× |
| AIME25 | 394.56 | 2.924× |
| LCB-v6 | 267.42 | 1.982× |
| MT-Bench | 258.54 | 1.916× |
| Aggregate | 501.60 | 3.717× |
The gap between the two sets of numbers is not a contradiction: the paper's 5.00× is the native single-request implementation with the full 8B drafter, while the table above is the SGLang backend with the smaller 1.7B drafter. Maths and code tasks gain the most; open-ended chat (MT-Bench) gains the least, because the draft model's guesses are harder to get right there.
Why it matters for India readers
Qwen3 is one of the most widely used open model families among Indian developers, startups and students building chatbots, coding assistants and campus projects — so a free, open method that makes Qwen3 run several times faster is directly useful here. Faster inference means lower GPU bills for anyone serving a model, shorter waits for users, and the chance to run a capable 8B or 32B model on hardware that would otherwise feel sluggish. The code is MIT-licensed and the six draft checkpoints are freely downloadable, so there is no paywall between an Indian team and trying it.
The honest limit is the hardware. The full asynchronous benefit needs two GPUs — one for the target, one for the draft — which is routine for a serving cluster but a real cost for a small team or a single workstation. And like other speculative-decoding methods, the gains are biggest when one request is served at a time; they shrink as batch sizes grow. Research on the quantum side of optimisation is moving just as fast — see our Rydberg atom error correction explainer and the Aharonov-Bohm trapped ions story.
Should you care?
✅ Yes, if you serve or self-host Qwen3. D3D is open, free and measurably faster than the previous diffusion drafter on the paper's benchmarks, with code and checkpoints already released. ⏳ Wait for independent replication before betting a production system on the 7.48× headline — the numbers are the authors' own, and the SGLang figures show a more modest 3.7× in one real serving setup. If you only use hosted chatbots, this changes nothing you will notice directly; it changes what your provider pays to answer you.
The catches to keep in mind
First, this is a preprint, not a peer-reviewed paper, and all the headline numbers come from the authors' own experiments — no independent benchmark has confirmed them yet. Second, the 5.00× and 5.71× averages are single-request, greedy-decoding figures with thinking disabled; real deployments with sampling, long contexts or heavy batching will see different numbers. Third, the two-GPU requirement is not optional for the asynchronous mode — on one GPU, draft and verify share the same hardware and the advantage narrows. Finally, D3D does not make the model smarter: it only makes the same model answer faster.
FAQs
What is speculative decoding? A technique that speeds up LLM text generation without changing the output: a small draft model proposes several tokens at once, and the large target model verifies them in one pass, keeping only the tokens it agrees with. The output stays identical to normal generation because the target has the final say on every token.
What makes D3D different from other speculative decoding methods? Earlier diffusion drafters needed the target model's hidden states before they could draft the next block, forcing drafting and verification to run in turns. D3D is a standalone drafter — it drafts successor blocks during verification, so the two stages run in parallel on two GPUs.
How much faster is D3D really? The paper reports average speedups of 5.00× with Qwen3-8B and 5.71× with Qwen3-32B across seven benchmarks, peaking at 7.48× on GSM8K. The project's own SGLang backend shows a lower aggregate of 3.72× with the smaller 1.7B drafter — so expect real-world gains in the 2–5× range, depending on setup and task.
Does D3D change what the model answers? No. The paper describes the method as lossless speculative decoding: the target model verifies every draft token and discards anything it disagrees with, so the final text is the same as the target model would have produced on its own — it just arrives sooner.
TAKEAWAY: D3D removes the waiting between drafting and verification in speculative decoding, and the reported speedups — up to 7.48× on Qwen3 — make it one of the most practical inference papers of the month. The code and checkpoints are already public, so the next step is independent testing, not more promises. If you run Qwen3 yourself, the project's GitHub repository is the place to start.
Comments
Post a Comment