# ๐ Awesome Speculative Decoding
> Companion Method Profiles for a systems co-design survey of speculative decoding.
[](https://github.com/sindresorhus/awesome)
[](https://doi.org/10.1109/ACCESS.2026.3739598)
[](https://lifeissosolong.github.io/awesome-speculative-decoding/)
Our survey organizes speculative decoding from a **systems co-design perspective** across four connected aspects: **Draft Source**, **Draft Geometry**, **Verification Fidelity**, and **Runtime Execution**. This companion repository provides Method Profiles for all 312 included works, including category assignments, paper and code links, a complete chronological table, and an interactive explorer for filtering, comparing, sharing, and exporting results.
> **Help us keep this resource accurate and up to date.** If you spot an incorrect taxonomy assignment or know of a relevant new work that should be included, please [open an issue](https://github.com/LifeIsSoSolong/awesome-speculative-decoding/issues/new). When possible, include the paper link and a brief explanation of the suggested correction or addition.
## ๐ฐ News
- **2026‑10‑16** The [published version](https://doi.org/10.1109/ACCESS.2026.3739598) of our paper is now available.
- **2026‑09‑28** Updated to 312 Method Profiles, with revised classifications and links. The [19 September 2026 fixed snapshot](https://github.com/LifeIsSoSolong/awesome-speculative-decoding/releases/tag/method-profiles-2026-09-19) is available with evidence and reproducible counts.
- **2026‑08‑12** The [survey preprint](https://doi.org/10.20944/preprints202608.0746.v1) is now available.
- **2026‑07‑23** The [Interactive Method Profiles Explorer](https://lifeissosolong.github.io/awesome-speculative-decoding/) is now open.
- **2026‑07‑20** Initial release: 304 Method Profiles were organized across four systems-view aspects.
## ๐ Citation
If you find this survey or repository useful, please consider citing our survey:
[*From Draft-Then-Verify to Systems Co-Design: A Survey of Speculative Decoding for Autoregressive Large Language Models*](https://doi.org/10.1109/ACCESS.2026.3739598)
```bibtex
@article{zhao2026draftthenverify,
title = {From {Draft-Then-Verify} to {Systems Co-Design}: A Survey of Speculative Decoding for Autoregressive Large Language Models},
author = {Zhao, Kaikai and Liu, Zhaoxiang and Jiang, Che and Chen, Ping and Wang, Xin and Tian, Kai and Wang, Ning and Wang, Yuru and Shen, Yi and Yan, Jiangze and Hua, Minjie and Zhang, Wenjing and Wang, Xiang and Wang, Kai and Zhang, Kaiyan and Sun, Youbang and Ding, Ning and Lian, Shiguo and Zhou, Bowen},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {153987--154013},
doi = {10.1109/ACCESS.2026.3739598},
url = {https://doi.org/10.1109/ACCESS.2026.3739598}
}
```
## ๐ Interactive Method Profiles Explorer
[**Open Explorer โ**](https://lifeissosolong.github.io/awesome-speculative-decoding/)
Filter profiles by year, taxonomy category, contribution role, or resource availability. Compare methods, inspect the current landscape, share a filtered view, or export it as CSV. Try the one-click [2026 + Tree view](https://lifeissosolong.github.io/awesome-speculative-decoding/?year=2026&geometry=tree).
## ๐ฆ Fixed snapshot and downloads
- [**Download the 19 September 2026 snapshot (ZIP)**](https://github.com/LifeIsSoSolong/awesome-speculative-decoding/releases/download/method-profiles-2026-09-19/Supplementary_Material.zip) ยท [Release and checksum](https://github.com/LifeIsSoSolong/awesome-speculative-decoding/releases/tag/method-profiles-2026-09-19)
- [Current explorer data (CSV)](data/coding_matrix.csv)
The ZIP preserves the complete submitted supplementary package. `method_profiles_snapshot/` contains **312 profiles**, classification evidence, category definitions, the matching manuscript, and a standalone script for reproducing the counts. `supplement_source/` contains the readable supplement and its LaTeX source. The package records **1,459 assignments** (1,057 core and 402 inherited) and **83 unassigned workโaxis combinations**.
The original corpus was frozen on **20 July 2026**. The **19 September 2026** snapshot incorporates six restored works, two additions, and documented corrections; its date is not a new literature-search cutoff. The current explorer matches this snapshot. Future website updates will be recorded in Git; this versioned download will remain unchanged. For reproducibility, identify the snapshot date and checksum.
After extracting the ZIP, run `python3 reproduce_counts.py --check` from `method_profiles_snapshot/` (Python 3.10 or later). The script checks all seven generated result tables. Category-specific source locators are included where available; empty evidence fields do not imply a separately documented primary-source locator for every assignment.
**SHA-256 โ `Supplementary_Material.zip`:**
```text
7890d986b74de26e526bc56bdd52977e0d47669c4cbb405dbd7fa42d74cda15c
```
## ๐งญ Taxonomy
Figure 2 from the final manuscript, showing representative core assignments across 19 categories. Reference numbers and section labels follow the [published paper](https://doi.org/10.1109/ACCESS.2026.3739598). [View the full-size vector image](assets/taxonomy_tree_final.svg).
## ๐ Method Profiles
Updated **2026-09-28** ยท **312 profiles** ยท [Download CSV](data/coding_matrix.csv).
Each profile records documented supported configurations or direct analysis across the four aspects. Multiple labels within a work may describe different modes; their co-occurrence does not imply that every combination is supported by one mode. Paper links identify the source work. `[core]` marks a category introduced, substantively changed, or directly studied; `[inherited]` marks a category explicitly adopted as a substrate. The symbol `--` means no supported assignment on that axis; it is not a negative category label.
| Work | Draft
Source | Draft
Geometry | Verification
Fidelity | Runtime
Execution | Links |
| --- | --- | --- | --- | --- | --- |
| DSpark ยท 2026‑07
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation | Semi-Autoregressive
External Drafter [core] | Block [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Performance Analysis [core] | 
|
| AdaPLD ยท 2026‑06
AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] | |
| BudgetDraft ยท 2026‑06
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Speculation Control [core] | |
| CADDTree ยท 2026‑06
Cost-Aware Diffusion Draft Trees for Speculative Decoding | Non-Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| D2SD ยท 2026‑06
D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models | Non-Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [inherited] | |
| DFlare ยท 2026‑06
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding | Non-Autoregressive
External Drafter [core] | Block [core] | Token-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| Dynamic Exits ยท 2026‑06
Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| EfficientRollout ยท 2026‑06
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts | Target-Path
Execution [core] | Chain [inherited] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] |
|
| Hybrid Verified Decoding ยท 2026‑06
Hybrid Verified Decoding: Learning to Allocate Verification in Speculative Decoding | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [inherited] | Token-Level
Exactness [inherited] | Speculation Control [core] | |
| JetSpec ยท 2026‑06
JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting | Semi-Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [core] |
|
| SENSE ยท 2026‑06
SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decoding | Context-Derived
Nonparametric Source [core] | Collection [core] | Relaxed
Acceptance [core] | -- | |
| TAPS-Tree ยท 2026‑06
TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding | Non-Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [core] |
|
| TreeFlash ยท 2026‑06
TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding | Non-Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| VIA-SD ยท 2026‑06
VIA-SD: Verification via Intra-Model Routing for Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core] |
|
| WhiFlash ยท 2026‑06
WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing | Non-Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Block [inherited]
Tree [inherited] | Token-Level
Exactness [inherited] | Speculation Control [core] | |
| Acceptance-Guided SD ยท 2026‑05
Acceptance-Guided Adaptive Speculative Decoding for Efficient Large Language Model Inference | Proposal-State
Modeling [inherited] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [core] | 
|
| Attention Drift ยท 2026‑05
Attention Drift: What Autoregressive Speculative Decoding Models Learn | Proposal-State
Modeling [core] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Performance Analysis [core] |
|
| Bastion ยท 2026‑05
Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting | Non-Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] |
|
| Cassandra ยท 2026‑05
Cassandra: Enabling Reasoning LLMs at Edge via Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| CATS ยท 2026‑05
CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration | Target-Path
Execution [core] | Tree [core] | Token-Level
Exactness [inherited]
Relaxed
Acceptance [inherited] | Serving Systems [core] |
|
| Component-Aware SD ยท 2026‑05
Component-Aware Self-Speculative Decoding in Hybrid Language Models | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core]
Token-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] |
|
| CoSpec ยท 2026‑05
Beyond the Target: From Imitation to Collaboration in Speculative Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | -- | |
| Cross-Language SD ยท 2026‑05
Speculative Decoding Across Languages | Autoregressive
External Drafter [core]
Context-Derived
Nonparametric Source [core] | Chain [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] | |
| Domino ยท 2026‑05
Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding | Semi-Autoregressive
External Drafter [core] | Block [core] | Token-Level
Exactness [inherited] | Serving Systems [core] |
|
| EVICT ยท 2026‑05
Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding | Proposal-State
Modeling [inherited] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| EvoSpec ยท 2026‑05
EvoSpec: Evolving Speculative Decoding via Real-Time Vocabulary and Parameter Adaptation | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [core] | Tree [core]
Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| FlexDraft ยท 2026‑05
FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration | Target-Path
Execution [core] | Block [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core] | |
| Future Validity ยท 2026‑05
Future Validity is the Missing Statistic: From Impossibility to $ฮฆ$-Estimation for Grammar-Faithful Speculative Decoding | -- | Chain [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] | |
| Graft ยท 2026‑05
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding | Non-Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core] | |
| Latency Model ยท 2026‑05
An Interpretable Latency Model for Speculative Decoding in LLM Serving | Proposal-State
Modeling [inherited] | Chain [inherited] | -- | Performance Analysis [core] | |
| Mistletoe ยท 2026‑05
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding | Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited] | Tree [core]
Collection [core] | Relaxed
Acceptance [core] | Performance Analysis [core] | |
| NanoSpec ยท 2026‑05
NanoSpec: Accelerating Speculative Decoding using Minimalist In-Context Vocabularies | Proposal-State
Modeling [inherited] | Tree [inherited] | Distribution-Level
Exactness [inherited] | Performance Analysis [core] | |
| PARD-2 ยท 2026‑05
PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding | Non-Autoregressive
External Drafter [core] | Block [core] | Token-Level
Exactness [core] | -- |
|
| PARSE ยท 2026‑05
Parallel Prefix Verification for Speculative Generation | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Chain [core]
Collection [core] | Relaxed
Acceptance [core] | State Management [core]
Serving Systems [core] | |
| PELM ยท 2026‑05
PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling | Target-Path
Execution [core] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] | |
| PipeSD ยท 2026‑05
PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Serving Systems [core] |
|
| PPOW ยท 2026‑05
Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing | Proposal-State
Modeling [inherited] | Tree [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [core] | |
| RAC ยท 2026‑05
RAC: Request-adaptive Configuration for Efficient Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core]
Performance Analysis [core] |
|
| RTP-LLM ยท 2026‑05
RTP-LLM: High-Performance Alibaba LLM Inference Engine | Autoregressive
External Drafter [inherited]
Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [inherited]
Tree [inherited] | Distribution-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] |
|
| SlimSpec ยท 2026‑05
SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding | Proposal-State
Modeling [inherited] | Tree [inherited] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Performance Analysis [core] | |
| SNLP (SSD mode) ยท 2026‑05
SNLP: Layer-Parallel Inference via Structured Newton Corrections | Target-Path
Execution [core] | Chain [inherited] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Performance Analysis [core] |
|
| SPD ยท 2026‑05
Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism | Proposal-State
Modeling [core] | Chain [inherited] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Serving Systems [core] | |
| SpecBlock ยท 2026‑05
SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting | Semi-Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [core] | |
| SpecKV ยท 2026‑05
SpecKV: Adaptive Speculative Decoding with Compression-Aware Gamma Selection | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] |
|
| SPECTRE ยท 2026‑05
SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] |
|
| UniVer ยท 2026‑05
UniVer: A Unified Perspective for Multi-step and Multi-draft Speculative Decoding | Proposal-State
Modeling [inherited] | Tree [core]
Collection [core] | Distribution-Level
Exactness [core] | -- | |
| A-IO (SD/PLD study) ยท 2026‑04
A-IO: Adaptive Inference Orchestration for Memory-Bound NPUs | Context-Derived
Nonparametric Source [inherited]
Autoregressive
External Drafter [inherited] | Chain [inherited] | -- | Speculation Control [core]
Performance Analysis [core] | |
| Acceptance ยท 2026‑04
Acceptance Dynamics Across Cognitive Domains in Speculative Decoding | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] |
|
| Adaptive Hybrid SD ยท 2026‑04
Adaptive hybrid speculative decoding for accelerating large language model inference | Autoregressive
External Drafter [core] | Chain [core]
Block [core] | Relaxed
Acceptance [core] | State Management [core]
Speculation Control [core] | |
| AdaServe ยท 2026‑04
AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding | Autoregressive
External Drafter [inherited] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] |
|
| AHASD ยท 2026‑04
AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices | Autoregressive
External Drafter [inherited] | Chain [inherited] | -- | Serving Systems [core] | |
| Area- ยท 2026‑04
Area- and Utilization-Efficient LLM Accelerator With Fused Speculative Decoding for Edge-Side Inference | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| Cactus ยท 2026‑04
Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling | Autoregressive
External Drafter [inherited] | Chain [inherited] | Bounded
Approximation [core] | Speculation Control [inherited] |
|
| Calibrated ยท 2026‑04
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference | Autoregressive
External Drafter [core]
Context-Derived
Nonparametric Source [core] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| Commodity-GPU SD ยท 2026‑04
Memory-Efficient Speculative Decoding for Large Language Model Inference on Commodity GPUs | Autoregressive
External Drafter [core] | Chain [inherited] | Token-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| ConfLayers ยท 2026‑04
ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] |
|
| Cross-Family ยท 2026‑04
Cross-Family Speculative Decoding for Polish Language Models on Apple~Silicon: An Empirical Evaluation of Bielik~11B with UAG-Extended MLX-LM | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] |
|
| DDTree ยท 2026‑04
Accelerating Speculative Decoding with Block Diffusion Draft Trees | Non-Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| DIVERSED ยท 2026‑04
DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification | Autoregressive
External Drafter [core] | Chain [core] | Relaxed
Acceptance [core] | -- |
|
| Dynamic Multi-Path SD ยท 2026‑04
Optimizing speculative decoding via dynamic multi-path and dual-stream networks | Autoregressive
External Drafter [core] | Collection [core] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| ECHO ยท 2026‑04
ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [core] | |
| ELMoE-3D ยท 2026‑04
ELMoE-3D: Leveraging Intrinsic Elasticity of MoE for Hybrid-Bonding-Enabled Self-Speculative Decoding in On-Premises Serving | Target-Path
Execution [core] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Serving Systems [core] | |
| FPGA Tree SD ยท 2026‑04
Boosting the Performance of Tree-Based Speculative Decoding of LLMs on FPGAs | Proposal-State
Modeling [inherited] | Tree [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| Multi-LoRA Edge SD ยท 2026‑04
Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM | Target-Path
Execution [core] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [core] | |
| On-Device MoE SD ยท 2026‑04
Self-Speculative Decoding for On-device MoE Acceleration | Target-Path
Execution [core] | Chain [core] | Relaxed
Acceptance [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] | |
| RACER ยท 2026‑04
RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [inherited] |
|
| RL Rollout SD ยท 2026‑04
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding | Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited] | Chain [inherited]
Tree [inherited] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| SMART ยท 2026‑04
SMART: When is it Actually Worth Expanding a Speculative Tree? | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| SpecGuard ยท 2026‑04
From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning | Autoregressive
External Drafter [inherited] | Collection [core] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| SpecTr-GBV ยท 2026‑04
SpecTr-GBV: Multi-Draft Block Verification Accelerating Speculative Decoding | Autoregressive
External Drafter [inherited] | Block [core]
Collection [core] | Distribution-Level
Exactness [core] | Speculation Control [inherited] | |
| SPEED-Bench ยท 2026‑04
SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding | Autoregressive
External Drafter [inherited]
Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [inherited]
Tree [inherited] | -- | Performance Analysis [core] | |
| When Hidden States Drift ยท 2026‑04
When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding? | Proposal-State
Modeling [core] | Tree [inherited] | Distribution-Level
Exactness [inherited] | State Management [core]
Performance Analysis [core] | |
| AdaSpec-Multilingual ยท 2026‑03
AdaSpec: Adaptive Multilingual Speculative Decoding with Self-Synthesized Language-Aware Training and Vocabulary Simplification | Autoregressive
External Drafter [core]
Context-Derived
Nonparametric Source [core] | Chain [core] | Token-Level
Exactness [core] | -- | |
| ConFu ยท 2026‑03
ConFu: Contemplate the Future for Better Speculative Sampling | Proposal-State
Modeling [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Serving Systems [inherited] | |
| DFVG ยท 2026‑03
DFVG: A Heterogeneous Architecture for Speculative Decoding with Draft-on-FPGA and Verify-on-GPU | Autoregressive
External Drafter [inherited] | Tree [core] | -- | Speculation Control [core]
Serving Systems [core] |
|
| Distributed Adaptive SD ยท 2026‑03
Distributed Adaptive Speculative Decoding: Accelerating Large Language Model Inference With Context-Aware Draft Selection | Autoregressive
External Drafter [core] | Chain [inherited] | Token-Level
Exactness [inherited] | State Management [core]
Speculation Control [core]
Serving Systems [core] | |
| EAGLE-Pangu ยท 2026‑03
EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs | Proposal-State
Modeling [inherited] | Tree [inherited] | Token-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] | |
| Empirical Study ยท 2026‑03
An Empirical Study of Speculative Decoding for Small Language Models | Autoregressive
External Drafter [inherited]
Target-Path
Execution [inherited]
Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [inherited]
Tree [inherited]
Collection [inherited] | -- | Performance Analysis [core] | |
| Learning to Draft ยท 2026‑03
Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning | Proposal-State
Modeling [inherited] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [core] |
|
| Lookahead Gate ยท 2026‑03
Look Before You Leap: A Lookahead Reasoning Quality Gate for Speculative Decoding | Target-Path
Execution [inherited] | Collection [core] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| MineDraft ยท 2026‑03
MineDraft: A Framework for Batch Parallel Speculative Decoding | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | State Management [core]
Speculation Control [core]
Serving Systems [core] | |
| Quasar ยท 2026‑03
Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification | Context-Derived
Nonparametric Source [inherited] | Chain [core] | Relaxed
Acceptance [core] | Serving Systems [core]
Performance Analysis [core] |
|
| Re-SpS ยท 2026‑03
Re-SpS: A Reinforcement Learning Approach to Speculative Sampling | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | -- |
|
| SpecForge ยท 2026‑03
SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | -- |
|
| Speculative-Speculative ยท 2026‑03
Speculative Speculative Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | State Management [core]
Speculation Control [core]
Serving Systems [core] |
|
| TAPS-Distribution ยท 2026‑03
TAPS: Task Aware Proposal Distributions for Speculative Sampling | Proposal-State
Modeling [inherited] | Collection [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [inherited] |
|
| VocabTrim ยท 2026‑03
Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding | Proposal-State
Modeling [inherited] | Tree [inherited] | Distribution-Level
Exactness [inherited] | Performance Analysis [core] |
|
| Compiler-Assisted SD ยท 2026‑02
Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] | |
| DFlash ยท 2026‑02
DFlash: Block Diffusion for Flash Speculative Decoding | Non-Autoregressive
External Drafter [core] | Block [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| Energy Benchmarking ยท 2026‑02
Benchmarking the Energy Savings with Speculative Decoding Strategies | Autoregressive
External Drafter [inherited]
Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited] | Chain [inherited]
Tree [inherited] | -- | Performance Analysis [core] | |
| GBV ยท 2026‑02
Greedy Multi-Path Block Verification for Faster Decoding in Speculative Sampling | Autoregressive
External Drafter [inherited] | Block [core]
Collection [core] | Distribution-Level
Exactness [core] | Speculation Control [inherited] | |
| KnapSpec ยท 2026‑02
KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] |
|
| P-EAGLE ยท 2026‑02
P-EAGLE: Parallel-Drafting EAGLE with Scalable Training | Proposal-State
Modeling [core] | Block [core] | Distribution-Level
Exactness [inherited] | Serving Systems [core] | |
| PRISM ยท 2026‑02
PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models | Proposal-State
Modeling [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| SDFP ยท 2026‑02
SDFP: Speculative Decoding with FIT-Pruned Models for Training-Free and Plug-and-Play LLM Acceleration | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | -- | |
| SemanticSpec ยท 2026‑02
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States | Autoregressive
External Drafter [inherited] | Chain [core] | Relaxed
Acceptance [core] | -- | |
| SpecVocab ยท 2026‑02
Speculative Decoding with a Speculative Vocabulary | Proposal-State
Modeling [inherited] | Tree [inherited] | Distribution-Level
Exactness [inherited] | Performance Analysis [core] | |
| Split Lookahead ยท 2026‑02
Privacy-Aware Split Inference with Speculative Decoding for Large Language Models over Wide-Area Networks | Context-Derived
Nonparametric Source [core] | Chain [core] | Token-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] | |
| TransKV ยท 2026‑02
Transactional KV Caching for Speculative Decoding under Paged KV Memory | -- | Chain [inherited] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] | |
| Vegas ยท 2026‑02
Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] |
|
| VSD ยท 2026‑02
Variational Speculative Decoding: Rethinking Draft Training from Token Likelihood to Sequence Acceptance | Proposal-State
Modeling [inherited] | Tree [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [inherited] | |
| Watermark Trade-off ยท 2026‑02
Improving the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [core] | Performance Analysis [core] |
|
| DART ยท 2026‑01
DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference | Non-Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [inherited] | Speculation Control [inherited] |
|
| FlexSpec ยท 2026‑01
FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Token-Level
Exactness [inherited] | Serving Systems [core] | |
| HSD ยท 2026‑01
Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Block [core]
Collection [inherited] | Distribution-Level
Exactness [core] | Speculation Control [inherited] |
|
| AdaSD ยท 2025‑12
AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference | Autoregressive
External Drafter [core] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| Aggressive ASD ยท 2025‑12
Aggressive Speculative Decoding for Efficient Chain-of-Thought Reasoning | Autoregressive
External Drafter [inherited] | Chain [inherited] | Bounded
Approximation [core]
Relaxed
Acceptance [core] | Speculation Control [core] | |
| ASDT ยท 2025‑12
ASDT: Adaptive Speculative Decoding Tree Based on Token Acceptance Feedback | Autoregressive
External Drafter [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| Device-Server SD ยท 2025‑12
Device-Server Collaborative Speculative Decoding for Real-Time LLM Streaming | Autoregressive
External Drafter [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | Serving Systems [core] | |
| FailFast ยท 2025‑12
Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs | Non-Autoregressive
External Drafter [inherited] | Chain [inherited] | Token-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] |
|
| Nightjar ยท 2025‑12
Nightjar: Dynamic adaptive speculative decoding for large language models serving | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| Parallel Token Prediction ยท 2025‑12
Parallel Token Prediction for Language Models | Non-Autoregressive
External Drafter [core] | Block [core] | Token-Level
Exactness [core] | -- |
|
| Performance or Illusion ยท 2025‑12
Speculative Decoding: Performance or Illusion? | Autoregressive
External Drafter [inherited]
Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [inherited]
Tree [inherited] | -- | Performance Analysis [core] |
|
| RADAR ยท 2025‑12
RADAR: Accelerating Large Language Model Inference With RL-Based Dynamic Draft Trees | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] |
|
| Sparse Self-SD ยท 2025‑12
Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] |
|
| SpecPV ยท 2025‑12
SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification | Proposal-State
Modeling [inherited] | Tree [inherited] | Relaxed
Acceptance [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] |
|
| Speed-of-Light ยท 2025‑12
Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks | -- | Tree [core] | Token-Level
Exactness [inherited] | Performance Analysis [core] | |
| T-pro 2.0 (EAGLE) ยท 2025‑12
T-pro 2.0: An Efficient Russian Hybrid-Reasoning Model and Playground | Proposal-State
Modeling [inherited] | Tree [inherited] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [core]
Performance Analysis [core] |
|
| Cacheback ยท 2025‑11
Cacheback: Speculative Decoding With Nothing But Cache | Context-Derived
Nonparametric Source [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Serving Systems [core] |
|
| Collective Adaptive SD ยท 2025‑11
Towards Efficient LLM Inference via Collective and Adaptive Speculative Decoding | Autoregressive
External Drafter [inherited] | Collection [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| FLy ยท 2025‑11
Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match | Autoregressive
External Drafter [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | -- | |
| FractalLLM ยท 2025‑11
FractalLLM: Lossless Self-Speculative Decoding with Layer Embedded Self-Compression | Proposal-State
Modeling [core] | Chain [core] | Token-Level
Exactness [core] | -- |
|
| Global Resolution ยท 2025‑11
Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization | Autoregressive
External Drafter [inherited] | Tree [inherited]
Collection [core] | Bounded
Approximation [core] | -- | |
| N-Gram Trie SD ยท 2025‑11
Faster In-Context Learning for LLMs via N-Gram Trie Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Token-Level
Exactness [core] | -- |
|
| Next-Latent ยท 2025‑11
Next-Latent Prediction Transformers Learn Compact World Models | Proposal-State
Modeling [core] | Chain [core] | Token-Level
Exactness [core] | -- |
|
| SDSquare ยท 2025‑11
Steering Pretrained Drafters During Speculative Decoding | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [inherited] |
|
| SpecDiff-2 ยท 2025‑11
SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding | Non-Autoregressive
External Drafter [core] | Block [core]
Collection [core] | Token-Level
Exactness [inherited] | Speculation Control [core] | |
| SpecFormer ยท 2025‑11
Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios | Proposal-State
Modeling [core] | Block [core] | Token-Level
Exactness [inherited] | Performance Analysis [core] |
|
| WiP ยท 2025‑11
WiP: Efficient Speculative Decoding for AI PCs via Hierarchical N-Gram Retrieval | Context-Derived
Nonparametric Source [core] | Tree [core] | Token-Level
Exactness [core] | Serving Systems [core] | |
| 3-Model ยท 2025‑10
3-Model Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core]
Performance Analysis [core] | |
| AdaSPEC ยท 2025‑10
AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | -- |
|
| BatchSpec ยท 2025‑10
Batch Speculative Decoding Done Right | Autoregressive
External Drafter [inherited] | Chain [inherited] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core]
Performance Analysis [core] |
|
| CAS-Spec ยท 2025‑10
CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs | Target-Path
Execution [core]
Context-Derived
Nonparametric Source [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| Comm-Efficient SD ยท 2025‑10
Communication-Efficient Speculative Decoding for Large Language Models Inference | Autoregressive
External Drafter [core] | Chain [inherited] | Relaxed
Acceptance [core] | Speculation Control [core]
Serving Systems [core] | |
| DiffuSpec ยท 2025‑10
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding | Non-Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| Disparate Impacts ยท 2025‑10
The Disparate Impacts of Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] | |
| Draft-Verify-Improve ยท 2025‑10
Draft, Verify, and Improve: Toward Training-Aware Speculative Decoding | Target-Path
Execution [core]
Target-Feature
Readout [core] | Chain [core] | Token-Level
Exactness [core] | -- | |
| DynaSpec ยท 2025‑10
DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models | Proposal-State
Modeling [inherited] | Tree [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] | |
| HiSpec ยท 2025‑10
HiSpec: Hierarchical Speculative Decoding for LLMs | Target-Path
Execution [core]
Target-Feature
Readout [inherited] | Chain [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | State Management [core]
Serving Systems [core]
Performance Analysis [core] | |
| Mirror SD ยท 2025‑10
Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference | Autoregressive
External Drafter [core]
Target-Path
Execution [core] | Tree [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| Polybasic ยท 2025‑10
Polybasic Speculative Decoding Through a Theoretical Perspective | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Performance Analysis [core] | |
| ReSpec ยท 2025‑10
ReSpec: Towards Optimizing Speculative Decoding in Reinforcement Learning Systems | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Performance Analysis [core] | |
| TokenTiming ยท 2025‑10
TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | -- | |
| Collaborative SD ยท 2025‑09
Communication-Efficient Collaborative LLM Inference via Distributed Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] | |
| DSDE ยท 2025‑09
DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving | Autoregressive
External Drafter [inherited] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| FastMTP ยท 2025‑09
FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction | Proposal-State
Modeling [core] | Chain [inherited] | Token-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] |
|
| Pipeline Early-Exit ยท 2025‑09
Pipeline Parallelism is All You Need for Optimized Early-Exit Based Self-Speculative Decoding | Target-Path
Execution [core]
Target-Feature
Readout [core] | Chain [inherited] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| SN40L SD ยท 2025‑09
Speculative Decoding on the SN40L Reconfigurable Dataflow Unit | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Serving Systems [core] | |
| SPEC-RL ยท 2025‑09
SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts | Context-Derived
Nonparametric Source [core] | Chain [inherited] | Relaxed
Acceptance [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] | |
| SpecMamba ยท 2025‑09
SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | Serving Systems [core] | |
| SSBD ยท 2025‑09
Self-Speculative Biased Decoding for Faster Re-Translation | Context-Derived
Nonparametric Source [core] | Chain [core] | Token-Level
Exactness [inherited]
Relaxed
Acceptance [core] | State Management [core] | |
| CARD ยท 2025‑08
CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference | Autoregressive
External Drafter [core]
Context-Derived
Nonparametric Source [core] | Chain [inherited] | Distribution-Level
Exactness [core] | -- |
|
| Confidence-Modulated ยท 2025‑08
Confidence-Modulated Speculative Decoding for Large Language Models | Non-Autoregressive
External Drafter [inherited] | Block [inherited] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| KVPSN (text-to-text mode) ยท 2025‑08
Novel Parasitic Dual-Scale Modeling for Efficient and Accurate Multilingual Speech Translation | Autoregressive
External Drafter [core] | Chain [inherited] | Token-Level
Exactness [inherited]
Relaxed
Acceptance [core] | State Management [core] | |
| ResDecode ยท 2025‑08
ResDecode: Accelerating Large Language Models Inference via Residual Decoding Heads | Proposal-State
Modeling [core] | Tree [core] | Distribution-Level
Exactness [core] | -- |
|
| SSD (Safety-Aware) ยท 2025‑08
Speculative Safety-Aware Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | Speculation Control [core] |
|
| SSS ยท 2025‑08
Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner | Autoregressive
External Drafter [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | -- |
|
| SuperSpec ยท 2025‑08
SuperSpec: Enhanced Verification and Sampling for End-to-End LLM Speculative Decoding | Autoregressive
External Drafter [inherited] | Collection [core]
Tree [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] | |
| Disaggregated ยท 2025‑07
Disaggregated Speculative Decoding for Carbon-Efficient LLM Serving | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| DSSD ยท 2025‑07
DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decoding | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] |
|
| EDD/PCT ยท 2025‑07
Faster Speculative Decoding via Effective Draft Decoder with Pruned Candidate Tree | Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [core] | Speculation Control [core] | |
| HVT ยท 2025‑07
Hierarchical Verification of Speculative Beams for Accelerating LLM Inference | Autoregressive
External Drafter [inherited] | Tree [core] | -- | Speculation Control [core] | |
| 3D-TokSIM ยท 2025‑06
3D-TokSIM: Stacking 3D Memory with Token-Stationary Compute-in-Memory for Speculative LLM Inference | -- | Chain [inherited] | Token-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| Dynamic-K SD ยท 2025‑06
Dynamic K Mechanism for Accelerating Speculative Decoding in vLLM | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| GSI ยท 2025‑06
Guided Speculative Inference for Efficient Test-Time Alignment of LLMs | Autoregressive
External Drafter [inherited] | Collection [core] | Bounded
Approximation [core]
Relaxed
Acceptance [core] | Speculation Control [core] |
|
| List-Level ยท 2025‑06
List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression | Autoregressive
External Drafter [inherited] | Collection [core] | Distribution-Level
Exactness [core] | -- | |
| Lookahead Reasoning ยท 2025‑06
Scaling Speculative Decoding with Lookahead Reasoning | Autoregressive
External Drafter [inherited] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core]
Performance Analysis [core] |
|
| Mamba ยท 2025‑06
Mamba Drafters for Speculative Decoding | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| Model-Free Test-Time Scaling ยท 2025‑06
Accelerated Test-Time Scaling with Model-Free Speculative Sampling | Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] | |
| POSS ยท 2025‑06
POSS: Position Specialist Generates Better Draft for Speculative Decoding | Proposal-State
Modeling [core] | Tree [core] | Distribution-Level
Exactness [core] | -- |
|
| S4C ยท 2025‑06
S4C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models | Proposal-State
Modeling [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| SpecBranch ยท 2025‑06
SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism | Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| SpecEE ยท 2025‑06
SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting | Proposal-State
Modeling [inherited] | Tree [inherited] | Relaxed
Acceptance [core] | Speculation Control [core]
Serving Systems [core] |
|
| SpecMemo ยท 2025‑06
SpecMemo: Speculative Decoding is in Your Pocket | Target-Feature
Readout [inherited] | Tree [core] | Token-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] | |
| SwiftSpec ยท 2025‑06
SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding | Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] | |
| Utility-Driven ยท 2025‑06
Utility-Driven Speculative Decoding for Mixture-of-Experts | Context-Derived
Nonparametric Source [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Performance Analysis [core] | |
| VOCABTRIM ยท 2025‑06
VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Chain [core]
Tree [core] | Distribution-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] | |
| Alignment-Augmented ยท 2025‑05
Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification | Context-Derived
Nonparametric Source [core] | Chain [core] | Relaxed
Acceptance [core] | -- | |
| Automatic ยท 2025‑05
Automatic Task Detection and Heterogeneous LLM Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| BanditSpec ยท 2025‑05
BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [inherited] | Chain [core]
Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] |
|
| CLaSp ยท 2025‑05
CLaSp: In-Context Layer Skip for Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core] | |
| Ghidorah ยท 2025‑05
Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism | Target-Feature
Readout [inherited] | Tree [core] | Token-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| KNN-SSD ยท 2025‑05
KNN-SSD: Enabling Dynamic Self-Speculative Decoding via Nearest Neighbor Layer Set Optimization | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] |
|
| LAPS-SD ยท 2025‑05
Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| MTP Curriculum ยท 2025‑05
Pre-Training Curriculum for Multi-Token Prediction in Language Models | Target-Feature
Readout [core] | Block [inherited] | Token-Level
Exactness [inherited] | -- |
|
| PipeSpec ยท 2025‑05
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Token-Level
Exactness [inherited] | Serving Systems [core] |
|
| Quantization Compat. ยท 2025‑05
Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design | Autoregressive
External Drafter [core]
Proposal-State
Modeling [inherited] | Tree [core]
Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] | |
| SpecRouter ยท 2025‑05
SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core] | |
| Traversal ยท 2025‑05
Traversal Verification for Speculative Tree Decoding | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| ASSD ยท 2025‑04
Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding | Target-Path
Execution [core] | Block [core] | Distribution-Level
Exactness [core] | Speculation Control [inherited] |
|
| DEL ยท 2025‑04
DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] |
|
| EdgeLLM ยท 2025‑04
EdgeLLM: Fast On-Device LLM Inference With Speculative Decoding | Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| Exponential Races ยท 2025‑04
Speculative Sampling via Exponential Races | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| Hierarchical ยท 2025‑04
Hierarchical Speculative Decoding with Dynamic Window | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| PARD ยท 2025‑04
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation | Non-Autoregressive
External Drafter [core] | Block [core] | Distribution-Level
Exactness [core] | -- |
|
| SD$^2$ ยท 2025‑04
SD$^2$: Self-Distilled Sparse Drafters | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | -- | |
| SpecReason ยท 2025‑04
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning | Autoregressive
External Drafter [inherited] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core]
Performance Analysis [core] |
|
| SPIRe ยท 2025‑04
SPIRe: Boosting LLM Inference Throughput with Speculative Decoding | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [core] | State Management [core]
Performance Analysis [core] | |
| AdaSpec-SLO ยท 2025‑03
AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Serving | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [core] |
|
| Domain Draft Models ยท 2025‑03
Training Domain Draft Models for Speculative Decoding: Best Practices and Insights | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | -- | |
| DuoDecoding ยท 2025‑03
DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting | Autoregressive
External Drafter [core] | Collection [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] |
|
| Dynamic Adjustment ยท 2025‑03
Improving Speculative Decoding with Dynamic Adjustment and Probability Smoothing | Autoregressive
External Drafter [core] | Collection [core] | Distribution-Level
Exactness [core] | -- | |
| EAGLE-3 ยท 2025‑03
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test | Proposal-State
Modeling [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| ML-SpecQD ยท 2025‑03
ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts | Autoregressive
External Drafter [core] | Chain [inherited] | Token-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| Multi-Sample SD ยท 2025‑03
Speculative Decoding for Multi-Sample Inference | Context-Derived
Nonparametric Source [core] | Collection [core] | Distribution-Level
Exactness [core] | -- | |
| RASD ยท 2025‑03
RASD: Retrieval-Augmented Speculative Decoding | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| vLLM SpecInfer ยท 2025‑03
Speculative Inference with vLLM: Optimizing Heterogeneous Computing in Training-Inference Integrated Environments | -- | Chain [inherited] | -- | Serving Systems [core] | |
| CopySpec ยท 2025‑02
CopySpec: Accelerating LLMs with Speculative Copy-and-Paste | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [core] | Chain [core] | Token-Level
Exactness [core] | -- |
|
| DReSD ยท 2025‑02
DReSD: Dense Retrieval for Speculative Decoding | Context-Derived
Nonparametric Source [core] | Chain [core]
Collection [core] | Token-Level
Exactness [core] | -- |
|
| Dynamic Tree Attention ยท 2025‑02
Acceleration Multiple Heads Decoding for LLM via Dynamic Tree Attention | Target-Feature
Readout [inherited] | Tree [core] | Token-Level
Exactness [core] | Serving Systems [core] |
|
| FR-Spec ยท 2025‑02
FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling | Target-Feature
Readout [inherited]
Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] |
|
| Fuzzy ยท 2025‑02
Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff | Autoregressive
External Drafter [core]
Proposal-State
Modeling [inherited] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core]
Performance Analysis [core] |
|
| GRIFFIN ยท 2025‑02
GRIFFIN: Effective Token Alignment for Faster Speculative Decoding | Proposal-State
Modeling [core] | Tree [core] | Token-Level
Exactness [core] | -- |
|
| Heterogeneous Vocab SD ยท 2025‑02
Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] |
|
| Hierarchical Drafting ยท 2025‑02
Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding | Context-Derived
Nonparametric Source [core] | Chain [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] |
|
| Optimal Multi-Draft ยท 2025‑02
Towards Optimal Multi-draft Speculative Decoding | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Collection [core] | Distribution-Level
Exactness [core] | -- | |
| QuantSpec ยท 2025‑02
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| TETRIS ยท 2025‑02
TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding | Autoregressive
External Drafter [inherited]
Target-Feature
Readout [inherited] | Collection [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| Drop-In Adaptation ยท 2025‑01
A Drop-In Solution for On-the-Fly Adaptation of Speculative Decoding in Large Language Models | Autoregressive
External Drafter [core]
Proposal-State
Modeling [inherited] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| Judge Decoding ยท 2025‑01
Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment | Autoregressive
External Drafter [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | Performance Analysis [core] | |
| RSD ยท 2025‑01
Reward-Guided Speculative Decoding for Efficient LLM Reasoning | Autoregressive
External Drafter [inherited] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core] |
|
| SpecStreaming-MSA ยท 2025‑01
Speculative Streaming: Efficient and Scalable Speculative Decoding with Multi-Stream Attention | Proposal-State
Modeling [core] | Collection [core] | Distribution-Level
Exactness [core]
Relaxed
Acceptance [core] | -- | |
| CTC Drafter ยท 2024‑12
Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration | Target-Feature
Readout [core] | Block [core]
Tree [core] | Token-Level
Exactness [core] | -- | |
| Dovetail ยท 2024‑12
Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference | Proposal-State
Modeling [core] | Chain [core] | Distribution-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] |
|
| Falcon ยท 2024‑12
Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree | Proposal-State
Modeling [core] | Block [core]
Tree [core] | Token-Level
Exactness [core] | -- | |
| PLD+ ยท 2024‑12
PLD+: Accelerating LLM inference by leveraging Language Model Artifacts | Context-Derived
Nonparametric Source [core] | Chain [core] | Token-Level
Exactness [core] | -- | |
| FastDraft ยท 2024‑11
FastDraft: How to Train Your Draft | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | -- | |
| SAM Decoding ยท 2024‑11
SAM Decoding: Speculative Decoding via Suffix Automaton | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| Self-Verification SD ยท 2024‑11
Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation | Autoregressive
External Drafter [core]
Proposal-State
Modeling [inherited] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| SpecHub ยท 2024‑11
SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Collection [core] | Distribution-Level
Exactness [core] | -- |
|
| Speculative Side Channels ยท 2024‑11
When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMs | Proposal-State
Modeling [inherited] | Chain [inherited]
Tree [inherited] | -- | Serving Systems [core]
Performance Analysis [core] | |
| SSSD ยท 2024‑11
SSSD: Simply-Scalable Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core]
Performance Analysis [core] |
|
| SuffixDecoding ยท 2024‑11
SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications | Proposal-State
Modeling [inherited]
Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] |
|
| The N-Grammys ยท 2024‑11
The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation | Context-Derived
Nonparametric Source [core] | Collection [core] | Token-Level
Exactness [inherited] | Serving Systems [core]
Performance Analysis [core] | |
| Theoretical ยท 2024‑11
A Theoretical Perspective for Speculative Decoding Algorithm | Autoregressive
External Drafter [inherited] | Chain [inherited] | Bounded
Approximation [core] | Performance Analysis [core] | |
| AMUSD ยท 2024‑10
AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| Cerberus ยท 2024‑10
Cerberus: Efficient Inference with Adaptive Parallel Decoding and Sequential Knowledge Enhancement | Proposal-State
Modeling [core] | Tree [core] | Token-Level
Exactness [core] | Speculation Control [core] | |
| Draft on the Fly ยท 2024‑10
Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core] | |
| DySpec ยท 2024‑10
DySpec: Faster speculative decoding with dynamic token tree structure | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| Inevitable ยท 2024‑10
Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [core] | Performance Analysis [core] | |
| Multi-Draft ยท 2024‑10
Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits | Autoregressive
External Drafter [inherited] | Collection [core] | Distribution-Level
Exactness [core] | -- | |
| ParallelSpec ยท 2024‑10
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding | Non-Autoregressive
External Drafter [core] | Block [core]
Tree [core] | Distribution-Level
Exactness [core] | -- | |
| QSpec ยท 2024‑10
QSpec: Speculative Decoding with Complementary Quantization Schemes | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | Serving Systems [core]
Performance Analysis [core] | |
| SWIFT ยท 2024‑10
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration | Target-Path
Execution [core] | Chain [core]
Tree [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core] |
|
| Temperature-Centric ยท 2024‑10
Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [core] | Performance Analysis [core] |
|
| Tensor-MTP ยท 2024‑10
Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition | Target-Feature
Readout [core] | Block [inherited]
Tree [inherited] | Token-Level
Exactness [inherited] | Speculation Control [inherited] | |
| Dynamic Depth Decoding ยท 2024‑09
Dynamic Depth Decoding: Faster Speculative Decoding for LLMs | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| Multi-Candidate SD ยท 2024‑09
Improving Multi-candidate Speculative Decoding | Autoregressive
External Drafter [core]
Target-Path
Execution [core] | Tree [core]
Collection [core] | Relaxed
Acceptance [core] | Speculation Control [core] | |
| Context-Aware ยท 2024‑08
Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Models | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| CREST ยท 2024‑08
CREST: Effectively Compacting a Datastore For Retrieval-Based Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Token-Level
Exactness [core] | Speculation Control [core]
Performance Analysis [core] | |
| DISD ยท 2024‑08
Coupling Without Communication and Drafter-Invariant Speculative Decoding | Autoregressive
External Drafter [inherited] | Collection [inherited] | Distribution-Level
Exactness [core] | -- |
|
| Draft & Verify ยท 2024‑08
Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core] |
|
| Feature Alignment ยท 2024‑08
Boosting Lossless Speculative Decoding via Feature Sampling and Partial Alignment Distillation | Proposal-State
Modeling [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| Harmonized Repr. ยท 2024‑08
Learning Harmonized Representations for Speculative Sampling | Proposal-State
Modeling [core] | Tree [core] | Distribution-Level
Exactness [core] | -- | |
| KOALA ยท 2024‑08
KOALA: Enhancing Speculative Decoding for LLM via Multi-Layer Draft Heads with Adversarial Learning | Target-Feature
Readout [core]
Proposal-State
Modeling [core] | Chain [core] | Distribution-Level
Exactness [core] | -- | |
| LASER ยท 2024‑08
Efficiency Unleashed: Inference Acceleration for LLM-based Recommender Systems with Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Relaxed
Acceptance [core] | -- |
|
| PEARL ยท 2024‑08
PEARL: Parallel Speculative Decoding with Adaptive Draft Length | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| SCD ยท 2024‑08
Speculative Contrastive Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Relaxed
Acceptance [core] | Performance Analysis [core] | |
| Dynamic Self-SD ยท 2024‑07
Accelerating Large Language Model Inference with Self-Supervised Early Exits | Target-Path
Execution [core]
Target-Feature
Readout [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core] |
|
| Graph-Structured ยท 2024‑07
Graph-Structured Speculative Decoding | Autoregressive
External Drafter [core] | Tree [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | Speculation Control [core] | |
| S2D ยท 2024‑07
S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Models | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | Speculation Control [core]
Serving Systems [core] | |
| Amphista ยท 2024‑06
Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference | Proposal-State
Modeling [core] | Tree [core]
Collection [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | -- | |
| EAGLE-2 ยท 2024‑06
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees | Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [inherited] | |
| Early-Exit Thompson ยท 2024‑06
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism | Target-Path
Execution [core]
Target-Feature
Readout [core] | Chain [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | State Management [inherited]
Speculation Control [core] | |
| GenZ ยท 2024‑06
Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models | Autoregressive
External Drafter [inherited] | Chain [inherited] | -- | Performance Analysis [core] |
|
| Multilingual Drafters ยท 2024‑06
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Performance Analysis [core] |
|
| OPT-Tree ยท 2024‑06
OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure | Autoregressive
External Drafter [inherited]
Proposal-State
Modeling [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] |
|
| Optimized ยท 2024‑06
Optimized Speculative Sampling for GPU Hardware Accelerators | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited]
Relaxed
Acceptance [core] | Serving Systems [core] | |
| SpecExec ยท 2024‑06
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices | Autoregressive
External Drafter [core] | Tree [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Serving Systems [core] |
|
| Clover ยท 2024‑05
Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge | Proposal-State
Modeling [core] | Tree [core]
Collection [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | -- | |
| Dynamic Lookahead ยท 2024‑05
Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| How Speculative ยท 2024‑05
How Speculative Can Speculative Decoding Be? | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Performance Analysis [core] |
|
| NEST ยท 2024‑05
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution | Context-Derived
Nonparametric Source [core] | Chain [inherited] | Relaxed
Acceptance [core] | Speculation Control [inherited] | |
| S3D ยท 2024‑05
S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs | Target-Path
Execution [core] | Block [core] | Token-Level
Exactness [core]
Relaxed
Acceptance [core] | Serving Systems [core]
Performance Analysis [core] | |
| SpecDec++ ยท 2024‑05
SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| Speculative Cascades ยท 2024‑05
Faster Cascades via Speculative Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [core]
Relaxed
Acceptance [core] | Performance Analysis [core] | |
| BASS ยท 2024‑04
BASS: Batched Attention-optimized Speculative Sampling | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | State Management [core]
Serving Systems [core] | |
| Exploring ยท 2024‑04
Exploring and Improving Drafts in Blockwise Parallel Decoding | Target-Feature
Readout [core]
Context-Derived
Nonparametric Source [core] | Block [core]
Collection [core] | Token-Level
Exactness [core] | -- | |
| Hidden Transfer ยท 2024‑04
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration | Target-Path
Execution [core]
Proposal-State
Modeling [core] | Tree [core] | Token-Level
Exactness [core] | -- | |
| LayerSkip ยท 2024‑04
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding | Target-Path
Execution [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] |
|
| Block Verification ยท 2024‑03
Block Verification Accelerates Speculative Decoding | Autoregressive
External Drafter [inherited] | Block [core] | Distribution-Level
Exactness [core] | -- | |
| Direct ยท 2024‑03
Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | -- | |
| Recurrent ยท 2024‑03
Recurrent Drafter for Fast Speculative Decoding in Large Language Models | Proposal-State
Modeling [core] | Chain [core] | Token-Level
Exactness [core] | -- |
|
| SDSAT ยท 2024‑03
SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens | Target-Path
Execution [core] | Block [core]
Tree [core] | Token-Level
Exactness [core] | Speculation Control [core] |
|
| Chimera ยท 2024‑02
Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens | Target-Feature
Readout [core] | Tree [core] | Token-Level
Exactness [core] | Speculation Control [core] | |
| Decoding ยท 2024‑02
Decoding Speculative Decoding | Autoregressive
External Drafter [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Performance Analysis [core] | |
| GliDe with a CaPE ยท 2024‑02
GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding | Autoregressive
External Drafter [core] | Collection [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core] | |
| Hydra ยท 2024‑02
Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding | Target-Feature
Readout [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited]
Relaxed
Acceptance [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| Lookahead ยท 2024‑02
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding | Context-Derived
Nonparametric Source [core] | Collection [core] | Distribution-Level
Exactness [core] | State Management [core]
Speculation Control [core] |
|
| Ouroboros ยท 2024‑02
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding | Autoregressive
External Drafter [core]
Context-Derived
Nonparametric Source [core] | Block [core]
Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| ProPD ยท 2024‑02
ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding | Target-Feature
Readout [inherited] | Tree [core] | Token-Level
Exactness [core] | State Management [core]
Speculation Control [core] | |
| Sequoia ยท 2024‑02
Sequoia: Scalable and Robust Speculative Decoding | Autoregressive
External Drafter [inherited] | Tree [core] | Distribution-Level
Exactness [core] | Speculation Control [core]
Performance Analysis [core] |
|
| Smart Auto-Correct ยท 2024‑02
Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding | Target-Path
Execution [core] | Block [core] | Distribution-Level
Exactness [core] | -- | |
| SpecStreaming ยท 2024‑02
Speculative Streaming: Fast LLM Inference without Auxiliary Models | Proposal-State
Modeling [core] | Tree [core] | Token-Level
Exactness [core] | State Management [core]
Serving Systems [core] | |
| EAGLE ยท 2024‑01
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty | Proposal-State
Modeling [core] | Chain [core]
Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Serving Systems [inherited] | |
| Kangaroo ยท 2024‑01
Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting | Target-Path
Execution [core]
Target-Feature
Readout [core] | Chain [core] | Token-Level
Exactness [core] | State Management [inherited]
Speculation Control [core] | |
| Medusa ยท 2024‑01
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads | Target-Feature
Readout [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited]
Relaxed
Acceptance [core] | Serving Systems [core] |
|
| Multi-Candidate ยท 2024‑01
Multi-Candidate Speculative Decoding | Autoregressive
External Drafter [core] | Collection [core]
Tree [core] | Distribution-Level
Exactness [core] | -- | |
| PaSS ยท 2023‑11
PaSS: Parallel Speculative Sampling | Target-Path
Execution [core] | Block [core] | Distribution-Level
Exactness [core] | -- | |
| REST ยท 2023‑11
REST: Retrieval-Based Speculative Decoding | Context-Derived
Nonparametric Source [core] | Tree [core] | Token-Level
Exactness [inherited]
Distribution-Level
Exactness [inherited] | Speculation Control [inherited]
Serving Systems [inherited] |
|
| Batching Synergy ยท 2023‑10
The Synergy of Speculative Decoding and Batching in Serving Large Language Models | Autoregressive
External Drafter [core] | Chain [core] | Token-Level
Exactness [core] | State Management [core]
Speculation Control [core]
Performance Analysis [core] | |
| DistillSpec ยท 2023‑10
DistillSpec: Improving Speculative Decoding via Knowledge Distillation | Autoregressive
External Drafter [core] | Chain [inherited] | Distribution-Level
Exactness [inherited]
Relaxed
Acceptance [core] | Speculation Control [inherited] | |
| Online SD ยท 2023‑10
Online Speculative Decoding | Autoregressive
External Drafter [core]
Target-Feature
Readout [inherited] | Chain [inherited] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Serving Systems [core] |
|
| SpecTr ยท 2023‑10
SpecTr: Fast Speculative Decoding via Optimal Transport | Autoregressive
External Drafter [inherited] | Tree [core]
Collection [core] | Distribution-Level
Exactness [core] | -- | |
| Staged ยท 2023‑08
Accelerating LLM Inference with Staged Speculative Decoding | Autoregressive
External Drafter [core]
Context-Derived
Nonparametric Source [core] | Tree [core] | Distribution-Level
Exactness [inherited] | Speculation Control [core]
Performance Analysis [core] | |
| SpecInfer ยท 2023‑05
SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification | Autoregressive
External Drafter [inherited] | Tree [core] | Token-Level
Exactness [core]
Distribution-Level
Exactness [core] | State Management [core]
Serving Systems [core] |
|
| Inference with Reference ยท 2023‑04
Inference with Reference: Lossless Acceleration of Large Language Models | Context-Derived
Nonparametric Source [core] | Chain [core] | Token-Level
Exactness [core] | -- |
|
| Big-Little Decoder ยท 2023‑02
Speculative Decoding with Big Little Decoder | Autoregressive
External Drafter [core] | Chain [core] | Relaxed
Acceptance [core] | Speculation Control [core] |
|
| Speculative Sampling ยท 2023‑02
Accelerating Large Language Model Decoding with Speculative Sampling | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| Speculative Decoding ยท 2022‑11
Fast Inference from Transformers via Speculative Decoding | Autoregressive
External Drafter [core] | Chain [core] | Distribution-Level
Exactness [core] | Speculation Control [core] | |
| SpecDec ยท 2022‑03
Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation | Non-Autoregressive
External Drafter [core] | Block [core] | Relaxed
Acceptance [core] | Speculation Control [core]
Serving Systems [core] | |
| Blockwise ยท 2018‑11
Blockwise Parallel Decoding for Deep Autoregressive Models | Target-Feature
Readout [core] | Block [core] | Token-Level
Exactness [core]
Relaxed
Acceptance [core] | Speculation Control [core] |
|
## โ๏ธ License
This repository is licensed under the [Creative Commons Attribution 4.0 International License](LICENSE).
