All topics Topic

Speculative Decoding

1 article

Apr 14, 2025 1 min read

Speculative Decoding: Accelerating LLM Generation

Speculative decoding can achieve 2-3x speedup without any quality degradation.

AI LLM Performance