Tag: 投机解码 (2 articles)

vLLM introduces adaptive verification that dynamically adjusts the verification budget based on token confidence, making speculative decoding effective even at high concurrency and reducing tuning overhead.

EAGLE 3.1 addresses the performance degradation of speculative decoding in long-context and varied chat templates by introducing FC normalization and post-norm design, doubling acceptance length in long-context scenarios and significantly improving the robustness and practicality of inference acceleration.