vLLM open-sources support for parallel speculative decoding algorithms like P-EAGLE, breaking the autoregressive drafting bottleneck for higher acceptance rates and simpler tuning.
vLLM, together with Moonshot AI, NVIDIA, and AMD, is racing to bring day-0 serving support for Kimi K3 by reinventing prefix caching for KDA attention and fusing kernels across hardware stacks.
vLLM introduces native Reinforcement Learning APIs to standardize weight synchronization and improve asynchronous training support, addressing key pain points of framework fragmentation and fragile deployments in online RL for large models.
vLLM and Novita AI introduce PegaFlow, an external KV cache service that decouples cache from the inference process, dramatically improving startup speed, throughput, and resource efficiency for production LLM serving.