SkyRL introduces IsoExec, a cross-framework unified execution abstraction that eliminates floating-point mismatch between training and inference engines in RL workloads through a shared execution contract and bitwise-consistent kernels.
Liquid AI releases DSpark draft models for its LFM2.5 family, achieving up to 3.2x faster inference through speculative decoding without compromising output quality.
By keeping unsummarized lessons and retrieving them on demand, ALTK-Evolve slashes inference token costs, showing agentic memory is shifting from context dumping to precision delivery.
vLLM achieves 25K tokens/sec/GPU on Qwen3.5 through Blackwell-optimized GDN kernels, hybrid cache/state transfer, and race-free async scheduling — the optimization story goes beyond raw numbers.
OpenAI slashed GPT-5.6 Luna's price by 80% by using the Sol model to self-optimize inference kernels, undercutting Google's cheapest model and signaling a shift toward AI self-improvement.
vLLM Semantic Router evolves from single model selection to a Mixture-of-Models system architecture, coordinating multiple specialized models through a unified interface for integrated training, evaluation, and inference.
vLLM integrates TileRT via a public connector, making decode pluggable for latency-critical workloads without sacrificing the existing ecosystem.
Tencent Hunyuan contributed production-grade high-performance Attention and MoE kernels to vLLM, achieving up to 2.95x attention speedup for mixed-length decode and 1.59x MoE speedup on H20 GPUs, reducing end-to-end latency by up to 24%.
vLLM Semantic Router introduces Fusion, a routing primitive that lets a panel of models produce independent answers, has a judge model analyze them, and synthesizes a single response — making model composition a first-class serving pattern.
Poolside's 33B-parameter agentic coding model, Laguna XS.2, achieves 2-3x inference speedup without quality loss through native vLLM integration, DFlash speculative decoding, and LLM Compressor quantization.
NVIDIA's new diffusion language models generate tokens in parallel and refine them iteratively, potentially breaking the latency limits of traditional autoregressive models and enabling self-correction.
Hugging Face reveals the bottleneck of alternating CPU/GPU waits in continuous batching, and shows how asynchronizing their workloads can yield a free 24% throughput boost.
DeepSeek-V4 makes million-token context windows practically usable for long-running AI agents by dramatically cutting inference costs and memory usage through its novel hybrid attention architecture.
OpenAI's huge price drop isn't just a price war—they used a smarter model (Sol) to rewrite inference kernels, cutting costs by 20%. This reveals a new trend: AI is becoming the optimizer of its own infrastructure.
DeepSeek V4 achieves efficient million-token long-context inference on vLLM through innovative KV cache compression and sparse attention mechanisms, marking a new era for long-text processing.
vLLM introduces Elastic Expert Parallelism (Elastic EP), enabling runtime scaling of MoE inference deployments by adding or removing GPU workers without restarts, adapting to demand fluctuations and laying the groundwork for fault-tolerant serving.
vLLM integrates Mooncake's distributed KV cache to solve the bottleneck of recomputing long context prefixes in agentic workloads, achieving a 3.8x throughput increase and a 46x reduction in time-to-first-token.
The Speculators v0.5.0 release introduces the DFlash algorithm for speculative decoding, which generates draft tokens in a single forward pass, significantly reducing inference latency, and unifies online and offline training workflows.
vLLM uses FP8 quantization for KV cache to halve memory usage and double throughput for long-context inference while maintaining accuracy, though specific performance pitfalls need attention.
Hybrid models significantly outperform pure Transformers in semantic understanding and dynamic context tracking, but lag in verbatim repetition, revealing a clear architectural division of labor.