Tag: 模型量化 (3 articles)

Liquid AI uses Quantization-Aware Distillation to recover 97% of the accuracy lost in 4-bit quantization, enabling faster, high-quality model inference on edge devices.

vLLM provides Day 0 native support for Qwen's 2.4 trillion parameter open-weight model Qwen3.8, demonstrating the rapid co-evolution of open-source inference frameworks and frontier models.

vLLM provides day-0 support for TML Inkling, achieving 380 tok/s on 4 GB200 GPUs with full feature parity, 1M context, and multimodal input.