← Back to Home

Tag: 工程实践 (16 articles)

Making Knowledge Distillation Cheap Enough to Run at Scale

Multiverse Computing introduces a memory-efficient knowledge distillation technique that uses cached top-K logits and a fused chunked KL loss to train on long contexts with just a single GPU, drastically reducing costs.

Hugging Face Blog · Aug 10, 2026

Model Routing Is Simple. Until It Isn’t.

Model routing isn't a classification problem—it's a systems optimization challenge: real costs depend on cache hit rates, not just token pricing, and task difficulty is often invisible at routing time.

Hugging Face Blog · Jul 16, 2026

Rewriting Bun in Rust

AI coding agents have changed the fundamental assumptions of software engineering: Bun's developer used AI agents to rewrite the project from Zig to Rust in just 11 days, proving that large-scale rewrites are no longer taboo.

Simon Willison · Jul 9, 2026

Quoting Kenton Varda

Kenton Varda halted AI-generated PR descriptions on his team, as AI focuses on obvious code details while missing the high-level context, revealing a critical flaw in AI-assisted software communication.

Simon Willison · Jul 9, 2026

Engineering TTS Inference in vLLM-Omni

TTS inference is a heterogeneous pipeline combining latency-bound and throughput-bound stages, making traditional LLM optimization strategies ineffective and requiring architecture-aware scheduling.

vLLM Blog · Jun 23, 2026

Quoting Drew Breunig

When top models no longer hide engineering flaws with a 'free lunch,' developers must rethink the balance of context strategies, workflow design, and cost efficiency.

Simon Willison ·