Hugging Face and IBM launch the Open Agent Leaderboard, shifting evaluation from standalone models to full agent systems (including tools, planning, memory), while measuring both performance and cost.
Hugging Face reveals the bottleneck of alternating CPU/GPU waits in continuous batching, and shows how asynchronizing their workloads can yield a free 24% throughput boost.
ServiceNow AI discovered that subtle differences in vLLM V1's inference engine could crash RL training, and restored stability by fixing four critical backend issues.
Hugging Face argues that the rise of AI-driven autonomous cybersecurity systems (like Mythos) reveals the critical structural advantage of open source in enabling distributed defense and mitigating risks from closed-source software.
Anthropic clarifies that Claude Code quality issues were not model-related, but stemmed from three complex bugs in the engineering framework, revealing deep challenges in AI Agent system engineering.
LangChain introduces the 'Better-Harness' system, treating evaluations as 'training data' for agents, iteratively optimizing the engineering framework (harness) to improve agent performance, with a core focus on avoiding overfitting and achieving generalization.