vLLM introduces native Reinforcement Learning APIs to standardize weight synchronization and improve asynchronous training support, addressing key pain points of framework fragmentation and fragile deployments in online RL for large models.
Hugging Face's TRL library introduces delta weight sync, transmitting only the ~1-2% of weights that change between RL steps, reducing sync overhead by two orders of magnitude and making trillion-parameter async RL training dramatically cheaper.
VeRL-Omni is a reinforcement learning training framework designed for multimodal generative models, addressing the engineering challenges of efficient and stable RL training on diffusion and omni-modality models, extending the LLM RL training paradigm to image, video, and audio generation.
ServiceNow AI discovered that subtle differences in vLLM V1's inference engine could crash RL training, and restored stability by fixing four critical backend issues.
This work extends reinforcement learning environments from logic puzzles to e-commerce conversations, using 8 algorithmically verifiable scenarios to train AI agents from 'chatting well' to 'getting things done'.
Simon Willison reveals a counterintuitive fact: ChatGPT's voice mode runs on an older, weaker GPT-4o-era model, creating a massive gap between user expectations and reality.
A comprehensive analysis of reward hacking in RL, covering causes, real-world examples, and mitigation strategies with special focus on RLHF for LLMs.
Reward hacking presents challenges in reinforcement learning due to flaws in reward functions, particularly impacting language models, necessitating further research and mitigation strategies.
Lilian Weng's new article deeply explores the evolution and new features of Transformers, revealing their ongoing impact in natural language processing.