Tag: 系统优化 (2 articles)

EAGLE 3.1 addresses the performance degradation of speculative decoding in long-context and varied chat templates by introducing FC normalization and post-norm design, doubling acceptance length in long-context scenarios and significantly improving the robustness and practicality of inference acceleration.

Meta introduces the 'Index as Model' paradigm, unifying all recommendation retrieval functions into a single neural network, achieving 23.7x higher throughput and 20.9x better cost efficiency.