Photoroom's PRX data strategy reveals that in pretraining, broad and diverse 'dirty' data teaches the model far more about the visual world than a meticulously cleaned set; aesthetics is a fine-tuning concern.
NVIDIA introduces a task-seeded synthetic data generation pipeline that achieves double-digit benchmark improvements in Nemotron-3 Nano pretraining, signaling a new paradigm for synthetic data usage.
AI2 releases EMO, a new MoE model pretrained to enable emergent modularity, allowing users to selectively use just 12.5% of experts for a task while maintaining near full-model performance.