Salesforce AI Research reports that training separate specialist models and then distilling their knowledge into a single model can reduce total training compute by 42% compared with joint reinforcement learning. In a setup combining mathematics and coding, the resulting student model matched the performance of GRPO while requiring 42% less total compute. Distillation itself accounted for only 5.2% of the specialists’ training FLOPs in this two-domain configuration, rising to 8.0% across four domains. The approach was less efficient on a per-FLOP basis when it relied on externally trained teachers, while teachers derived from the student’s own earlier checkpoints reduced distillation FLOPs by as much as a factor of 50.
Source: Read the original article
