A study posted on arXiv shows that reasoning models suffer an entropy collapse in the first tokens of their reasoning, leading them to repeatedly fall back on the same opening patterns. Two low-cost interventions were tested: prefilling the beginning of the reasoning or banning common opening tokens while adding examples of reasoning that follows the instructions. These methods improved instruction compliance by 34 to 41 percentage points across the models tested. The gains were strongest when models had to add information or change their style, but their effects were inconsistent when models had to suppress specific content. The authors therefore argue that addition, style and suppression should be evaluated separately to avoid overstating how controllable the models are.
Source: Read the original article
