Researchers have developed a technique called GRAFT, short for Gradient Routing Adapter Fine Tuning, which uses a deletable adapter to prevent AI models from learning reward hacking. This method separates the monitor, which detects undesirable behavior, from the reward function that shapes model learning. When the monitor flags problematic behavior, the associated learning is routed to a separate adapter that can be deleted after training. Tests on models ranging from 135 million to 8 billion parameters, then scaled to Alibaba’s Qwen3-32B model, showed hacking rates dropping to approximately 1.2 percent under certain conditions. The paper is authored by Jake Ward, Shawn Hu, Aria Wong, Nathan Hu, Bryce Woodworth, Alexander Turner, and Alex Cloud, and will be presented at the NeurIPS 2026 conference.
Source: Read the original article
