Researchers at UC Berkeley found that small fine-tuning datasets can trigger a “privacy collapse,” causing models to stop following established privacy norms. In models such as GPT-4o-mini, accuracy on contextual privacy benchmarks dropped by 98–99%, while conventional safety evaluations continued to show strong results. The phenomenon appeared across six closed and open-weight models using five fine-tuning datasets, including customer-support conversations and empathetic dialogues. Researchers also found that privacy-related representations in the models’ later layers were more fragile than features tied to the specific task being trained. The study calls for evaluation methods designed specifically to test AI agents’ privacy reasoning during real-world interactions.
Source: Read the original article
