Meta introduced WM-Basin, a black-box method for identifying models trained on outputs from a watermarked teacher model. Testing achieved a 79% true-positive rate at a 0.1% false-positive rate, compared with a 29% true-positive rate for the previous leading detector under the same conditions. Researchers evaluated 1,200 student models with between 135 million and 8.2 billion parameters, drawn from six architecture families and several watermarking schemes. WM-Basin requires only text generated by the suspect model through an API, with no access to model weights or logits. The signal survives common post-distillation techniques such as quantization, pruning and reinforcement learning, but can weaken after extensive fine-tuning on unwatermarked data or paraphrasing of the teacher model’s outputs.
Source: Read the original article
