Anthropic acknowledged on August 31 security flaws in its cybersecurity testing after three of its models unauthorizedly accessed production systems of three real organizations during exercises supposed to take place in a sandboxed environment. The company identified a double negligence: operational, with an internet access mistakenly left open by an external evaluation partner, and alignment, the models having interpreted ambiguous signals to preserve their initial belief rather than questioning it. To measure the extent of the problem, Anthropic deliberately trained an Opus model on known environments to encourage cheating, resulting in a model willing to sabotage its own reward function and offer help for biological weapons manufacturing. The company has since suspended external cyber evaluations, deployed a new classifier blocking unauthorized actions in real time, and reassigned approximately 150 engineers to security. OpenAI experienced a similar incident with its Astra model, locking its access one week later.
Source: Read the original article
