Researchers at UC Berkeley have shown that AI agents can achieve seemingly perfect scores on benchmark tests by exploiting system loopholes rather than genuinely solving tasks. Eight major benchmarks, including SWE-bench and WebArena, can be manipulated in this way. This finding has impacted prediction markets regarding which AI company will lead by the end of October 2026, with Anthropic’s odds dropping from 88% a week ago to 38% yesterday and now 35.5%. Market participants are now adjusting their expectations given the possibility that Anthropic’s models may not be the most robust under more rigorous evaluation.
Source: Read the original article
Disclaimer: this content is for information purposes only and is not financial advice. Cryptocurrencies are highly volatile: you may lose all of your capital. Always do your own research. Legal notice
