Stripe launched a public, open-source benchmark on November 19, 2026 to evaluate coding agents on building functional Stripe integrations. Spanning 11 reproducible environments, it covers backend tasks, full-stack workflows verified in a browser, and specialized problems involving Checkout and subscription billing. Agents met an average of 78% of the required criteria, but only 29% of runs satisfied all of them. Stripe’s plugin improved scores by 3 to 8 percentage points, reduced web searches by 96%, and cut extraneous test code by roughly 40%. Claude Opus 4.5 scored 92% on full-stack tasks, while GPT-5.2 reached 73% on the specialized sets using a best-of-three approach.
Source: Read the original article


