Articles Tagged "Benchmarks"

75% of AI Coding Agents Break Working Code Over Time

75% of AI Coding Agents Break Working Code Over Time

Alibaba's SWE-CI benchmark tested 18 AI models on 100 real codebases across 233 days of maintenance. Most agents accumulate technical debt and break previously working code. Only Claude Opus stays above 50% zero-regression.

Gemini 3.1 Pro Tops Benchmarks but Developers Can't Rely on It

Gemini 3.1 Pro Tops Benchmarks but Developers Can't Rely on It

Gemini 3.1 Pro leads ARC-AGI-2, LiveCodeBench, and 11 other benchmarks with 750 million users and 21.5% market share - but developers report stalled responses, leaked thinking tokens, and API outages that make it unusable for production coding and agent workflows.