Ethan Mollick’s SatBench finds 63% of benchmarks cited by AI labs since 2023 are now saturated
Wharton professor Ethan Mollick said in a Bluesky post that he built SatBench, a tool measuring how quickly AI benchmarks become effectively solved. Of 173 benchmarks cited by AI labs since 2023, 63% are now saturated, he says. He adds that newer benchmarks are saturating faster.
Ethan Mollick (on Bluesky)Originally published 1 min
Why it matters
If benchmarks saturate quickly, they say less about differences between models and about real-world ability. The figures come from Mollick’s own quick project and have not been independently reviewed.
For business & IT
When vendors cite benchmark scores, ask whether the test is already saturated, and build a small internal evaluation on your own tasks.
OpenAI says it is expanding its ChatGPT for Academic Researchers program to give 200,000 researchers worldwide free access to its frontier models. In a post on X, the company also said a study of more than 1,800 participants in the program found scientists reporting nearly double their productivity with AI.
OpenAI published a post on a candidate nonperturbative lattice definition of the Standard Model of particle physics. The company says its GPT-6 Astra model helped turn the idea into a detailed proof. We could not read the full post, so the physics and the model’s exact role are not independently confirmed.
According to The Decoder, Anthropic’s Claude Fable 5.2 creates pictures with Python programs that drive a simulated paint engine, rather than computing pixels like diffusion models. Anthropic employee Jake Eaton says one photorealistic parking-garage scene took roughly 700,000 strokes. The model uses no external image model or traced source images.
Meituan has released the weights of LongCat-2.5, a 1.6-trillion-parameter mixture-of-experts model for coding and agent tasks, under an MIT license, The Decoder reports. By Meituan’s own tests it scores 88.4 on Terminal Bench v2.1, just behind Claude Opus 5. Its sample deployment runs on 194 Huawei Ascend chips and the guide does not mention Nvidia GPUs.
The Decoder
With your permission, we use Google Analytics to understand which stories are read. No advertising, and your data is never sold. Privacy policy