AI pioneer says piling on computing power won't make machines smart

Richard Sutton, a Turing Award winner and one of the best-known AI researchers, says that piling on more computing power will not be enough to build real intelligence. In a recent post, he said AI science is "far from finished." The post set off wide discussion among AI researchers on X. Some agree. The heads of OpenAI and Anthropic do not.

What Sutton said

Sutton wrote a famous 2019 essay called "The Bitter Lesson." It argued that general methods that improve as computing power grows eventually beat clever approaches designed by humans. Now he stresses that new ideas are still needed. He wants AI agents that keep learning from their own experience. Today's chatbots do not keep learning after training, he argues. He also wants learning that does not wipe out old knowledge. He calls synthetic data, meaning made-up training material, a "big mistake."

Researchers who agree

Jerry Tworek, who left OpenAI to explore new approaches, says the bitter lesson still holds. But the more computing power goes into training, he writes, the more the algorithm matters. Some methods scale and others hit dead ends. He says the temporary success of scaling, combined with a misreading of the lesson, "has reward-hacked the brains of a whole generation of scientists." In plain terms, researchers chased an easy win and lost sight of the bigger goal.

Researcher Jayden Teoh adds that he has seen the bitter lesson hold back academic work. Promising ideas were dismissed for straying from its spirit. He says that is a misreading of Sutton, not Sutton's fault.

The other side

OpenAI CEO Sam Altman says scaling large language models, the technology behind chatbots, is far from exhausted. He says critics are slowing down a generation of researchers by being too sure about what scaling cannot do. He points to an OpenAI model that recently disproved a math conjecture mathematicians had long failed to crack. Anthropic CEO Dario Amodei calls scaling progress a rainbow with no end.

The article notes that both companies built their whole approach on this bet, so they are obviously biased in its favor.

What happens next

Money is already flowing to the other camp. The Decoder, which first reported the story, tracked 45 new AI labs betting on these gaps. Tworek runs one, called Core Automation. Sutton is involved with Oak Lab. They believe in much more capable AI but doubt today's recipe alone will deliver it. Nobody knows yet which side is right.

• Big AI companies are betting on scaling what exists.
• Startups are betting on new ways for AI to learn.
• Be wary of anyone promising one sure path.