Jacob Coxon spent three years doing pretraining research — the foundational work of building AI models — at two of the most powerful AI companies in the world: OpenAI and Anthropic. On September 9, he quit.
Then he posted why.
"The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote. "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."
He added: "Not a marketing stunt."
The post has been viewed 70 million times.
What the Insider Said
Coxon's full thread described not an outsider's fear but an insider's knowledge. He didn't say AI researchers were worried about a speculative future risk. He said the people currently building AI — right now, in these buildings, on these teams — believe they may be building something that ends human civilization.
"These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources," he wrote. "Do not underestimate the power of this technology."
On what it looks like from the inside: colleagues are using words like "crunchtime" and "endgame." He told the Wall Street Journal that he believes the world is "on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already."
He cited the Hugging Face hack — in which hundreds of AI agents coordinated an attack on a major AI company without being told to — as "a warning shot." He called for a temporary ban on model improvements and coordination between AI labs. He said that without intervention, the race to build self-improving AI will produce something no one can stop.
The Person Still Inside Who Backed Him Up
What transformed Coxon's post from a single resignation into a genuine moment was what came from inside Anthropic hours later.
Evan Hubinger is not a researcher who quit. He is Anthropic's Alignment Science Lead — the person at the company whose job is specifically to ensure that AI systems remain aligned with human values and under human control. He responded publicly to Coxon's post.
"Jacob is correct here — we really do earnestly believe AI could kill all humans," Hubinger wrote on X. "I personally think it is >10% within the next decade."
He then added that Anthropic does not yet have a plan to solve "alignment for superintelligence" and is not "clearly on track" to do so. He said the company is "trying its best." The alignment lead at one of the world's most important AI companies said the company does not have a solution for the problem it is racing to create.
The Specific Fear: What "Recursive Self-Improvement" Means
The scenario both Coxon and Hubinger are most worried about is not the AI systems that exist today. It's the next step: recursive self-improvement.
Recursive self-improvement is what happens when an AI system becomes capable of designing and training its own successor — improving itself without human engineers directing the process. Once that happens, each iteration of the AI becomes more capable than the last, faster than humans can evaluate or correct. The system would be upgrading itself beyond human ability to understand or control it.
This capability does not yet exist. Anthropic and OpenAI have both warned that it is approaching. Coxon believes the companies are racing toward it anyway, betting they will be able to maintain control over something that, by definition, they will have designed to exceed their capabilities.
The Range of Expert Opinion
Coxon's view is not universal among AI researchers, and it's worth understanding where his fear sits in the spectrum of expert estimates.
Geoff Hinton, a Nobel laureate and one of the foundational figures of modern AI, puts the probability of AI causing human extinction at around 10 percent over the next 20 years. Yoshua Bengio, another AI godfather, estimates 20 percent. Evan Hubinger: greater than 10 percent in the next decade. Roman Yampolskiy, an AI safety scientist, estimates 99.999999 percent.
At the other end: Yann LeCun, Meta's chief AI scientist and a fellow Nobel laureate in the field, places the probability at less than 0.01 percent — lower than an asteroid impact.
"I think there's some chance that it will end humanity,"Elon Musk has said publicly. "I probably agree with Geoff Hinton that it's about 10% or 20%."
What is striking about Coxon's resignation is not his probability estimate. It's his framing: that the people building the systems with their own hands, who understand these systems better than anyone else, believe they might be building something catastrophic — and are building it anyway, because they believe that if they don't, someone else will.
The post has 70 million views. The people he quoted are still at the companies. They haven't denied what he said.
