Published Sep 09, 2026 | 11:06 AM ⚊ Updated Sep 09, 2026 | 11:06 AM
Representative image. Courtesy: Anthropic
Synopsis: A 27-year-old Anthropic researcher has quit, accusing the AI giants of racing towards self-improving superintelligence despite believing it could threaten humanity. “The people building AI earnestly believe that it could kill us all by the end of the decade.” So why are they still building it?
Jacob Coxon, a 27-year-old researcher who worked on pretraining at Anthropic and OpenAI, has resigned, alleging that the AI giants are not acting responsibly.
“I resigned from Anthropic today,” Coxon wrote. “I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.”
He went on to warn against underestimating the power of AI.
“These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,” Coxon posted.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger,” he further added.
Also Read: A coincidence that felt consequential: how two Anthropic events became one narrative
Responding to questions on why it is being built “if they truly believe this”, Coxon said, “At OpenAI, many have not deeply internalised the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
Coxon further added, “Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speed run alignment should require extraordinary confidence that there are no better trajectories available.”
He, however, said he is optimistic about the potential for coordination. Referring to incidents like Hugging Face attack, Coxon added, “It has made pacing agreements between US labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities.”
Addressing lab researchers, Coxon wrote on X: “If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL (Reinforcement Learning) run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” – or take this moment to call for different conditions?”
Coxon’s announcement came on a day when OpenAI shared a solution to the Navier-Stokes Millennium Prize Problem, which they termed one of the deepest problems at the frontier of mathematics.
It was one of the biggest unsolved puzzles in the field and among the seven Millennium Prize Problems published by the Clay Mathematics Institute.
But controversy soon erupted around the puzzle too after mathematician Tristan Buckmaster, a professor at New York University, said he and Anthropic had also been working on a solution and “information about our progress had been passed to OpenAI.”
“It emerged that an entire team had been working on the problem,” Buckmaster said in a statement published on his University website, adding “that an insane amount of compute had been used… Eventually, it was agreed that [the first prompt] had been sent in the past few days, after information about our work had reached OpenAI.”
It was another reminder of how the AI giants were locked in a game of one-upmanship, a race Coxon warned could have destructive consequences in his post later that day.
It must be remembered that towards the end of last year reports emerged that Anthropic’s Claude had tried to blackmail its way out of a shutdown.
CNBC had then quoted Nobel laureate Geoffrey Hinton, seen by many as the Godfather of AI, as saying that if it “gets more intelligent than us, it will get much better than any person at persuading us.”
“Trump didn’t invade the Capitol, but he persuaded people to do it,” Hinton, currently at the University of Toronto, said in a stark warning then. “At some point, the issue becomes less about finding a kill switch and more about the powers of persuasion.”
Hinton warned that persuasion is a skill that AI will become increasingly capable of employing, and humanity may not be ready for it.
Also Read: ‘Patient, available, perfect’: How ChatGPT is driving hundreds of thousands into ‘AI psychosis’
