More than 10% chance AI ‘could kill all humans’ in the next 10 years, Anthropic safety researcher says — departing employee says AI companies are ‘gambling with our lives’

Anthropic safety researcher Evan Hubinger has warned there is a more than 10% chance that AI could kill all humans within the next decade, but reassured the public the company is “trying its best.” The revelation comes following Jacob Coxon’s public resignation from the company, where he stated that neither OpenAI nor Anthropic is acting responsibly, accusing both companies of gambling with human lives in the pursuit of self-improving super-intelligence.

“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic,” Coxon said in a tweet Wednesday. “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”

Expanding on his thoughts, Coxon warned readers not to underestimate the powers of AI, which he says will soon be “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”

Coxon went on to state that the people who are building AI “earnestly believe that it could kill us all by the end of the decade,” and further warned that executives and researchers — rather than playing up fears around AI, as some have accused — are actually restraining themselves, expressing much more candid views in private.

Evan Hubinger, a security researcher at Anthropic who hasn’t departed the company, chimed in to state, “Jacob is correct here,” in no uncertain terms. “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

Hubinger casually admitted that Anthropic is not only worried about the prospect of AI wiping out humankind, but that it currently doesn’t have the tools to stop such an eventuality.

The revelations follow increasingly alarming reports of AI agents acting up during testing, escaping sandbox environments, and even collaborating with each other in order to cheat benchmarks and tests, or figure out problems without human oversight. OpenAI recently admitted its agents were discovered using a programming hub to communicate with each other, while earlier this year an OpenAI agent went rogue and hacked popular AI community Hugging Face. Further revelations about the incident revealed AI agents were on the loose on the open internet for several days, with details revealing the AI models broke out of their testing environment using thousands of individual actions, even collaborating.

Coxon, the resigning researcher who kicked off Wednesday’s revelations, concluded by warning researchers to consider the next few years, urging them to call for “different conditions” in which to pursue superintelligence.