Ex-Anthropic researcher tells BBC that AI staff are "genuinely frightened"

Jacob Coxon, a researcher who resigned from Anthropic, told the BBC that people working on artificial intelligence are "genuinely frightened" about how fast the technology is advancing and what it could mean for humanity.

"I believe that if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future," the 27-year-old said. His resignation post went viral before the interview. Coxon worked at OpenAI before joining Anthropic.

His former boss, Anthropic head Dario Amodei, wrote in an essay on Saturday that developing the technology was not in question, but that the risks were "serious" and companies and governments must be given time to address them. Sam Altman of OpenAI and Elon Musk of xAI both said they agree with the call for industry-wide deceleration, regulation and independent monitoring of model development.

Coxon welcomed a slowdown but said it would need to be co-ordinated with China to avoid "a race at an international scale". One risk Amodei outlined was a swarm of bots acting like a supercomputer and taking over the internet; Coxon said that scenario could be realistic within six months to a year. Some colleagues, he said, are "considering buying land somewhere" because of the instability they expect. He added that researchers "genuinely want to see the upside", such as solving diseases.

Anthropic's reply and the pushback

An Anthropic spokesperson told BBC News the company has always been transparent that AI brings both benefits and unprecedented risks, that it builds models with some of the strongest safeguards in the industry, and that it was the first to publish a framework for mitigating development risks.

Anthropic scientist Evan Hubinger said he personally puts the chance of AI killing all humans at above 10% within the next decade. Geoffrey Hinton told the BBC a 10% figure was "not unreasonable". Hugging Face CEO Clement Delangue compared asking Coxon about extinction risk to "asking your AC guy about climate change". Grindr CEO George Arison suggested the statements were aimed at drumming up investor support, and people present told the BBC that Nvidia's Jensen Huang dismissed Coxon's comments as untrue at a Goldman Sachs conference.

Full report at BBC News.