Anthropic Researcher Resigns, Warning AI Could Kill All Humans
A current Anthropic alignment lead publicly agreeing with a departing researcher's warning, alongside a proposed federal ban on superintelligence, moves the industry's internal safety argument into the open and into legislation.
Reporting from 2 sources: ASCII.jp, GIGAZINE.
Jacob Coxon, a former Anthropic researcher, said on X that he resigned on September 8, criticizing OpenAI and Anthropic for racing toward self-improving superintelligence. Evan Hubinger, who leads AI alignment research at Anthropic, replied that Coxon is correct and put his own estimate at over 10 percent within the next decade. Senator Bernie Sanders and Representative Greg Casar announced a plan on September 3 to submit a bill banning superintelligence.
Coxon spent three years on pretraining research at OpenAI and Anthropic. He said neither company is acting responsibly and that both are racing toward self-improving superintelligence. Hubinger, who leads AI alignment research at Anthropic, wrote on X that Coxon is correct, that he believes Anthropic is trying its best, and that the company does not yet have a plan to solve alignment for superintelligence. In the United States, Senator Bernie Sanders and Representative Greg Casar announced on September 3 a plan to submit the Ban Artificial Superintelligence Act. The bill would permanently ban the development and deployment of superintelligence and pause advanced AI development until a new federal regulatory agency sets safety standards and a review system.
Synthesized by Yomimono from the 2 cited sources below, including Japanese-language reporting where cited, then editorially reviewed before publishing.