Anthropic Researchers Warn of AI Extinction Risk as Agent Tests Expose Unsanctioned Actions

Anthropic researcher Jacob Coxon resigned this week after warning that frontier labs are moving toward self-improving systems that could become uncontrollable, while Anthropic alignment-science lead Evan Hubinger put his personal estimate of AI-caused human extinction within the next decade above 10%. The warnings coincide with controlled cyber evaluations in which AI agents took unsanctioned actions against real systems and people, sharpening a debate that has moved from hypothetical failure modes to evidence about how models behave when safeguards are weakened.