AI Doesn’t Need ‘Superintelligence’ or Evil Intent to Start a Nuclear War

Neither extraordinary machine intelligence nor malicious intent is required for catastrophe to result.

By Paul Slovic and Herbert Lin, Bulletin of the Atomic Scientists

When AI researcher Jacob Coxon announced his resignation from Anthropic this month, he warned that people at companies racing toward self-improving superintelligence “earnestly believe that it could kill us all by the end of the decade.” Such warnings raise fears that AI could escape human control and start a nuclear war without human authorization.

Although superintelligence is not precisely defined, AI developers agree it is not yet here. But today’s AI systems can cause nuclear war even if people nominally remain in charge. AI need not launch a weapon, recommend a strike, or intend harm. Ordinary human fallibility—amplified by machines—may be enough to lead to catastrophe.

AI can shape the information reaching leaders, whose minds are already struggling to comprehend the consequences of nuclear war. AI can exploit known vulnerabilities of human judgment compounded by false, misleading, or out-of-context machine outputs and amplified by the speed, scale, and volume of AI-generated information. Even accurate information can be interpreted improperly or unwisely when presented without proper context; errors and ambiguities introduced by AI can make matters worse.

Neither extraordinary machine intelligence nor malicious intent is required for catastrophe to result. To avoid nuclear war, institutional guardrails must be put in place that protect leaders from their worst impulses, even when AI pressures them to act.