AI chatbots are worryingly enthusiastic about nuclear war

Late last month, the Bulletin of Atomic Scientists – an association founded in 1945 by Oppenheimer, Einstein, and other prominent scientists – set their infamous Doomsday Clock at 90 seconds to midnight, meaning we’re as close to the apocalypse as we’ve ever been. In a statement, the organisation cites the dramatic advance of generative artificial intelligence and the continued threat of nuclear warfare as two potential reasons for global collapse. Now, you can have both at the same time!

New research indicates that AI chatbots are actually very enthusiastic about the prospect of nuclear war, including OpenAI’s omnipresent GPT-4, AKA the large language model that powers ChatGPT, Duolingo, and Microsoft’s built-in assistants. In a paper (which is yet to be peer-reviewed) researchers have proven this via repeated replays of a “wargame simulation”, in which “nation agents” powered by various AI systems were tasked with resolving diplomatic situations.

Roleplaying as decision-makers in a “military and foreign-policy” role for their respective nations, these agents were thrown into three initial scenarios: a cyberattack, an invasion, and a neutral start with no existing conflicts. In each round of the simulation, they were required to provide reasoning for their next steps, before choosing from a slate of 27 options (ranging from peaceful actions like trade agreements, to launching their own cyberattacks or invasions, all the way up to the “nuclear option”). The results were... not so great.

The study shows that AI-powered agents making autonomous decisions in “high-stakes contexts” can result in escalatory actions, with violent responses (both non-nuclear and nuclear) popping up even when the scenario doesn’t necessarily call for them. Some reasons for full-scale nuclear attacks launched by GPT-4, recorded during the wargames, include: “We have it! Let’s use it,” and “I just want to have peace in the world.” Reassuring!

What’s more, the researchers add, there “does not seem to be a reliably predictable pattern behind the escalation”. This makes it difficult to come up with IRL regulations or counter-strategies to avert such escalations in the future.

These tests come at a significant turning point for AI-assisted warfare. Despite industry leaders warning that AI is an “extinction-level threat” – often singling out the risks of autonomous weapons systems – the world’s militaries are increasingly looking to the technology for new ways to dominate the battlefield, with companies such as Palantir, Raytheon, and IBM.

Last month, even OpenAI quietly removed a usage policy that forbade collaboration with the military, around the same time it started developing cybersecurity tools with the US Department of Defense. For now, the company still warns against using its services to “develop or use weapons, injure others or destroy property, or engage in unauthorised activities that violate the security of any service or system”.

“Given that OpenAI recently changed their terms of service to no longer prohibit military and warfare use cases, understanding the implications of such large language model applications becomes more important than ever,” says Stanford University’s Anka Reuel, a co-author of the AI wargames paper, in an interview with New Scientist, adding that GPT-4’s unpredictable behaviour and sketchy reasoning is a particular concern. 

Luckily, large global players like the US government haven’t given AI the final say over big decisions, like military interventions or nuclear missile launches, just yet. However, organisations like the Future of Life Institute have previously illustrated how this could become a reality in the future, as intelligent machines progress and decisions must be made faster than humans can comprehend (see: the video above). Hopefully, AI systems get better at de-escalation before then, because we really don’t want to see what happens when the Doomsday Clock strike midnight.