An AI Hallucination Nearly Triggered a US Military Operation

A US intelligence report—generated with the help of AI tools and later judged "entirely false"—nearly led the military to board a Chinese ship. Ars Technica and TechCrunch both relay a CNN report placing the episode this spring.

What happened

Per Ars, a US Special Operations Command analyst used a chatbot to analyze the ship's manifest, producing the false claim that a Chinese vessel was transporting nuclear-weapons-program components through the Middle East. The military was preparing to intercept and board the ship with air support before officials discovered that the chatbot had "inaccurately identified the material the ship was carrying." One source told CNN the AI-powered fiasco "almost started a war." TechCrunch adds that the report circulated during the conflict with Iran, and that the analyst used the tool a second time to format the erroneous findings into an "official-looking" summary that spread through command channels.

Why it went wrong

Per Ars, the chatbot "fused together open-source intelligence with secret signals intelligence in government holdings" and packaged it into a report. It wasn't deceived so much as it fabricated where its context ran out—the signature of a large-language-model "hallucination." Jake Steckler, a GovAI research scholar and retired US Army officer, told TechCrunch that service members must "understand the uncertainty inherent to LLMs," and that this is "especially critical for any decisions that could lead to use of force."

The bigger backdrop

Ars notes the Department of Defense has been accelerating AI adoption: it rolled out an "AI acceleration strategy" in January; last December it picked Google's Gemini for Government as the basis for its "GenAI.mil" platform, added Grok for Government last month, and Anthropic offers a customized Claude for US intelligence work. In June, a Pentagon representative told Congress that roughly 1.5 million active DoD personnel have used the military's generative-AI tools.

Our take: What should worry readers isn't how capable the chatbot was, but that false intelligence traveled up the chain of command and nearly triggered a military action before anyone questioned it. Speed is exactly why the military wants AI—and exactly how a hallucination climbs the ladder.