The US Almost Attacked a Chinese Ship Because AI Made Up a Report
The U.S. military called off a planned attack on a Chinese ship after learning the intelligence was made up by an AI chatbot. Military planes were already in the air when the mistake was caught. TechCrunch
The false report said the ship was carrying parts for nuclear weapons. It spread during the war with Iran. It moved far enough up the chain of command to put planes in the air before it was stopped.
The problem started with one analyst at U.S. Special Operations Command. The analyst asked an AI chatbot to combine public shipping information with secret intercepted signals about the ship. The chatbot got the ship's cargo list wrong in that combined answer.
The analyst then asked the chatbot to rewrite those wrong findings as a clean, official-looking summary. That summary was shared across military channels as if it were checked intelligence.
The process had two steps. First, public and secret sources were mixed together in the chatbot, with no record of where each fact came from. Second, the chatbot gave the answer a formal look without any new checking. Once it looked official, it moved through normal channels. Planes launched because of it.
Jake Steckler, a GovAI research scholar and veteran U.S. Army officer, told TechCrunch that "it is especially critical for decisions that could lead to use of force to account for uncertainty inherent to LLMs."
The U.S. military is rapidly turning to AI to help with targeting, according to Israel Hayom. Delegates to China's main defence conference shared fears about unchecked AI development and AI military risks, Reuters reported on Sept. 16. President Donald Trump said on Sept. 14 that the U.S. already has guardrails in place to regulate and prosecute AI companies, Reuters reported.
The broader context here is how these chatbots work. They predict likely words, like autocomplete, and they sound confident even when they are guessing. Public shipping records are messy and incomplete. Secret intercepts are broken into pieces. The chatbot filled the gaps with smooth text.
In my view, the formatting matters as much as the error itself. Intelligence teams have long used warnings and grades to show strong and weak evidence. The chatbot removed those signals. It used the same firm tone for solid facts and weak guesses, which can mislead a busy reader during wartime.
Looking at tools, the fix is not a better question. Systems near targeting need built-in links to sources, scores for each claim, and output that cannot be mistaken for final intelligence. Asking for a summary should not create a shareable report in the same chat.
The optimistic reading over the long term is that this close call speeds up those fixes. Militaries will keep using AI because the speed gain is large and the data is too much to sort by hand. This case shows what to design against, without lives lost.
