An AI hallucination nearly set a Pentagon operation in motion
CNN reported that fabricated chatbot intelligence spread through command channels before an armed operation was aborted with aircraft already airborne.
US military aircraft were already airborne this spring when officials realised the intelligence behind an armed operation had been hallucinated by an AI chatbot. The operation was called off at the last minute, according to a CNN report.
The false intelligence began with an analyst at Special Operations Command who asked a chatbot to combine open source data with classified signals intelligence. The tool misread the vessel's cargo manifest and produced a report stating that the ship carried components for a nuclear weapons programme. The analyst then used the same tool a second time to turn the erroneous findings into an official-looking summary, which circulated across command channels.
Speed against oversight
The incident comes as the US military pushes to adopt AI to speed up decision-making. The Pentagon has described AI as a significant advantage in accelerating its kill chain so commanders can respond in time. The same speed, however, can carry hallucinations up the chain of command when human oversight is thin.
Sources: CNN.
