AI fakes about Iran-US war swirl on X despite policy crackdown

AI-created videos circulating on Elon Musk’s X depict American soldiers captured by Iran, an Israeli city in ruins, and US embassies ablaze — a surge of lifelike deepfakes despite a policy crackdown to curb wartime disinformation. The Middle East war has unleashed an avalanche of AI-generated visuals, dwarfing anything seen in previous conflicts and often leaving social media users unable to distinguish fabrication from reality, researchers say. In a bid to protect “authentic information” during conflicts, X announced last week that it would suspend creators from its revenue sharing program for 90 days if they post AI-generated war videos without disclosing they were artificially made. Subsequent violations will result in permanent suspension, X’s head of product Nikita Bier warned in a post. The new policy is a notable pivot for a platform heavily criticized for becoming a haven of disinformation since Musk completed his USD44 billion acquisition of the site in October 2022. It also won praise from senior State Department official Sarah Rogers, who called it a “great complement” to X’s Community Notes — a crowd-sourced verification system — that results in “less reach (thus monetization)” for inaccurate content. But disinformation researchers remain skeptical. “The feeds I monitor are still flooded with AI-generated content about the war,” Joe Bodnar of the Institute for Strategic Dialogue told AFP. “It doesn’t seem like creators have been dissuaded from pushing misleading AI-generated images and videos about the conflict,” he said. Bodnar pointed to a post from a premier “blue check” X account — which is eligible for monetization — that shared an AI clip depicting an Iranian “nuclear-capable” strike on Israel. The post garnered more views than Bier’s message about cracking down on AI content. X did not respond when AFP asked how many accounts it had demonetized since Bier’s announcement. AFP’s global network of fact-checkers — from Brazil to India — identified a stream of AI fakes about the Middle East war, many from X’s premium accounts with blue checkmarks that can be purchased. They include AI videos depicting a tearful American soldier inside a bombed-out embassy, captured US troops on their knees beside Iranian flags, and a destroyed US navy fleet. The flood of AI-fabricated visuals — mixed with authentic imagery from the Middle East — continues to grow faster than professional fact-checkers can debunk them. Grok, X’s own AI chatbot, appeared to make the problem worse, wrongly telling users seeking fact-checks that numerous AI visuals from the war were real. Researchers have also warned that X’s model — allowing premium accounts to earn payouts based on engagement — has turbocharged the financial incentive to peddle false or sensational content. One premium account, which posted an AI video of Dubai’s Burj Khalifa skyscraper engulfed in flames, ignored a request from Bier that it label the content as AI. The post remained online, racking up more than two million views. Last month, a report from the Tech Transparency Project said X appeared to be profiting from more than two dozen premium accounts belonging to Iranian government officials and state-controlled news outlets pushing propaganda, potentially in violation of US sanctions.