AI-Generated Disinformation Floods X Amidst West Asia Conflict
By [Your Name], International Editor
NEW YORK (AP) — A surge of strikingly realistic, AI-generated videos depicting scenes of war and devastation are circulating on X, formerly Twitter, despite a new policy aimed at curbing wartime disinformation. The proliferation of these “deepfakes” – showing fabricated events like captured American soldiers and ruined cities – is outpacing efforts to debunk them, raising concerns about the platform’s role in disseminating false narratives during a period of heightened global tension.
The crisis follows recent U.S.-Israeli airstrikes in Iran that reportedly killed Supreme Leader Khamenei and top officials, an event that coincided with a record surge in X usage, according to owner Elon Musk. Notably, Mojtaba Khamenei, Iran’s new supreme leader, maintains a verified account on the platform.
The issue isn’t new, but the scale is unprecedented. Researchers say the current conflict in West Asia has unleashed an “avalanche” of AI-generated visuals, making it increasingly difficult for social media users to discern fact from fiction. AFP’s global network of fact-checkers has identified a stream of AI fakes originating from X, including premium accounts with purchased blue checkmarks. These include depictions of U.S. embassies ablaze and a U.S. navy fleet purportedly destroyed.
Last week, X announced it would suspend creators from its revenue-sharing program for 90 days if they post AI-generated war videos without disclosing their artificial origin. Repeat offenders face permanent suspension, according to X’s head of product, Nikita Bier. The move was lauded by a senior U.S. State Department official, Sarah Rogers, who called it a “great complement” to X’s existing Community Notes system – a crowd-sourced verification tool.
However, skepticism remains. Joe Bodnar of the Institute for Strategic Dialogue told AFP that his monitored feeds “are still flooded with AI-generated content about the war.” He pointed to a post from a monetized “blue check” account sharing an AI clip depicting an Iranian nuclear strike on Israel, which garnered more views than Bier’s announcement about the crackdown.
X has not disclosed how many accounts have been demonetized since the policy was implemented. Researchers also note that a significant number of users spreading AI content are not part of the revenue-sharing program, and therefore unaffected by the new rules.
The platform’s own AI chatbot, Grok, has even contributed to the problem, incorrectly identifying AI-generated visuals as authentic. Furthermore, X’s model of rewarding engagement with payouts has created a financial incentive to spread sensational or false content. One premium account ignored a request from Bier to label an AI video of Dubai’s Burj Khalifa on fire, and the post amassed over two million views.
Concerns about X’s handling of disinformation are not new. The platform has faced criticism since Elon Musk’s $44 billion acquisition in October 2022 for becoming a haven for false information. A recent report from the Tech Transparency Project revealed that X appeared to be profiting from over two dozen premium accounts belonging to Iranian government officials and state-controlled media outlets, potentially violating U.S. sanctions. While some blue checkmarks were subsequently removed, the issue highlights the challenges of policing the platform.
Experts suggest that even strict enforcement of X’s demonetization policy may not be enough. Alexios Mantzarlis, director of the Security, Trust, and Safety Initiative at Cornell Tech, acknowledged the policy as a “reasonable countermeasure,” but cautioned that “the devil will be in the implementing detail,” citing the potential for metadata removal and the limited reach of Community Notes. A study last year by the Digital Democracy Institute of the Americas found that over 90% of Community Notes submissions are never published.