OpenAI shuts down accounts tied to suspected Chinese surveillance tool

OpenAI has banned several accounts that allegedly used ChatGPT to build a social media monitoring tool meant to report anti-China protests in the US and other Western countries. The company claims the network was likely linked to China.

OpenAI is tightening its grip on AI misuse, shutting down several accounts that were reportedly using ChatGPT to power a social media surveillance tool. The accounts were allegedly tied to a Chinese-backed operation that aimed to collect real-time data on anti-China protests in the US, UK, and other Western nations, according to an OpenAI report.

The so-called "Qianyue Overseas Public Opinion AI Assistant" was described in marketing materials as software designed to track and report online discussions about human rights protests straight to Chinese authorities, intelligence agencies, and embassies. OpenAI couldn’t independently verify the tool’s existence but had access to promotional materials that detailed its capabilities – mainly scanning platforms like X, Facebook, and Instagram for relevant conversations.

Ben Nimmo, a lead investigator at OpenAI, called the case a troubling example of an "authoritarian regime trying to exploit US-based AI for anti-democratic purposes." The findings come as tensions grow over China’s use of Western technology, with OpenAI actively warning US policymakers about the national security risks of Chinese AI.

The banned accounts also referenced using other AI models, including Meta’s open-source Llama. Meta acknowledged the possibility but pointed out that bad actors have access to plenty of AI tools, including those developed in China.

This isn’t the first time OpenAI has taken action against questionable activity. The company has also banned accounts linked to Iranian influence operations, North Korean-related employment scams, and another China-affiliated network generating Spanish-language content critical of the US government.

With the rise of Chinese AI startups like DeepSeek, OpenAI is considering opening up some of its own models to compete. Meanwhile, US policymakers are grappling with how to balance AI innovation with national security concerns.