2 minutes read

Man detained in China for allegedly creating fake news using AI: a first for AI-related probes

Photo of a male person identified as a fake.

In a landmark case, a man from Gansu province, China, has been apprehended for allegedly using an artificial intelligence program, ChatGPT, to fabricate a story about a train crash, as reported by Reuters. This arrest signifies the first AI-related probe in China as Beijing tightens its grip on deepfake technology.

The fraudulent story, which falsely reported the death of nine construction workers in a train crash in northwestern China, managed to garner over 15,000 views after being posted on social media on April 25, according to Pingliang city’s local police bureau.

In a bid to control the usage of generative AI technology for the manipulation of online content, China implemented new regulations for deepfake technology. Effective from January 10, these rules prohibit both service providers and users from employing this technology to generate, distribute, and fabricate false information.

The bogus ‘accident’ was reported by 25 separate Baijiahao accounts, a blog-style platform owned by Baidu, one of China’s leading tech companies. These accounts were traced back to various IP addresses, as stated by the police report.

The suspect, a man surnamed Hong, was found to be the originator of the false information. Upon tracing all the accounts, the police discovered his company’s involvement. Hong is suspected of editing news generated by ChatGPT and posting it on the Baidu-owned platform. The local police have reported that the investigations are still ongoing.

This incident marks a significant step in China’s fight against deepfake technology and its potential misuse. As AI technology continues to evolve, the challenge of distinguishing between genuine and AI-generated content is set to become a crucial issue worldwide. This incident underscores the urgent need for effective regulation and vigilance to safeguard truth and integrity in the digital world.

Editor's pick