EU Proposes Ban on AI-Generated Child Abuse Content

The European Commission is considering the introduction of criminal liability for AI-generated images and deepfakes depicting child sexual abuse (CSA).

Child sexual abuse is a heinous crime which has evolved significantly over the past years.

Today, we are adopting a proposal to update the criminal law rules on child sexual abuse and sexual exploitation.

Learn more → https://t.co/mtz68GuEJJ#SecurityEU pic.twitter.com/QF7TsFX7cy

— European Commission (@EU_Commission) February 6, 2024

The proposed legislation criminalizes the distribution of content involving the abuse of minors. The document introduces a new criminal offense.

The provisions include penalties for possession and distribution of “pedophile manuals” for creating CSA materials using neural networks. The proposal also mentions unnamed AI-based chatbots used for these purposes.

“The threat of abuse is real and has increased across the EU. For instance, pedophile manuals continue to appear online, indicating a growing threat to the most vulnerable groups. The presence of children online and technological innovations create new opportunities for abuse. In 2022 alone, 1.5 million reports of CSA were registered in the European Union,” the statement from the agency reads.

To implement the initiative, the European Commission intends to amend the relevant law from 2011. However, the final decision on the proposed rules will be made by the European Parliament and the European Council.

These measures are part of a broader package aimed at preventing child sexual abuse, raising awareness of online risks, facilitating the reporting of crimes, and providing state support, including the right to financial compensation.

Earlier, the UK-based Internet Watch Foundation reported the rapid spread of AI-generated child sexual abuse materials. According to a report by researchers, over 20,254 CSA images were discovered on a darknet forum in just one month.

Back in May 2023, industry pioneer Geoffrey Hinton warned of potential threats associated with AI. Prior to this, former Google CEO Eric Schmidt stated that the technology poses an “existential risk,” which could cause many people to “suffer or die.”