EU Lawmakers Ban Non-Consensual AI Deepfakes
▼ Summary
– The EU’s AI Act will explicitly ban AI-generated non-consensual intimate images, including child sexual abuse material, following a March 11 political agreement on amendments.
– This prohibition was directly prompted by the Grok scandal, where users exploited an xAI chatbot feature to create sexualized deepfakes of real women and girls without consent.
– The European Commission opened a formal investigation into platform X for potential Digital Services Act breaches related to Grok, while national probes were launched in several countries.
– The deal also includes eased compliance rules for AI systems in regulated sectors like medical devices, though the Greens oppose these deregulation elements.
– The incident exposed a legal gap and a structural problem in EU AI regulation, as lawmakers had not anticipated the rapid abuse of public image-generation tools.
A significant political agreement has been reached to explicitly prohibit the creation of non-consensual intimate images using artificial intelligence within the European Union’s regulatory framework. This move directly addresses a dangerous legal gap exposed by a recent high-profile controversy, aiming to protect individuals from a rapidly evolving form of digital abuse. The amendment forms part of broader negotiations to update the bloc’s landmark AI legislation.
European Union lawmakers finalized a deal on March 11th to amend the AI Act, introducing a clear ban on AI-generated sexual deepfakes created without consent. This prohibition explicitly includes material that constitutes child sexual abuse. The agreement, forged between center-right and center-left parliamentarians, also adjusts compliance rules for AI systems within regulated sectors like medical devices. This package of changes, known as the AI Act Omnibus, seeks to refine the 2024 law as it nears full implementation.
The inclusion of this ban was not originally planned. It was propelled onto the agenda by the Grok scandal, which erupted in late 2025. Following an update to Elon Musk’s xAI chatbot, users of the social media platform X quickly exploited a new image-editing feature. They generated thousands of realistic, sexualized images of real women and girls without their permission. Researchers estimated that in just two days, at least 6,700 such images were created, with regulators identifying content depicting minors as illegal child sexual abuse material.
The European Commission reacted swiftly, condemning the “appalling” content and launching a formal investigation into whether X breached the Digital Services Act. This law carries the threat of substantial fines. Under intense scrutiny, xAI attempted to restrict the tool’s capabilities, but researchers found the safeguards could be bypassed. National investigations were launched in several member states, and some countries blocked access to Grok entirely.
A critical turning point came when the Commission confirmed a glaring loophole: existing EU law, including the current text of the AI Act, did not explicitly ban AI systems capable of generating this harmful material. This admission provided the necessary political momentum for change. A coalition led by France and Spain, later joined by Germany and Slovakia, insisted the ban be added to the Omnibus negotiations, threatening to block the entire file without it.
The agreed-upon amendment now proceeds to a committee vote, with further parliamentary and Council stages to follow. While the deepfake ban has broad support, other elements of the package, particularly those deregulating industrial AI, face opposition from groups like the Greens. The final text may still see adjustments.
This episode has highlighted a fundamental flaw in the EU’s regulatory approach. Legislators did not foresee the speed at which powerful, public-facing image-generation tools would emerge or the ease with which they could be weaponized. For Elon Musk and X, the Grok investigation opens a new and contentious front in their already strained relationship with Brussels regulators. With formal probes underway and ongoing transatlantic tech tensions, the friction between the company and EU authorities is set to continue.
(Source: The Next Web)
