Grok, an AI product launched by Elon Musk’s xAI in late December 2025, has faced intense regulatory scrutiny due to its capability to generate images, including sexualized content. This scrutiny intensified following a recent court ruling in the Netherlands, which has set a precedent for AI accountability in the creation of non-consensual images.
On March 26, 2026, a Dutch court ordered xAI to cease the generation and distribution of nude images without consent. The court’s decision came after a case brought forth by the organizations Offlimits and Fonds Slachtofferhulp, highlighting the urgent need for regulations surrounding AI-generated content.
The court found that xAI had not implemented sufficient measures to prevent the creation of non-consensual content, which included approximately 3 million sexualized images generated by Grok between December 29, 2025, and January 8, 2026. This alarming figure included 23,000 images that appeared to depict children, raising serious ethical concerns.
In its ruling, the court imposed fines of 100,000 euros per day for noncompliance, emphasizing that the responsibility lies with the company to ensure its tools are not misused. Robbert Hoving, a representative involved in the case, stated, “The burden is on the company to make sure its tools are not used to create and distribute nonconsensual sexual images, including of children.”
The court also rejected xAI’s argument that liability should rest with users issuing prompts to the AI, stating that the company must take proactive steps to prevent misuse. Karlijn Han, another key figure in the case, remarked, “Grok and X are ordered to cease offering the undressing functionality that makes it possible to undress Dutch people.”
This ruling is significant as it marks one of the first instances where a court has addressed the responsibility of AI in creating sexualized images, particularly in violation of the General Data Protection Regulation (GDPR).
In response to the court’s decision, xAI must confirm in writing to Offlimits how it has complied with the order. The injunction applies not only to xAI but also to its EU-facing entity, XIUC, indicating a broader impact on the company’s operations in Europe.
As the European Parliament has also approved a ban on AI systems generating sexualized deepfakes, the implications of this ruling are likely to resonate beyond the Netherlands, potentially influencing future regulations across the EU.
Observers are keenly watching how xAI will adapt its practices in light of this ruling and what measures will be implemented to prevent the generation of non-consensual content in the future. Details remain unconfirmed regarding the specific changes that will be enacted by the company.
