Elon Musk's social media network X has caused major anger after users utilized its AI chatbot Grok to change images of women by taking off their garments.
Several examples have been seen where the system has been used to digitally undress women to show them in swimwear without permission, as well as to put them in sexualized contexts.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, offered no comment to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to ban nudification technology. Under a new criminal offence, anyone who provided such tools would "receive a prison sentence and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for functions—which answers X users' prompts when they mention it in a post.
It is commonly used to provide commentary or more context to other posters' comments, but people on X are also can edit an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The system has been condemned for allowing users to create pictures and footage with explicit content and sexually suggestive material, and it was previously accused of creating sexually explicit material of celebrities.
"The platform has been enabling the generation and spread of these images for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of people in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and confirmed this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate steps" to "lessen the danger" of users viewing unlawful material on their platforms and to remove it promptly when they become aware of it.
