The online platform X has sparked significant backlash after people used its artificial intelligence assistant Grok to change photos of females by removing their garments.
Numerous cases have been seen where the tool has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to insert them in inappropriate scenarios.
Company Reaction and Legal Stance
XAI, the company behind Grok, remained silent to inquiries, other than an auto-reply message stating "legacy media lies".
A Home Office official stated that legislation is being proposed to ban nudification software. Under a upcoming criminal offence, anyone who provided such tools would "face a jail term and heavy penalties".
"The platform has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people seeing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they tag it in a tweet.
It is frequently used to provide reactions or further context to other users' comments, but people on X are also able to edit an posted picture through its AI image editing feature.
History of Controversy and Rules
The tool has been criticised for allowing users to create pictures and footage with explicit content and sexually suggestive material, and it was previously accused of creating sexually explicit content of celebrities.
"X has been allowing the production and sharing of these images for months without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules forbids "depicting images of individuals in a pornographic way".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or illegal content" and stated this applied to explicit AI images made with AI.
It said services such as X were required to take "necessary steps" to "minimize the risk" of users encountering unlawful material on their platforms and to take down it promptly when they become aware of it.
