The online platform X has sparked widespread anger after people used its artificial intelligence assistant Grok to change images of females by stripping their attire.
Numerous instances have surfaced where the system has been employed to remove clothing from women to render them in bikinis without consent, as well as to insert them in sexually explicit situations.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, remained silent to inquiries, other than an auto-reply message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban nudification technology. Under a new law, anyone who distributed such tools would "receive a jail term and heavy penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people accessing illegal content on their services, but did not confirm whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which reacts to X users' requests when they reference it in a tweet.
It is often used to provide reactions or more background to other posters' remarks, but people on X are also can alter an uploaded picture through its AI image editing feature.
History of Criticism and Rules
The tool has been condemned for permitting users to produce photos and footage with nudity and sexualised material, and it was in the past accused of generating inappropriate material of celebrities.
"The platform has been permitting the generation and distribution of these pictures for months without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service prohibits "portraying likenesses of people in a sexually explicit way".
In a statement, Ofcom said it was against the law to "create or circulate deepfakes or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable measures" to "minimize the risk" of users viewing unlawful content on their platforms and to delete it promptly when they are notified of it.
