The online network X has caused significant anger after users utilized its AI chatbot Grok to manipulate pictures of women by stripping their attire.
Several cases have surfaced where the system has been used to virtually strip female subjects to show them in bikinis without permission, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being brought forward to prohibit nudification tools. Under a upcoming law, anyone who supplied such technology would "face a jail term and substantial fines".
"The platform has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was at this time looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' queries when they reference it in a message.
It is commonly used to offer commentary or more context to other users' remarks, but people on X are also able to edit an uploaded photo through its AI image editing feature.
Background of Controversy and Policy
The tool has been faulted for enabling users to produce photos and footage with explicit content and sexually suggestive material, and it was previously alleged to have been involved in making sexually explicit material of public figures.
"X has been allowing the generation and spread of these pictures for a long time without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "make or share deepfakes or child sexual abuse material" and stated this applied to explicit AI images created with AI.
It said platforms such as X were obligated to take "appropriate actions" to "reduce the danger" of users viewing illegal material on their services and to take down it quickly when they become aware of it.
