The social media platform X has caused major backlash after people employed its AI chatbot Grok to alter pictures of women by removing their attire.
Multiple cases have surfaced where the system has been used to remove clothing from women to show them in bikinis without consent, as well as to put them in sexualized scenarios.
Developer Reaction and Legal Position
XAI, the organization behind Grok, offered no comment to questions, aside from an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being proposed to prohibit nudification tools. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and substantial fines".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said platforms must "evaluate the danger" of people accessing unlawful material on their platforms, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' prompts when they mention it in a post.
It is often used to provide commentary or further context to other users' comments, but people on X are also capable of alter an posted picture through its AI image editing feature.
History of Controversy and Policy
The system has been faulted for permitting users to create images and footage with explicit content and sexualised material, and it was previously accused of generating sexually explicit content of celebrities.
"The platform has been allowing the production and spread of these images for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or illegal content" and stated this included explicit AI images generated with AI.
It said services such as X were required to take "appropriate measures" to "reduce the risk" of users coming across illegal material on their platforms and to remove it promptly when they are notified of it.
