The social media platform X has sparked major backlash after individuals used its AI chatbot Grok to change photos of women by removing their attire.
Multiple instances have emerged where the system has been used to digitally undress female subjects to render them in swimwear non-consensually, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the firm that created Grok, offered no comment to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government official stated that legislation is being introduced to ban nudification tools. Under a new law, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"The platform could prevent these types of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their services, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some premium functions—which reacts to X users' requests when they mention it in a post.
It is frequently used to provide commentary or more background to other posters' remarks, but people on X are also can modify an uploaded photo through its AI image editing feature.
History of Controversy and Rules
The system has been condemned for enabling users to produce images and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in making graphic content of public figures.
"X has been allowing the generation and distribution of these pictures for months without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own usage rules bans "depicting images of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "make or share non-consensual intimate images or illegal content" and stated this included explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate measures" to "lessen the danger" of users coming across illegal content on their platforms and to delete it quickly when they become aware of it.
