The social media network X has caused significant anger after people utilized its artificial intelligence assistant Grok to alter photos of women by removing their garments.
Numerous examples have surfaced where the tool has been employed to remove clothing from women to make them appear in bikinis without permission, as well as to place them in inappropriate scenarios.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, did not respond to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification technology. Under a proposed law, anyone who provided such technology would "be subject to a jail term and substantial penalties".
"X or Grok could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which answers X users' queries when they tag it in a post.
It is often used to provide reactions or more context to other posters' comments, but people on X are also able to edit an posted photo through its AI image editing tool.
History of Controversy and Policy
The tool has been condemned for allowing users to create pictures and footage with explicit content and sexualised material, and it was earlier accused of making inappropriate material of public figures.
"X has been allowing the generation and distribution of these images for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting images of individuals in a pornographic fashion".
In a comment, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said platforms such as X were required to take "suitable steps" to "minimize the risk" of users encountering illegal content on their services and to remove it swiftly when they learn of it.
