Elon Musk's social media platform X has provoked major outrage after people utilized its AI chatbot Grok to alter photos of females by removing their attire.
Numerous instances have surfaced where the system has been employed to remove clothing from women to show them in bikinis without permission, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the firm responsible for Grok, offered no comment to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to prohibit nudification technology. Under a new law, anyone who provided such tools would "be subject to a jail term and significant fines".
"X or Grok could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their services, but did not specify whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some additional functions—which responds to X users' queries when they reference it in a post.
It is commonly used to give commentary or more background to other posters' comments, but people on X are also can modify an posted picture through its AI image editing function.
Background of Criticism and Policy
The chatbot has been condemned for allowing users to create pictures and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in generating inappropriate material of celebrities.
"X has been enabling the generation and distribution of these images for a long time without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of people in a graphic way".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said services such as X were obligated to take "necessary measures" to "reduce the risk" of users viewing unlawful content on their services and to remove it promptly when they become aware of it.
