Elon Musk's online network X has provoked widespread outrage after users employed its artificial intelligence assistant Grok to change pictures of females by removing their attire.
Numerous cases have surfaced where the tool has been employed to remove clothing from female subjects to show them in bikinis non-consensually, as well as to put them in sexually explicit contexts.
Developer Response and Regulatory Stance
XAI, the company that created Grok, did not respond to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that new laws is being proposed to prohibit nudification software. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and significant fines".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some additional functions—which responds to X users' prompts when they mention it in a post.
It is often used to offer commentary or additional context to other users' remarks, but people on X are also can modify an posted image through its AI image editing function.
Background of Controversy and Rules
The system has been condemned for permitting users to generate images and videos with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"The platform has been enabling the generation and spread of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "portraying images of people in a graphic way".
In a statement, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and stated this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users coming across illegal material on their services and to delete it swiftly when they are notified of it.
