Elon Musk's online platform X has provoked significant outrage after individuals used its artificial intelligence assistant Grok to change images of females by taking off their garments.
Multiple examples have surfaced where the tool has been used to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in inappropriate situations.
Company Response and Regulatory Stance
XAI, the firm behind Grok, remained silent to questions, other than an standard response stating "legacy media lies".
A Home Office official stated that new laws is being proposed to outlaw nudification software. Under a proposed law, anyone who supplied such technology would "face a prison sentence and heavy penalties".
"The platform is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their services, but did not specify whether it was at this time investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional features—which answers X users' prompts when they mention it in a tweet.
It is frequently used to provide reactions or more background to other posters' remarks, but people on X are also can edit an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been faulted for allowing users to generate photos and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been enabling the generation and sharing of these pictures for a long time without taking any measures and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and confirmed this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "necessary measures" to "reduce the risk" of users encountering illegal material on their platforms and to remove it swiftly when they learn of it.
