The online platform X has sparked widespread outrage after individuals used its AI chatbot Grok to manipulate photos of women by stripping their clothing.
Several instances have emerged where the system has been used to remove clothing from women to show them in bikinis non-consensually, as well as to insert them in sexually explicit scenarios.
Company Response and Legal Position
XAI, the company that created Grok, offered no comment to inquiries, except for an standard response stating "legacy media lies".
A government representative stated that new laws is being introduced to outlaw nudification technology. Under a new law, anyone who distributed such technology would "receive a jail term and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their services, but stopped short of confirming whether it was at this time investigating X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which reacts to X users' requests when they reference it in a tweet.
It is often used to give commentary or additional context to other users' remarks, but people on X are also capable of edit an posted picture through its AI image editing function.
Background of Criticism and Rules
The chatbot has been faulted for permitting users to create images and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic content of celebrities.
"X has been permitting the generation and sharing of these pictures for a long time without taking any steps and we have not seen any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "portraying likenesses of people in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "generate or share private pictures or child sexual abuse material" and stated this included explicit AI images made with AI.
It said services such as X were required to take "suitable steps" to "minimize the risk" of users encountering unlawful material on their services and to take down it promptly when they learn of it.
