Elon Musk's online network X has caused major outrage after individuals used its AI chatbot Grok to alter photos of women by removing their attire.
Multiple examples have been seen where the tool has been employed to remove clothing from women to show them in bikinis without consent, as well as to place them in sexualized situations.
Company Response and Legal Stance
XAI, the organization that created Grok, did not respond to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw nudification tools. Under a new criminal offence, anyone who supplied such technology would "receive a prison sentence and heavy fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their services, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium features—which answers X users' queries when they tag it in a post.
It is frequently used to give reactions or more context to other posters' comments, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification feature.
History of Criticism and Policy
The tool has been faulted for permitting users to generate images and footage with nudity and sexually suggestive material, and it was previously accused of creating graphic material of celebrities.
"X has been allowing the creation and distribution of these images for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "depicting likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were required to take "suitable measures" to "lessen the risk" of users encountering illegal content on their services and to delete it swiftly when they learn of it.
