🔗 Share this article

The online platform X has caused major backlash after users employed its AI chatbot Grok to manipulate photos of women by removing their attire.

Numerous instances have emerged where the technology has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to put them in sexualized scenarios.

Company Reaction and Legal Stance

XAI, the company that created Grok, remained silent to inquiries, except for an standard message stating "legacy media lies".

A government official stated that new laws is being introduced to ban nudification software. Under a upcoming law, anyone who supplied such tools would "be subject to a prison sentence and substantial penalties".

"The platform could prevent these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."

Regulator Position and Platform Functionality

The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently looking into X or Grok in regarding artificial intelligence pictures.

Grok is a free AI assistant—with some paid-for features—which answers X users' queries when they tag it in a tweet.

It is commonly used to offer commentary or additional background to other users' comments, but people on X are also able to edit an posted photo through its artificial intelligence picture modification function.

Background of Controversy and Rules

The chatbot has been faulted for enabling users to produce photos and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in generating graphic material of celebrities.

"The platform has been enabling the generation and spread of these pictures for months without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.

XAI's own acceptable use policy prohibits "showing likenesses of individuals in a graphic manner".

In a comment, Ofcom said it was against the law to "generate or circulate private pictures or illegal content" and clarified this included sexual deepfakes created with AI.

It said networks such as X were obligated to take "appropriate actions" to "minimize the danger" of users coming across unlawful content on their services and to delete it swiftly when they are notified of it.