X, owned by Elon Musk, has been in the news over AI-generated illegal content being created and posted on his social media platform X. Specifically, multiple reports have found that Grok, X’s artificial intelligence (AI) chatbot is allowing users to request nonconsensual nude images.
X users have been creating and sharing non-consensual sexual deepfakes of intimate images that include child sexual abuse material (CSAM). In addition to other violations, such fake sexualized images can run afoul of revenge porn laws in the United States and the U.K.’s Online Safety Act. The United Kingdom’s Office of Communications (Ofcom) has launched a formal investigation of Grok and the platform. In response to the anger emanating from both sides of the ocean, X has reportedly limited the ability to create images via Grok AI.
While allegedly making it a highest priority to hold those abusing the Grok AI chatbot responsible, Musk has also accused U.K. Prime Minister Keir Starmer, Ofcom Technology Secretary Liz Kendall, and the British government of seeking “any excuse for censorship” against free speech. Those who found their ordinary pictures transformed into non-consensual intimate images may not consider the investigation, or the potential of a court order having X banned, to be an overreach.
Stranger in the Strangest Land Is More Like It
The AI tool Grok derived its name from a word coined by science-fiction author Robert Heinlein to represent a deeper level of understanding. Like all large language models (LLMs), Grok is designed to gather and analyze as much information as possible, allowing it to continuously learn (although the legality of how LLMs procure data remains under scrutiny). Those accessing Grok can give the AI specific prompts to find answers, generate text, or share a “conversation” with it. It can also provide image generation for almost any request, which led to the issue at hand.
The arrival of the new year saw an increase in requests by X users to have Grok create images of women in bikinis, but things quickly took a dark turn. Users began to demand that Grok alter existing images of people into skimpier clothing, make them appear to be completely nude, or put them into sexually suggestive and explicit positions and situations. Some asked Grok to add bruises, blood, and forced smiles to the modified likenesses.
Perhaps even more troubling was Grok’s willingness to create output containing sexualized images of children containing CSAM or pictures of Renee Nicole Good, an unarmed woman shot and killed by a United States Immigration and Customs agent in Minnesota last week, with bullet holes added to her face. Some of the LLM’s creations face receiving a legal classification as pornography.
Using generative AI or nudify apps to create nonconsensual sexualized images violates federal law.
The Images Are Coming From Inside the House!
Under its Safety account, X posted that using Grok to create CSAM or any other type of illegal content would incur the same penalties as the uploading of any type of forbidden material. This includes non-consensual nudity, the personal data of other people, and anything that might compromise an account’s security. X also limited the image-generating ability of Grok for non-paying users. Since paying customers are required to provide a credit card, this would presumably allow the site to positively identify those violating X’s Terms of Service (TOS).
Detractors note that the Grok app, which is separate from X, allows anonymous users but did not receive the same restrictions. CSAM and revenge porn images can be created and posted through other internet service providers without being subject to any X-imposed sanctions.
Acting as the U.K.’s media regulator, Ofcom’s investigation could lead to a lot of trouble for Musk, Grok, and X. If Grok’s creations are deemed pornography or CSAM, penalties could include fines of either 10% of X’s worldwide revenue or £18 million, whichever is greater. For continuing noncompliance, Ofcom can attempt to obtain a court order resulting in the removal of advertisers from X’s platform, the exclusion of payment providers, or even the blocking of the site itself. While similar penalties might be available in the U.S. as well, the current administration has a much warmer relationship with Musk.
Related Resources
- Revenge Porn Website Sued for Privacy Violations (FindLaw’s Legally Weird)
- Can Elon Musk Afford To Break the Law? (FindLaw’s Legally Weird)
- Are There Illegal Internet Search Terms? (FindLaw’s Law and Daily Life)
