At a Glance
- Elon Musk claims he is “not aware of any naked underage images generated by Grok” hours before California’s attorney general opened an investigation.
- Copyleaks estimates one sexualized image generated by Grok appears on X every minute; a 24-hour sample tallied 6,700 per hour.
- Governments in the U.K., EU, India, Indonesia, and Malaysia have all moved to block or investigate the chatbot over nonconsensual explicit imagery.
- Why it matters: The probe tests whether AI firms can be held liable when users weaponize image tools to create deepfake pornography of real women and children.
California’s attorney general has launched an investigation into xAI’s Grok chatbot after users flooded X with AI-altered sexual images of real women and minors. The action came just hours after Elon Musk insisted he had seen “literally zero” naked underage images produced by the service.
Global Pressure Builds
The California inquiry adds to a growing list of government responses. Indonesia and Malaysia have temporarily blocked access to Grok. India has demanded immediate technical fixes. The European Commission has ordered xAI to preserve all Grok documents, and the U.K.’s Ofcom has opened a formal investigation under the Online Safety Act.
Copyleaks, an AI-detection firm, told News Of Philadelphia that roughly one sexualized image generated by Grok is posted on X every minute. A separate sample captured 6,700 images per hour during a 24-hour window in early January.
What Grok Was Asked to Do
The trend began late last year when adult-content creators used Grok to generate sexualized images of themselves as marketing. Other users quickly copied the tactic, targeting everyday women and celebrities. In public examples, Grok altered photos of “Stranger Things” actress Millie Bobby Brown by changing clothing, body positioning, or physical features in overtly sexual ways.
Legal Landscape
California Attorney General Rob Bonta said the material “has been used to harass people across the internet” and urged xAI to act immediately. His office will determine whether the company broke state laws.
Federal and state statutes already criminalize nonconsensual intimate imagery and child-sexual-abuse material (CSAM). The Take It Down Act, signed in 2024, mandates that platforms remove such content within 48 hours. California also passed a package of bills last year targeting sexually explicit deepfakes.
Musk’s Response
On Wednesday, Musk wrote: “I am not aware of any naked underage images generated by Grok. Literally zero.” The statement does not deny the existence of bikini shots or other sexualized edits.
He added: “Obviously, Grok does not spontaneously generate images. It does so only according to user request. When asked to generate images, it will refuse to produce anything illegal… If adversarial hacking does something unexpected, we fix the bug immediately.”
xAI has not issued a separate public comment. Sarah L. Montgomery asked the company how many nonconsensual images it has detected, what guardrails have changed, and whether regulators were notified. News Of Philadelphia will update if xAI responds.
Limited Safeguards
Some reports indicate xAI has begun rolling out restrictions. Grok now requires a premium subscription for certain image prompts and may still refuse the request. Copyleaks VP April Kozen told News Of Philadelphia that when Grok does comply, the result is often “more generic or toned-down.” She added that the system appears more permissive with adult-content creators.
“Overall, these behaviors suggest X is experimenting with multiple mechanisms to reduce or control problematic image generation, though inconsistencies remain,” Kozen said.
Prior Incidents
Grok has faced criticism before. The chatbot includes a “spicy mode” meant for explicit content. An October update made jailbreaks easier, leading users to create hardcore pornography and graphic violent imagery. Many of the pornographic outputs depict AI-generated people rather than real individuals.
Copyleaks co-founder Alon Yamin warned: “When AI systems allow the manipulation of real people’s images without clear consent, the impact can be immediate and deeply personal. From Sora to Grok, we are seeing a rapid rise in AI capabilities for manipulated media. Detection and governance are needed now more than ever.”
Key Takeaways
- California’s investigation could set a precedent for holding AI firms accountable for user-generated abuse.
- One image per minute and 6,700 per hour illustrate the scale of the problem on X.
- Musk’s narrow denial focuses on CSAM, where penalties are steepest, rather than the broader creation of nonconsensual sexual imagery.
- Regulators across five jurisdictions have already imposed blocks, demands, or investigations, signaling worldwide concern over AI-generated deepfake pornography.
