At a Glance
- Malaysia and Indonesia have blocked access to Grok AI after it generated non-consensual sexual images of women and children.
- Officials say the tool lacks safeguards to stop users from creating fake pornographic content from real photos.
- The restriction will stay until xAI and X Corp. implement stronger controls.
- Why it matters: The first national bans on Grok signal rising pressure on AI firms to police deepfake abuse.
Indonesia and Malaysia have cut off access to Grok, Elon Musk’s AI chatbot, becoming the first governments to impose such bans after authorities documented repeated misuse that produced sexually explicit and non-consensual images.
What Went Wrong
Launched in 2023 and available free within the X platform, Grok added an image generator last summer. Regulators say the feature, including an optional “spicy mode” that permits adult content, has been exploited to create manipulated photos showing:
- Women in bikinis or sexual poses
- Children in sexualized scenes
- Deepfakes based on real photos of local residents
Indonesia’s Ministry of Communication and Digital Affairs said initial findings showed the system “lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents.”
Indonesia Acts First
On Saturday the ministry ordered internet service providers to temporarily block Grok. Minister Meutya Hafid said the government “sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space.”
Alexander Sabar, director general of digital space supervision, warned that manipulating or sharing photos without consent risks “psychological, social and reputational harm” and breaches privacy and image rights.
Malaysia Follows Suit
The Malaysian Communications and Multimedia Commission imposed its own restriction on Sunday, citing “repeated misuse” to generate “obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.”
The regulator said it had issued notices to X Corp. and xAI earlier in the month demanding stronger controls. The companies’ replies relied mainly on user-reporting mechanisms, which the commission deemed insufficient.
> “The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing,” the commission said, adding that access will remain blocked “until effective safeguards are put in place.”
Global Scrutiny Grows
The Southeast Asian bans add to mounting pressure on Grok elsewhere. The European Union, Britain, India and France are all reviewing the service after users produced sexualized deepfakes of public figures and private citizens.
Last week xAI limited image generation and editing to paying subscribers, a move critics say still leaves room for abuse.
Robert K. Lawson reported for News Of Philadelphia that regulators across the region view current controls as inadequate to protect women, children and the broader community from fake pornographic content generated using AI.
Key Takeaways
- First national bans on Grok set a precedent for AI regulation.
- Regulators want built-in safeguards, not just user reporting.
- Restrictions will stay until xAI and X Corp. prove stronger protections are in place.
- Expect more governments to weigh similar blocks as deepfake abuse spreads.
