In the United States, three Democratic senators asked Apple and Google to remove both the X social media app and the Grok app from their respective stores until xAI implements stronger safeguards against the generation of nonconsensual explicit images and deepfakes involving child sexual abuse. Several advocacy organizations, including UltraViolet, the National Organization for Women and ParentsTogether Action, issued similar requests. Apple and Google have not commented publicly on the matter.
The policy debate gained momentum on Capitol Hill this week when the Senate approved the DEFIANCE Act, legislation that would give individuals portrayed in explicit deepfake images the right to sue platforms or developers that create or distribute such content. The measure, which previously passed the Senate in 2024, awaits consideration in the House of Representatives. Details of the bill are available on the U.S. Congress website.
Elon Musk responded to the mounting scrutiny in a post on X, stating that he was unaware of any instances where Grok generated nude images of minors. He attributed any prohibited content that may have appeared on the platform to specific user requests and a possible software bug. Musk had previously announced that users who attempt to create illegal images through Grok would face the same penalties as those who directly upload such material to X.
xAI has taken interim steps by restricting certain image-generation and editing functions to paying subscribers. According to a recent post by Musk, the service is intended to allow depictions of adult, fictional upper-body nudity at a level comparable to that permitted in R-rated films on mainstream streaming platforms. He added that content policies could vary by region to comply with local regulations.
The investigation comes at a critical moment for xAI. The company recently closed a $20 billion growth funding round that drew participation from Nvidia, Cisco Investments, Valor Equity Partners, Stepstone Group, Fidelity, the Qatar Investment Authority, Abu Dhabi’s MGX and Baron Capital Group. The influx of capital is earmarked for building data centers in and around Memphis, Tennessee, and for expanding the firm’s research and engineering teams.
Imagem: Internet
xAI is incorporated in Nevada and maintains its headquarters in Palo Alto, California, adjacent to an engineering office for Tesla, Musk’s automotive company. Tesla has already integrated Grok into its in-vehicle infotainment systems, broadening the reach of the chatbot beyond smartphones and desktop browsers.
The California Department of Justice has not provided a timeline for the inquiry. Potential outcomes include fines, mandated policy changes or other enforcement actions aimed at curbing the distribution of illicit deepfake content. The agency also indicated that it is coordinating with federal and international counterparts that have initiated similar reviews.
Company executives, including Musk and xAI board member Jared Birchall, have not disclosed additional information regarding technical safeguards or future policy changes. xAI did not respond to requests for comment on the state investigation or the reported incidents involving minors.
As regulatory pressure intensifies, industry analysts are watching to see whether app store operators will take independent action or await the results of government probes. For now, Grok remains available to users in the United States, subject to the new feature limits and the company’s evolving content policies.
Crédito da imagem: Cheng Xin | Getty Images
