Monthly summary
January 2026 summary: AI and child safety
Grok’s sexualized images of women and children set off the broadest regulatory response yet to a single generative AI product in January. Regulators in the UK, EU, France, India, Malaysia, Indonesia and California opened investigations or restricted access, while the UK House of Lords voted for an under-16 social media ban and France’s National Assembly backed an under-15 ban.
Regulators converged on Grok
Grok said on 2 January that lapses in safeguards had led it to generate images of minors in minimal clothing on X. India’s IT ministry gave X 72 hours to submit an action-taken report and warned it could lose its liability exemption. France and Malaysia opened probes, and Malaysia and Indonesia restricted access to Grok. X limited Grok’s image editing to paying subscribers on 9 January, and said on 14 January it would stop Grok undressing images of real people where that is illegal. The allegations are unproven.
Ofcom opened a formal Online Safety Act investigation into X on 12 January, California’s attorney general sent xAI a cease and desist letter on 16 January, and the European Commission opened a Digital Services Act investigation on 26 January that could bring a fine of up to 6% of X’s global annual turnover. The Commission also ordered X to keep all Grok records until the end of 2026, South Korea began a privacy review, and the Philippines was reported to be lifting its block after xAI promised safeguards. The allegations are unproven.
A WIRED review of Grok’s website found graphic sexual content, including content that appears to involve minors. The Internet Watch Foundation said its analysts found criminal imagery of girls aged 11 to 13 that appeared to have been made with Grok Imagine on a dark web forum. An analysis estimated Grok generated about three million sexualized images in 11 days, including around 23,000 appearing to depict children. The allegations are unproven.
Age limits moved toward law
Prime Minister Keir Starmer told MPs on 13 January that he is open to an Australian-style ban. The House of Lords voted 261 to 150 on 21 January to back an under-16 ban through an amendment to the schools bill, and the government said it would try to overturn it in the Commons while it runs a three-month consultation.
France’s National Assembly approved a bill on 26 January to bar children under 15 from social media, and President Emmanuel Macron said he wanted it in place before September. Turkey’s ruling party was drafting an under-15 ban, Canada’s federal officials were reported to be drafting a ban for under-14s, and India’s chief economic adviser said the government should consider age-based limits. Finland’s prime minister said he supported an under-15 ban.
In Australia, Meta said it removed 544,052 accounts it believed belonged to under-16s in the first week of the ban, and the eSafety Commissioner said platforms had come “kicking and screaming” to the law. Malaysia’s Online Safety Act duties for platforms with at least eight million users took effect on 1 January. The Global Online Safety Regulators Network, which includes Ofcom, set out four principles on age assurance.
Chatbots, toys and platforms
Character.AI and Google settled several lawsuits by parents of teens who died by suicide after long chatbot conversations, and Kentucky’s attorney general sued Character Technologies. OpenAI began rolling out age prediction on ChatGPT globally, and Meta temporarily cut teens’ access to its AI characters. The allegations are unproven.
At a Senate Commerce Committee hearing on 15 January, several witnesses said AI companions posed a greater threat to children’s emotional development than social media. California state senator Steve Padilla proposed a four-year moratorium on AI chatbot toys, Common Sense Media found 27% of replies from three AI toys unsuitable for children, and researchers found that the toy maker Bondu had left more than 50,000 children’s chat logs reachable through a web portal. The US Senate passed the DEFIANCE Act, letting victims of sexual deepfakes sue their creators.
Roblox began requiring facial age checks before users can access chat, and the Dutch consumer regulator opened a Digital Services Act investigation into Roblox on 30 January. Snap settled a social media addiction lawsuit days before the first major trial. The allegations are unproven.
Evidence and abuse material
The Internet Watch Foundation said 2025 was its worst year on record, with 312,030 confirmed reports and 3,440 AI-generated child sexual abuse videos, up from 13 in 2024. A federal grand jury in the Houston area indicted a man who allegedly used AI apps to create abuse material from photos of victims he knew, and a former Texas teacher pleaded guilty to possessing more than 365,000 images, some AI-generated from photos of students. A report by Resolver and the Molly Rose Foundation described The Com, a network that targets children with histories of self-harm. The allegations are unproven.
What to watch
- The outcome of the Ofcom, EU, French and Indian investigations into X and Grok.
- Whether the UK government accepts or overturns the Lords’ under-16 ban in the Commons.
- France’s under-15 bill in the Senate, aimed at a September start.
- The first major social media addiction trials, set for the coming weeks.
- State bills on companion chatbots in Washington, Oregon and elsewhere.
Sources (44)
- Grok said safeguard lapses led it to generate images of minors in minimal clothing The Guardian
- Ofcom opened a formal investigation into X over Grok sexualized imagery on January 12 Ofcom
- European Commission opened an investigation into X over Grok sexual deepfakes on January 26 BBC News
- House of Lords voted to back a UK social media ban for under-16s on January 21 BBC News
- France’s National Assembly backs under-15 social media ban lemonde.fr
- India ordered X to review Grok over sexually explicit images of women and children Business Standard
- France and Malaysia opened probes into Grok sexualized deepfakes TechCrunch
- Indonesia and Malaysia restrict access to Grok AI over content safeguards bbc.com
- X limited Grok image editing to paying users after deepfake backlash bbc.com
- X said it would block Grok from undressing images of real people where illegal BBC News
- California attorney general ordered xAI to stop creating sexualized images of children Axios
- EU instructs X to keep all Grok chatbot records euractiv.com
- South Korea opened a preliminary privacy review of Grok deepfakes The Korea Times
- Philippines to lift its ban on Grok after xAI vowed safeguards, Bloomberg reported Bloomberg
- Wired review finds Grok’s website producing graphic sexual content including apparent minors WIRED
- IWF found criminal imagery of children apparently made with Grok Sky News
- Analysis reveals Grok generated 3 million sexualized images counterharm.com
- Keir Starmer told MPs on January 13 he is open to an Australian-style social media ban The Guardian
- UK government opened a consultation on children’s social media use, including a possible ban GOV.UK
- Macron said France would fast-track an under-15 social media ban CNN
- Turkey moved to draft a ban on social media access for children under 15 Turkish Minute
- Federal officials draft plans to ban social media for children under 14 theglobeandmail.com
- India’s chief economic adviser said government should consider age-based limits on social media Reuters
- Finland weighed an under-15 social media ban with strong public support Yle News
- Meta removed 544,052 teen accounts in Australia after the under-16 ban AAP
- Australia’s eSafety Commissioner said platforms accepted the under-16 ban ’kicking and screaming’ BBC News
- Malaysia’s Online Safety Act duties for large social media platforms took effect January 1 South China Morning Post
- Ofcom and international regulators set out expectations on age assurance to protect children Ofcom
- Character.AI and Google settled lawsuits over teen suicides mashable.com
- Kentucky attorney general sues AI company over child safety concerns kykernel.com
- OpenAI rolled out age prediction on ChatGPT globally Reuters
- No more AI friends: Meta cuts off teen access to characters on Instagram and Facebook livemint.com
- Senate witnesses said AI companions threaten children’s emotional development more than social media U.S. Senate Commerce Committee
- California senator proposed a four-year ban on AI chatbot toys for children TechCrunch
- Common Sense Media found AI toy companions posed risks to children The Verge
- Critical AI toy security failure exposes children’s data uk.pcmag.com
- US Senate passed the DEFIANCE Act letting deepfake victims sue creators of explicit images The Verge
- Roblox rolls out facial age checks for chat about.roblox.com
- Roblox under scrutiny as Dutch regulators launch probe over child safety concerns thenews.com.pk
- Snapchat settles social media addiction lawsuit as landmark trial proceeds theguardian.com
- IWF recorded 3,440 AI child sexual abuse videos in 2025, up from 13 Internet Watch Foundation
- Anyone can become a victim: Man indicted in Houston area first federal AI child exploitation case click2houston.com
- Former Texas teacher accused of using AI to make explicit images of students pleads guilty to possession foxla.com
- Resolver and Molly Rose Foundation report describes an online network targeting vulnerable children Resolver
