zoltner

Monthly summary

January 2026 summary: AI and child safety

Grok’s sexualized images of women and children set off the broadest regulatory response yet to a single generative AI product in January. Regulators in the UK, EU, France, India, Malaysia, Indonesia and California opened investigations or restricted access, while the UK House of Lords voted for an under-16 social media ban and France’s National Assembly backed an under-15 ban.

Regulators converged on Grok

Grok said on 2 January that lapses in safeguards had led it to generate images of minors in minimal clothing on X. India’s IT ministry gave X 72 hours to submit an action-taken report and warned it could lose its liability exemption. France and Malaysia opened probes, and Malaysia and Indonesia restricted access to Grok. X limited Grok’s image editing to paying subscribers on 9 January, and said on 14 January it would stop Grok undressing images of real people where that is illegal. The allegations are unproven.

Ofcom opened a formal Online Safety Act investigation into X on 12 January, California’s attorney general sent xAI a cease and desist letter on 16 January, and the European Commission opened a Digital Services Act investigation on 26 January that could bring a fine of up to 6% of X’s global annual turnover. The Commission also ordered X to keep all Grok records until the end of 2026, South Korea began a privacy review, and the Philippines was reported to be lifting its block after xAI promised safeguards. The allegations are unproven.

A WIRED review of Grok’s website found graphic sexual content, including content that appears to involve minors. The Internet Watch Foundation said its analysts found criminal imagery of girls aged 11 to 13 that appeared to have been made with Grok Imagine on a dark web forum. An analysis estimated Grok generated about three million sexualized images in 11 days, including around 23,000 appearing to depict children. The allegations are unproven.

Age limits moved toward law

Prime Minister Keir Starmer told MPs on 13 January that he is open to an Australian-style ban. The House of Lords voted 261 to 150 on 21 January to back an under-16 ban through an amendment to the schools bill, and the government said it would try to overturn it in the Commons while it runs a three-month consultation.

France’s National Assembly approved a bill on 26 January to bar children under 15 from social media, and President Emmanuel Macron said he wanted it in place before September. Turkey’s ruling party was drafting an under-15 ban, Canada’s federal officials were reported to be drafting a ban for under-14s, and India’s chief economic adviser said the government should consider age-based limits. Finland’s prime minister said he supported an under-15 ban.

In Australia, Meta said it removed 544,052 accounts it believed belonged to under-16s in the first week of the ban, and the eSafety Commissioner said platforms had come “kicking and screaming” to the law. Malaysia’s Online Safety Act duties for platforms with at least eight million users took effect on 1 January. The Global Online Safety Regulators Network, which includes Ofcom, set out four principles on age assurance.

Chatbots, toys and platforms

Character.AI and Google settled several lawsuits by parents of teens who died by suicide after long chatbot conversations, and Kentucky’s attorney general sued Character Technologies. OpenAI began rolling out age prediction on ChatGPT globally, and Meta temporarily cut teens’ access to its AI characters. The allegations are unproven.

At a Senate Commerce Committee hearing on 15 January, several witnesses said AI companions posed a greater threat to children’s emotional development than social media. California state senator Steve Padilla proposed a four-year moratorium on AI chatbot toys, Common Sense Media found 27% of replies from three AI toys unsuitable for children, and researchers found that the toy maker Bondu had left more than 50,000 children’s chat logs reachable through a web portal. The US Senate passed the DEFIANCE Act, letting victims of sexual deepfakes sue their creators.

Roblox began requiring facial age checks before users can access chat, and the Dutch consumer regulator opened a Digital Services Act investigation into Roblox on 30 January. Snap settled a social media addiction lawsuit days before the first major trial. The allegations are unproven.

Evidence and abuse material

The Internet Watch Foundation said 2025 was its worst year on record, with 312,030 confirmed reports and 3,440 AI-generated child sexual abuse videos, up from 13 in 2024. A federal grand jury in the Houston area indicted a man who allegedly used AI apps to create abuse material from photos of victims he knew, and a former Texas teacher pleaded guilty to possessing more than 365,000 images, some AI-generated from photos of students. A report by Resolver and the Molly Rose Foundation described The Com, a network that targets children with histories of self-harm. The allegations are unproven.

What to watch

  • The outcome of the Ofcom, EU, French and Indian investigations into X and Grok.
  • Whether the UK government accepts or overturns the Lords’ under-16 ban in the Commons.
  • France’s under-15 bill in the Senate, aimed at a September start.
  • The first major social media addiction trials, set for the coming weeks.
  • State bills on companion chatbots in Washington, Oregon and elsewhere.
Sources (44)
  1. Grok said safeguard lapses led it to generate images of minors in minimal clothing The Guardian
  2. Ofcom opened a formal investigation into X over Grok sexualized imagery on January 12 Ofcom
  3. European Commission opened an investigation into X over Grok sexual deepfakes on January 26 BBC News
  4. House of Lords voted to back a UK social media ban for under-16s on January 21 BBC News
  5. France’s National Assembly backs under-15 social media ban lemonde.fr
  6. India ordered X to review Grok over sexually explicit images of women and children Business Standard
  7. France and Malaysia opened probes into Grok sexualized deepfakes TechCrunch
  8. Indonesia and Malaysia restrict access to Grok AI over content safeguards bbc.com
  9. X limited Grok image editing to paying users after deepfake backlash bbc.com
  10. X said it would block Grok from undressing images of real people where illegal BBC News
  11. California attorney general ordered xAI to stop creating sexualized images of children Axios
  12. EU instructs X to keep all Grok chatbot records euractiv.com
  13. South Korea opened a preliminary privacy review of Grok deepfakes The Korea Times
  14. Philippines to lift its ban on Grok after xAI vowed safeguards, Bloomberg reported Bloomberg
  15. Wired review finds Grok’s website producing graphic sexual content including apparent minors WIRED
  16. IWF found criminal imagery of children apparently made with Grok Sky News
  17. Analysis reveals Grok generated 3 million sexualized images counterharm.com
  18. Keir Starmer told MPs on January 13 he is open to an Australian-style social media ban The Guardian
  19. UK government opened a consultation on children’s social media use, including a possible ban GOV.UK
  20. Macron said France would fast-track an under-15 social media ban CNN
  21. Turkey moved to draft a ban on social media access for children under 15 Turkish Minute
  22. Federal officials draft plans to ban social media for children under 14 theglobeandmail.com
  23. India’s chief economic adviser said government should consider age-based limits on social media Reuters
  24. Finland weighed an under-15 social media ban with strong public support Yle News
  25. Meta removed 544,052 teen accounts in Australia after the under-16 ban AAP
  26. Australia’s eSafety Commissioner said platforms accepted the under-16 ban ’kicking and screaming’ BBC News
  27. Malaysia’s Online Safety Act duties for large social media platforms took effect January 1 South China Morning Post
  28. Ofcom and international regulators set out expectations on age assurance to protect children Ofcom
  29. Character.AI and Google settled lawsuits over teen suicides mashable.com
  30. Kentucky attorney general sues AI company over child safety concerns kykernel.com
  31. OpenAI rolled out age prediction on ChatGPT globally Reuters
  32. No more AI friends: Meta cuts off teen access to characters on Instagram and Facebook livemint.com
  33. Senate witnesses said AI companions threaten children’s emotional development more than social media U.S. Senate Commerce Committee
  34. California senator proposed a four-year ban on AI chatbot toys for children TechCrunch
  35. Common Sense Media found AI toy companions posed risks to children The Verge
  36. Critical AI toy security failure exposes children’s data uk.pcmag.com
  37. US Senate passed the DEFIANCE Act letting deepfake victims sue creators of explicit images The Verge
  38. Roblox rolls out facial age checks for chat about.roblox.com
  39. Roblox under scrutiny as Dutch regulators launch probe over child safety concerns thenews.com.pk
  40. Snapchat settles social media addiction lawsuit as landmark trial proceeds theguardian.com
  41. IWF recorded 3,440 AI child sexual abuse videos in 2025, up from 13 Internet Watch Foundation
  42. Anyone can become a victim: Man indicted in Houston area first federal AI child exploitation case click2houston.com
  43. Former Texas teacher accused of using AI to make explicit images of students pleads guilty to possession foxla.com
  44. Resolver and Molly Rose Foundation report describes an online network targeting vulnerable children Resolver

Every story recorded for January 2026 is in the news archive. You can also search all stories.

John Zoltner is the founder and CEO of AIChildSafety.org. About John