Meta to let AI judge child safety and misinformation risks

Written by

Published 2 Jun 2025

Fact checked by

We maintain a strict editorial policy dedicated to factual accuracy, relevance, and impartiality. Our content is written and edited by top industry professionals with first-hand experience. The content undergoes thorough review by experienced editors to guarantee and adherence to the highest standards of reporting and publishing.

disclosure

a white and blue square with a blue and white facebook logo

Meta is quietly replacing the human experts who evaluate whether new Facebook and Instagram features could harm users, handing those critical safety decisions to artificial intelligence (AI) instead.

The social media giant plans to automate up to 90% of product risk assessments that have traditionally required human judgment, according to internal documents obtained by NPR. These reviews determine whether new features might violate privacy, endanger children, or spread misinformation before reaching billions of users worldwide.

    Under the old system, teams of specialists would debate the possible consequences of platform changes. Now, product developers simply fill out a form and receive instant AI-generated decisions about risks and safety requirements.

    “Insofar as this process functionally means more stuff launching faster, with less rigorous scrutiny and opposition, it means you’re creating higher risks,” a former Meta executive told NPR. “Negative externalities of product changes are less likely to be prevented before they start causing problems in the world.”

    The shift affects sensitive areas, including youth safety, AI ethics, and what Meta calls “integrity” issues covering violent content and misinformation. Previously, no product updates could reach users without approval from risk assessors.

    Meta claims it still uses “human expertise” for “novel and complex issues” while automating only “low-risk decisions.” However, the company is considering AI reviews even for sensitive youth protection measures and algorithm changes that could affect how information spreads across its platforms.

    The change empowers engineers to make their own safety judgments rather than consulting specialized reviewers. This concerns former employees who worry about putting safety decisions in the hands of those primarily focused on shipping products quickly.

    “Most product managers and engineers are not privacy experts and that is not the focus of their job,” said Zvika Krieger, Meta’s former director of responsible innovation. “In the past, some of these kinds of self-assessments have become box-checking exercises that miss significant risks.”

    Current employees even echo these concerns. “I think it’s fairly irresponsible given the intention of why we exist,” one Meta worker told NPR. “We provide the human perspective of how things can go wrong.”

    The automation rollout began ramping up through April and May, said one current Meta employee. CEO Mark Zuckerberg has pushed for AI to handle more tasks across Meta, recently stating that AI should write “most of Meta’s code” within 18 months.

    Meta’s European operations may retain more human oversight due to stricter Digital Services Act regulations. However, the company is implementing these changes globally as it competes with TikTok, OpenAI, and other rivals.

    The policy shift comes shortly after Meta ended its fact-checking program and loosened hate speech policies, reflecting what critics see as a dismantling of safety guardrails built up over years of regulatory pressure.

    Meta says it has invested billions in user privacy and is auditing automated decisions. But former employees question whether speeding up safety reviews serves users’ best interests, given the intense scrutiny each new Meta product faces.