AI Content Moderation: Widely Used, Rarely Independently Measured, Weakest in Smaller Languages

The use case

On 19 March 2026, Meta said new AI enforcement systems had caught twice as much adult sexual solicitation content as its human review teams while cutting the rate of mistakes by more than 60% (operator-reported; the announcement does not describe the test design)¹. The claim sits on top of a much older arrangement. Automated classifiers already decide what is left up, taken down or sent to a person for review, and the Oversight Board, which describes itself as independent of Meta, stated in 2024 that most moderation decisions are now made by machines, without giving a figure⁴.

Meta's classifiers weigh the probability of a violation, its severity and how widely the content is spreading when choosing an action⁴. A threshold turns that score into a decision, so the threshold matters as much as the model: it sets the balance between missing harmful content and removing lawful content.

Scope. AI here means machine-learning systems that classify posts, images, video or accounts⁴. Three older approaches do similar work and are not credited as AI in this paper: rule-based tools that flag listed words or block repeat violators⁴; hash-matching, which compares uploads with libraries of content people have already judged and is the usual method for terrorist and copyrighted material⁵; and human reviewers working from user reports. Feed ranking, labelling of AI-generated media and fact-checking are out of scope.

Where it is running

Eight very large platforms filed statements of reasons for moderation decisions in the EU database between 25 September 2023 and 25 January 2024. Researchers analysed 439 million of them and found that the vast majority of decisions were enforced automatically (peer-reviewed analysis of self-reported data)⁵. The count is of automated decisions, not AI decisions, because the category also covers hash-matching and rules.

The analysed platforms also include Pinterest, Snapchat and LinkedIn⁵. Meta's March 2026 announcement says the more advanced systems will be deployed across its apps over the next few years, once they consistently outperform current methods, that reliance on third-party vendors will fall, and that people will keep high-risk decisions such as appeals of account disablement and reports to law enforcement (operator-reported)¹.

Outside the EU and the United States, a peer-reviewed audit published on 8 September 2026 tested hate-speech classifiers on Amharic and Afan Oromo content from Ethiopia⁶. It audits tools rather than a platform deployment. Older deployments exist: classifier-based removal of nudity on Instagram was already running in 2021, when Meta changed it after an Oversight Board case⁴. A peer-reviewed study also cites a 2024 report that TikTok dismissed 300 moderators in Amsterdam, which the authors read as a turn towards automation⁵.

What the evidence shows

No controlled trial of AI moderation against a human-only baseline was found. The evidence below runs from strongest to weakest.

Independent audit. A computational audit of three hate-speech classifiers against 838 hand-annotated posts from 2020 to 2025, with 20 interviews, found that the most widely used generic classifier recovers only about a tenth of hate speech in English translations of Amharic and Afan Oromo posts (independent audit, abstract)⁶. Locally oriented classifiers do far better on Amharic but collapse on Afan Oromo, and the dominant error across every tool is under-detection, not over-removal⁶. The mechanism is language: a generic tool that cannot read the language natively loses context in translation. The study audits three classifiers, not a named platform's system, so it shows a risk rather than a platform error rate.

Independent appeal data. Appeals Centre Europe, a body certified under the Digital Services Act to settle disputes outside court, received 9,822 disputes between November 2024 and August 2025, of which 3,337 were in scope, and made 1,565 decisions from January to August 2025 (dispute-body data)⁷. It overturned the platform in 77% of them⁷. About two-thirds of those overturns were default decisions, issued because the platform did not supply the content: 52% of all decisions were default overturns, 25% were overturns after review and 23% upheld the platform⁷. Where content was reviewed, the overturn rate was 65% for restricted goods and services, 57% for adult nudity and 50% for hate speech, in policy areas with more than 100 reviewed decisions each (dispute-body data, January to August 2025)⁷. A later commentary reports that in the body's most recent report 72% of decisions were procedural and that platforms were overturned in 59% of cases reviewed on the merits, while a second body, User Rights, reported four in five overturned across about 400 merits reviews⁹. These figures cover human and automated decisions together.

Speed. Across the 439 million statements of reasons, more automation correlated with faster enforcement, while manual detection correlated with longer delays: TikTok's median enforcement in France was more than 30 days for manual cases, and YouTube's median could exceed 100 days (observational, self-reported data)⁵. This measures speed, not accuracy.

Operator-reported accuracy. Meta reported that global enforcement precision averaged around 91% on Facebook and around 92% on Instagram at the end of the first half of 2026, generally consistent with its previous report (operator-reported; method not shown in the text read; covers AI and non-AI enforcement together)². Meta's March 2026 early tests also reported finding 5,000 scam attempts per day that no existing review team had caught (operator-reported)¹. A 2023 fix shows how gains were made: after the Board's breast-cancer awareness case, Meta added text-based context signals to the nudity classifier, and in the 30 days to 27 March 2023 this sent 2,500 more items to human review instead of removal (operator-reported via the Board)⁴. The gain came from better signals and routing to people, not from removing people.

What it takes: cost, integration, governance

Cost. No public cost per decision was found. The authors of a peer-reviewed study state that automation reduced moderation cost compared with hiring reviewers, without giving a figure⁵. Meta says reliance on third-party vendors will fall as the new systems expand¹.

People. Self-reported figures for April to September 2023 show 1,362 human moderators across Facebook and Instagram for 259 million average monthly EU recipients, and 5,827 at TikTok for 125 million (self-reported, non-standardised)⁵. That is about 190,000 recipients per moderator at Meta and about 21,000 at TikTok. Human coverage is uneven by language: TikTok reported 2,137 English-speaking reviewers and 69 for Bulgarian⁵.

Integration. The Board notes that tuning classifiers to context takes oversight and resources⁴. Appeals also depend on plumbing: Meta lets users generate a reference number that helps the dispute body find the content automatically, while TikTok provided none, so locating content was manual⁷.

Rules. Under the Digital Services Act, statements of reasons must state any use of automated means, decisions go into a Commission database, and fines can reach 6% of global turnover⁵. Out-of-court dispute bodies are free to users and paid by platforms⁹. The Board recommends accuracy audits of automated decisions, third-party data access and performance testing for vulnerable groups and languages⁴.

Where it falls short

The main claims weaken when tested against their design.

Self-reported numbers. Precision figures come from the operator, and the 2026 early tests come without a published design (operator-reported)¹². Meta also announced in 2025 that its automated and AI-assisted proactive enforcement would focus on illegal and high-severity violations². A narrower scope can raise a precision measure on its own, and the text read does not report precision for a constant scope. No independent platform-level audit of machine-learning accuracy was found, and no evidence on whether gains persist beyond the periods Meta reports.

Appeals. High overturn rates are a warning but not a clean error rate. About two-thirds of the Appeals Centre's overturns were defaults where the platform did not supply content, which reflects process failure rather than model error, and complainants self-select⁷. The commentators who relay the later figures say little can yet be concluded, because high overturn rates could reflect platform errors, bodies siding with complainants or the types of case brought⁹. Scale of contestation is also visible: in February 2024 Meta received more than seven million appeals on hate-speech removals, and eight in ten appellants used the new option to add context (operator-reported via the Board)⁴.

Context and crisis. In 2023 Meta temporarily lowered the confidence thresholds of its classifiers for content from Israel and Gaza, which led to drastic removal of non-violating content; in one case the removal and the rejection of the appeal were both decided automatically from a classifier score, with no human review (Board findings)⁴. Faster decisions invite more automation and a higher risk of ignoring context⁵.

Matching amplifies errors. A cartoon on police violence in Colombia was wrongly added by a human reviewer to a matching bank, which removed it at scale; 215 users appealed and 98% succeeded, yet it stayed in the bank until the case reached the Board (Board findings)⁴. Matching is older technology, not AI, but the case shows how automation magnifies a single mistake.

Language. The Ethiopian audit found under-detection in low-resource languages⁶. Meta stated in March 2026 that its new systems work in languages spoken by 98% of people online, against about 80 languages before (operator-reported)¹. The two are not like-for-like, since the audited tools are not Meta's systems. No platform reported the language of moderated content in the EU database, so language-level accuracy cannot be checked from public data⁵.

Failures and regulators. Meta reported that false positives rose by over 100% for a very brief period of the fourth quarter of 2025 because of a bug (operator-reported)³. On 24 October 2025 the European Commission preliminarily found that Facebook and Instagram breach obligations to offer simple tools to report illegal content and to let users effectively challenge moderation decisions; the findings are preliminary⁸. Searches for paused or withdrawn AI moderation systems found none.

Phase: In wide use — why

The hypothesis is kept for classifier-based detection. Machine-learning classifiers are confirmed at Meta at very large scale⁴, and automation is reported across eight very large platforms⁵. The share that is machine learning rather than hash-matching or rules cannot be established, which argues for caution. The newer advanced systems announced in March 2026 are in rollout and are not yet shown to outperform current methods at scale¹.

What to watch next

• The Commission's decision on the October 2025 preliminary findings against Meta's reporting and appeal tools⁸.
• Meta's next semi-annual enforcement report, covering the second half of 2026, to show whether precision stays near 91% and 92% as the new systems expand²³.
• Any publication of test design or independent audit for the March 2026 claims¹.
• The general review of the Digital Services Act, which a commentator says begins in 2027⁹.

Questions to ask before adopting

• Which parts of the automated enforcement use machine learning, and which use rules or hash-matching, with error rates for each?
• How is accuracy measured, on what sample and by whom, counting missed violations as well as wrongful removals?
• Which languages do users write in, and what measured performance exists for each?
• What share of automated removals can be appealed to a person, and how fast can the original content be retrieved for review?
• What happens to thresholds in a crisis, and who approves the change?