AI News Summaries: Live at Publishers, Paused by Apple, Rarely Measured

The use case

In December 2024 the BBC complained to Apple that a phone feature that condenses incoming notifications into one line had told readers that Luigi Mangione, charged with murder in the death of the UnitedHealthcare chief executive, had shot himself¹. On 16 January 2025 Apple released a developer beta that disabled the feature for news and entertainment apps¹, saying the summaries would be "temporarily unavailable" while it worked on improvements².

This paper covers language-model summarisation of news. Three arrangements are in use. Publisher-run summaries. A newsroom's own tool condenses its articles into bullet points or short text, or into audio, as at Bloomberg, the Financial Times, Singapore's CNA and Sweden's NTM³⁴. Platform-run summaries. A device maker condenses news alerts from many publishers, as Apple did¹. Assistants answering news questions. General chatbots are outside the use case, but they are the only source of systematic accuracy data, so that data is used below and labelled as adjacent.

Scope note. AI here means a language model that writes new summary text. Editor-written "key points", extractive summaries and templated automated stories are older technology and are not credited as AI. Where an editor checks a summary before readers see it, the result reflects the model and the human check together. Search-engine AI answers are a neighbouring use and are not covered.

Where it is running

The geography is narrow: US and UK publishers dominate, with single examples from Sweden and Singapore. Deployments older than three years were not examined.

What the evidence shows

Audience surveys (independent, self-reported). The Reuters Institute surveyed about 2,000 people in each of six countries between 5 June and 15 July 2025. Only 12% were comfortable with news made entirely by AI; comfort rose to 21% with a human in the loop, 43% when a human leads with some AI help, and 62% for entirely human-made news⁸. Only 33% thought journalists always or often check AI outputs before publishing⁸. The pattern suggests readers accept AI as an assistant to editors, not as the author. In the Digital News Report 2026 (48 markets, fieldwork mid-January to end February 2026), weekly use of AI chatbots for news rose from 7% to 10%, and 1% said AI was their main source⁹. Trust in answers from AI chatbots was 20%, against 37% for news overall⁹. These figures cover AI generally or chatbots, not summaries specifically.

Systematic accuracy testing (measured by journalist review, no control group, run by news organisations). Twenty-two public service media organisations in 18 countries and 14 languages assessed more than 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity in June and July 2025⁵⁶. At least one significant issue appeared in 45% of answers, serious sourcing problems in 31% and major accuracy problems in 20%; Gemini had significant issues in 76%⁵⁷. The questions were open news questions, not summaries of a supplied article. Summarising a given text is a narrower task, but no systematic test of that task was found, so the 45% cannot be transferred to publisher tools.

Operator-reported results. Bloomberg said 99% of its AI summaries meet its editorial standards (operator-reported, April 2025), and INMA noted the figure needs comparison with the correction rate on human-written content⁴. The New York Times reported that Bloomberg had corrected at least three dozen summaries published in 2025, including one that misstated when a broader tariff action would take place³. No source gives the number of summaries published, so the two figures cannot be reconciled. The Financial Times said no one had needed to change anything factual in several months and that engagement had not declined (operator-reported, no figures)⁴. CNA said its FAST summaries accounted for 32% of its monthly average pageview growth over the previous financial year (operator-reported, no control group)⁴. Pageviews measure attention, not accuracy, and growth could come from the format rather than the quality of the summary.

What it takes: cost, integration, governance

Cost. No price or per-summary cost was found in the sources read.

Human control. Bloomberg said journalists have full control over whether a summary appears, before and after publication, and can remove any that fail its standards (operator-reported, March 2025)³. NTM has an editor check summaries before they are converted to audio (operator-reported, April 2025)⁴. Berlingske in Denmark bans AI for direct article production but allows it as a research or summary tool¹³. The Wall Street Journal states that work with AI inputs is reviewed by a journalist before publishing (operator-reported, August 2026)¹³.

Labelling by the platform. When Apple re-enabled news summaries, users choose them during setup and see a red warning: "Summarization may change the meaning of the original headlines. Verify information."¹⁰ Summarised news notifications carry a "Summarized by Apple Intelligence" footer¹¹.

Rules. The European Commission says the AI Act's transparency rules take effect in August 2026 and describes the duty as clear labelling of text published to inform the public on matters of public interest¹⁴. Its page on the Code of Practice describes an exception where a text has undergone human review and is subject to editorial responsibility¹⁵. These are the Commission's descriptions; the Article 50 wording itself was not read. A political agreement on the AI omnibus was reached on 7 May 2026, and the amendments the Commission page names do not mention Article 50, though a grace period was not ruled out¹⁴. Draft guidelines on the transparency obligations were listed on 7 May 2026¹⁴.

Where it falls short

The withdrawal. Apple's summaries combined three New York Times articles into one alert that falsely said Israel's prime minister had been arrested, and a Washington Post summary on 15 January 2025 listed events CNN said had not happened¹. Reporters Without Borders called the feature "a danger to the public's right to reliable information on current affairs", and the National Union of Journalists said the public must not be placed in a position of second-guessing the accuracy of news¹. The structural risk is compression: one line stands in for several unrelated headlines, so context is lost. The feature was off in news apps for about six months, from 16 January to 22 July 2025¹¹⁰. Whether it was fixed is unknown: AppleInsider said it was too early to tell¹⁰, and 9to5Mac's author wrote that he had seen no reports of controversy, an anecdote rather than a measurement, adding that Apple had not said what changed¹¹.

Other withdrawals. No other pause or withdrawal of a news-summary product was confirmed in the pages read. Bloomberg's summaries were still running in April 2025⁴, and the Washington Post's podcast stayed in beta¹². Later status for both is not established.

Launch despite failed tests. A Post internal review, reported by Semafor in December 2025, found that between 68% and 84% of generated podcast scripts failed a standards metric across three test rounds, and concluded that further small prompt changes were unlikely to help without more risk; the product team still recommended release and the Post said the product was in beta¹². This is a podcast, not a text summary, and the figures are the company's own review as seen by a news outlet.

Human review is the control, and it fails. The New York Times corrected an article on 1 May 2026 after an AI-generated summary of a politician's views was printed as a quotation; the editors' note said the reporter should have checked what the tool returned¹³. In May 2026 Berlingske suspended a journalist after an AI summary tool produced fabricated quotes and sources¹³. These are reporters using AI on source material, adjacent to reader-facing summaries, but they show where the failure sits: in the check, not the model.

Exposure and trust. In six countries, 60% said they did not regularly see AI features on news sites or apps⁸, and trust in chatbot answers is about half the trust in news⁹. No data on uneven effects across groups was found.

Challenge question. The favourable numbers are operator-reported, uncontrolled and lack denominators. The accuracy data covers a different task. The best-known failure came from a platform summarising other publishers' text with no editor in the loop, while publisher-run tools depend on editors. The nearest systematic accuracy evidence covers chatbots answering open questions, not the summary tools that publishers and device makers run.

Phase: In early use — why

The hypothesis is kept. Confirmed language-model deployments exist at Apple and at publishers, but the share of readers reached cannot be established⁸. Apple's product was suspended and returned in beta with a warning¹⁰, the Post's product launched in beta¹², and no independent evaluation of publisher summaries was found. "In wide use" is not supported because scale is unknown, and "Pilot with results" is not supported because results are operator-reported.

What to watch next

• Publication status of the final Code of Practice on marking and labelling AI-generated content, which the Commission page indicated for May to June 2026 and which could not be confirmed on 28 October 2026¹⁵.
• Final Commission guidelines on the Article 50 transparency obligations, following the draft listed on 7 May 2026¹⁴.
• The next Reuters Institute Digital News Report, to see whether weekly chatbot news use moves from 10%⁹.
• The Washington Post's podcast leaving beta, or any published error rate for it¹².
• Any independent test of Apple's re-enabled news summaries; none was found as of October 2025¹¹.

Questions to ask before adopting

• Who reviews each summary before readers see it, and can that person remove it after publication?
• How many summaries have been corrected, out of how many published, and how does that compare with human-written copy?
• Was accuracy tested on summarising supplied articles, not on chatbots answering open questions?
• What does the label tell readers, and how do they report an error?
• Did engagement gains come from the summary or from other changes shipped with it, and was there a comparison group?