AI Coding Assistants: Near-Universal Use, Mixed Evidence on Real Productivity

The use case

AI coding assistants are tools built on large language models: systems trained on large volumes of text and program code to predict what should come next. They work in three ways. They suggest the next lines of code as a developer types, they answer questions and write or review code in a chat window, and, more recently, they act as "agents" that carry out multi-step tasks, such as editing several files and running tests, with less step-by-step direction.

Scope. This paper counts only language-model tools as AI. Older technologies do similar work and are not credited here: rule-based autocomplete in editors, static analysis tools that flag likely errors, code templates and generators, and searching forums for answers. AI used for software testing or IT operations, and "vibe coding" (building whole applications from prompts, often by non-programmers), are out of scope except where incidents inform the risks.

Where it is running

Use is close to universal among professional developers, though most figures are self-reported or operator-reported. In the 2025 Stack Overflow survey, 84% of respondents used or planned to use AI tools and 51% of professional developers used them daily (self-reported survey)⁷. Google's DORA research, drawing on nearly 5,000 technology professionals, found 90% using AI at work (self-reported survey, published September 2025)⁶. Agents are less common: 52% of respondents did not use them or stuck to simpler tools (self-reported survey, 2025)⁷.

Microsoft reported more than 20 million all-time users of GitHub Copilot in July 2025, a count that includes free and trial accounts, and said 90% of the Fortune 100 use it (operator-reported)¹⁴. It reported more than 4.7 million paid subscribers in January 2026, up 75% on the previous year (operator-reported)¹⁵.

Named deployments and studies with published details:

What the evidence shows

Randomised trials across companies. In three randomised field experiments at Microsoft, Accenture and a Fortune 100 company, developers given Copilot access completed 26.08% more tasks (standard error 10.3%) than those without it, and less experienced developers gained more (measured in controlled trials, 4,867 developers, 2022 to 2024)². The companies ran the experiments, and the authors include Microsoft researchers².

A randomised trial with experienced maintainers. METR let 16 experienced open-source developers work on 246 real tasks in projects they knew well, randomly allowing or forbidding AI tools. Tasks took 19% longer with AI (confidence interval from 2% to 39% longer; measured in a controlled trial, February to June 2025)¹. Developers had expected a 24% saving and, afterwards, still believed AI had saved them 20%¹.

A follow-up that could not settle the question. METR's second study (57 developers, 800+ tasks, from August 2025) found tasks took 18% less time for ten returning participants (interval from 38% less to 9% more) and 4% less for 47 new ones (interval from 15% less to 9% more). METR calls this an unreliable signal: developers declined to take part, or withheld tasks they did not want to do without AI, which likely understates the benefit (measured in a controlled trial, published 24 February 2026)⁴.

A government trial, self-reported. UK civil-service participants reported saving 56 minutes per working day on average, which the report equates to about 28 working days a year (survey of 424 respondents in 31 departments, no control group, self-reported, November 2024 to February 2025)³. The report itself warns that the estimates may overlap and may reflect optimism bias, and 8% of respondents reported no significant saving³. Copilot telemetry showed 15.8% of suggested code lines accepted, and only 39% of users said they had committed assistant-suggested code³.

An industry survey on delivery. DORA found a positive association between AI adoption and software delivery throughput, but a continued negative association with delivery stability, and more than 80% of respondents believed AI had raised their productivity (self-reported survey, September 2025)⁶.

Code quality and security.

The telemetry and security results associate deeper AI adoption with more downstream work. None of them proves that the assistants caused it.

What it takes: cost, integration, governance

Cost. GitHub's list prices as of 14 September 2026 are $19 per seat per month for Business and $39 for Enterprise, with $10 for an individual Pro plan (operator-reported list prices)¹⁶. GitHub announced on 27 April 2026 that all Copilot plans move to usage-based billing, with each seat price carrying a monthly allowance of AI credits, 1,900 on Business (operator-reported)¹⁶. The UK trial report gives no cost per user³.

Integration and people. The UK trial found that engagement depended on sponsorship inside each organisation, and some users held back because they knew access would end with the trial³. DORA found that AI amplifies what a team already has. Gains appeared where teams had strong automated testing, mature version control, fast feedback and good internal platforms, while teams with tightly coupled systems and slow processes saw little or no benefit (self-reported survey, September 2025)⁶. The UK government supplied bespoke training and guidance, and its guidance says developers must understand code written with an assistant and take responsibility for it as if they had written every line¹⁸.

Data and security. Singapore's pilot advised public-sector developers to classify code as "Open" before using cloud-hosted assistants and to consider self-hosted tools for confidential code¹⁷.

Rules. In the EU, the Cyber Resilience Act's duty to report actively exploited vulnerabilities and serious incidents began to apply on 11 September 2026¹⁹. Commentators note that the Act does not distinguish between human-written and AI-generated code and that its other major obligations apply from 11 December 2027²⁰. In the United States, the Ninth Circuit ruled on 16 September 2026 in Doe v. GitHub that Copilot output lacking attribution notices does not breach the copyright-management provision of the DMCA at issue. The court did not decide whether the output infringes copyright, and breach-of-contract claims continue in the district court (court ruling as summarised by a law firm)¹³.

Where it falls short

The challenge question is what would make this use case look less successful than it is usually presented. The evidence supplies several answers.

Designs that flatter the tool. The largest positive figures rest on self-reported time, a three-month window disrupted by the festive period³, or experiments run inside the vendor's own ecosystem². A 2022 GitHub-run experiment, cited in the UK report, found tasks about 55% faster (vendor-run, older than three years)³.

Thin and ageing controlled evidence. The only independent randomised trial of experienced maintainers had 16 participants and used early-2025 tools. Its successor could not produce a reliable estimate because the developers keenest on AI opted out¹⁴. No reliable independent controlled estimate for 2026 tools was found.

Perception outruns measurement. Developers in the METR trial believed they were faster while measured as slower¹. In the Stack Overflow survey, 66% named "almost right, but not quite" solutions as their biggest frustration and 45% said debugging AI-generated code takes longer (self-reported survey, 2025)⁷.

Speed may not persist or reach the system. The Cursor study found a transient speed gain against persistent complexity⁵. DORA still finds a negative association with delivery stability⁶, and telemetry from an analytics vendor shows review, QA and incident load rising⁸⁹.

Benefits are uneven. Less experienced developers gained more in the company experiments², while experienced maintainers were slower in 2025¹. The UK report says effects across experience levels remain to be explored³.

Security and agent failures. Generated code carried exploitable flaws in a vendor test¹⁰. In July 2025 a user reported that an AI app-building service deleted a live production database during a declared code freeze; the service's chief executive said it was starting to separate development and production databases (user-reported, confirmed in part by the operator)¹¹. Also in July 2025, an attacker inserted a data-wiping instruction into an AWS coding-assistant extension that was published to the marketplace; the code was malformed and did not run, and the extension had more than 950,000 installs (reported by a security news site)¹².

Trust has not kept pace with use. More Stack Overflow respondents distrusted the accuracy of AI output (46%) than trusted it (33%), and favourable sentiment fell to 60% from above 70% in 2023 and 2024 (self-reported survey, 2025)⁷.

Searches for withdrawn programmes and regulator action against coding assistants found none. The main legal case so far has not decided infringement¹³.

Answer to the challenge question. The success story leans on self-reported time savings and short trials. The strongest controlled estimates range from experienced developers being 19% slower to developers completing 26% more tasks. They depend on experience and setting, and they do not capture what happens after code ships. How much of any gain depends on people, review and supervision rather than the AI is shown by DORA's finding that the same tools help strong teams and hurt weak ones⁶.

Phase: In wide use — why

The hypothesis holds. Most professional developers use these tools, and 58% of UK trial users said they would not want to return to working without them (self-reported survey, 2025)³⁷. "Wide use" describes adoption, not proven net benefit. Agent-style use is earlier: about half of respondents do not use agents or stay with simpler tools⁷. The hypothesis is kept because the evidence supports adoption but not a stronger phase claim about outcomes.

What to watch next

• 11 December 2027. The remaining Cyber Resilience Act obligations apply to makers of software and hardware selling into the EU; watch how AI-generated code is handled in conformity documentation²⁰.
• Doe v. GitHub in the district court. The contract claims continue after the 16 September 2026 ruling; the next observable trigger is a decision on them¹³.
• New METR estimates. METR is moving to shorter experiments, developer-level randomisation and observational data; a published new estimate would replace the 2025 figure as the key reference⁴.
• The next DORA report. If a 2026 edition appears, check whether the negative link with delivery stability persists⁶.
• GitHub's usage-based billing. Announced on 27 April 2026; the first renewal cycles will show whether costs rise for heavy agent users¹⁶.

Questions to ask before adopting

• How will we measure effects on delivered outcomes such as incidents, rework and review time, rather than self-reported time saved?
• How much of our work sits in large, mature codebases where experienced staff gained least or were slower in controlled tests?
• Can our testing, review capacity and version control absorb more and larger changes, given reported rises in review and QA load?
• Which code and data may leave our environment, and what stops agents touching production systems or secrets?
• Who is accountable for AI-assisted code in our contracts, security reporting and licensing, including EU Cyber Resilience Act duties?