Thin-File Credit Underwriting: Machine Learning Gains Are Modest and Rarely Measured on Thin Files

The use case

In a 2025 study, researchers took credit-bureau and bank-account data on consumers who opened accounts in 2018 and 2019 and scored them two ways: with logistic regression, an older statistical method, and with XGBoost, a machine-learning method that combines many decision trees¹. The question was whether the newer method, and the extra bank cash-flow data, would approve more people who repay without approving more who default.

That design shows why the scope matters. In this paper, AI means machine-learning models. Logistic regression scorecards and conventional credit scores are older technology and are not credited as AI. Cash-flow data is an input, not a method. Where studies bundle the two, the paper says so. Fraud detection and generative AI are out of scope.

Where it is running

Named machine-learning underwriting is documented at three places, none with a usable scale figure in the sources read.

The share of lenders using machine learning for underwriting could not be established from the sources read. The FinRegLab study used bureau and aggregator data, not a lender's live portfolio¹.

What the evidence shows

Simulation on bureau data (2025). Most of the measured gain in the 2025 study came from the machine-learning method rather than from the cash-flow data. On a standard ranking score (ROC-AUC, where 1.0 is perfect), the logistic regression baseline on bureau data scored 0.8653. Adding cash-flow data to logistic regression added 0.0029; a machine-learning model on bureau data alone added 0.0178; machine learning plus cash flow added 0.0201 (simulation on historical data)¹. At a 3% risk cutoff, the baseline approved 65.18% of applicants; machine learning added 2.5 points on bureau data and 3.0 points with cash flow (simulation)¹. The authors summarise this as nearly 4% more approvals and at least 9% fewer approved defaulters than logistic regression analogues (simulation)¹.

Against a conventional credit score. The gain is smaller. The logistic regression baseline built on a traditional credit score scored 0.8662, and a machine-learning model combining that score with cash flow added 0.0061, with approvals up 0.4 points (simulation)¹. At higher-risk cutoffs, approval gains became smaller or negative, though machine learning still cut approvals of eventual defaulters¹.

A lender outside the United States. In a Chinese fintech lender's loans from 2017 to 2018, the machine-learning score reached an AUROC of 0.6391 against 0.5939 for a model using traditional information only (measured without a control group; the machine-learning model was tested out of sample and the others in sample, which the authors say biases against it)². The paper reports that the advantage was greater after a November 2017 tightening of shadow-banking rules, and that it tends to decline for borrowers with a longer credit history (abstract)². That second finding is the closest evidence to the thin-file claim.

Operator-reported outcomes. In a 2019 regulator blog, the tested Upstart model approved 27% more applicants with 16% lower average APRs than a traditional model (operator-provided; the CFPB stated it did not separately replicate the analyses)³. The figures reflect the combined effect of alternative data and machine learning, and the comparator was a traditional model validated by Upstart³.

What it takes: cost, integration, governance

No public cost per loan or integration cost was found. The FinRegLab authors note that raw cash-flow data is less standardised, so results depend on how each builder engineers features¹.

Governance carries more documented weight than cost. The 2017 no-action letter was conditioned on a confidential model risk management and compliance plan, is expressly not an endorsement of any variable, and could be modified or revoked at any time⁴. In the Earnest case, the Massachusetts Attorney General alleged that the lender failed to test its models for disparate impact and used a Cohort Default Rate variable that penalised Black and Hispanic applicants more than White applicants. Earnest paid $2.5 million (settlement of allegations, 10 July 2025), must adopt written AI policies and a governance structure, and stopped using that variable⁵.

In the European Union, Regulation (EU) 2024/1689 classes AI systems that evaluate the creditworthiness of natural persons as high-risk⁹. A May 2026 law-firm summary reports a political agreement moving compliance for stand-alone high-risk systems to 2 December 2027; formal adoption was not verified¹⁰.

Where it falls short

Unresolved approval gaps. A monitor appointed for Upstart reported in November 2021 no significant price disparities for any protected class but statistically significant approval disparities for Black applicants; the monitor said these did not account for legitimate creditworthiness differences and did not alone demonstrate a violation⁶. The final report of March 2024 again found approval disparities, no close proxies for protected classes and no pricing disparities. Upstart and the monitor reached an impasse because the monitor did not validate Upstart's claims for a less discriminatory alternative model⁷. Upstart says its model approves 43% more Black borrowers than traditional models (operator-reported; data not shown)⁸.

The target group is barely measured. The FinRegLab authors state that their data holds few consumers with little or no credit history and skews toward prime, higher-income consumers, and that the results might have been stronger with more thin-file observations, which they cannot confirm¹. The study is a 2018-2019 simulation, and the authors flag the pandemic period as a limit¹. It was funded by two large banks¹. The Chinese study covers a single anonymous lender².

Challenge test. The main claims rest on simulations or operator-supplied comparisons, not on live trials with a control group. Whether gains persist, what they cost, and how much depends on human supervision and compliance staff is not shown. No withdrawals or bans of the use itself were found in the sources read; the Upstart letter ended at the company's request⁸.

Phase: In early use — why

The hypothesis is kept. Machine-learning underwriting is confirmed in live lending (Upstart, Earnest, the Chinese lender), but the scale of use is unknown and no independent evaluation shows outcomes for genuine thin-file borrowers. The evidence does not support a higher phase.

What to watch next

• Formal adoption and Official Journal publication of the EU omnibus dates for Annex III systems, reported in May 2026 as 2 December 2027¹⁰.
• Publication of independent, lender-level outcomes for thin-file applicants scored with cash-flow or machine-learning models, which would close the main evidence gap.

Questions to ask before adopting

• Does the evidence cited for this model separate the gain from the algorithm from the gain from new data?
• How many of the validation cases were genuine thin-file borrowers, and in which period?
• Who produced the comparator, and was the comparison tested out of sample for both models?
• What approval and pricing differences by protected group have been tested, and with what less discriminatory alternatives?
• What written policies, documentation and adverse-action reasons will meet the rules that apply in each market?