Few-Shot Inspection: Close on Benchmarks, Unproven on the Production Line

The problem

The anchor paper in this series found deep-learning visual inspection in early use at named European carmakers, with results mostly reported by the companies or vendors themselves¹. Its one independent trial, on aero-engine blades, found a commercial deep-learning prototype less accurate than human inspectors¹². It also noted that models can degrade after changes in lighting, camera setup or material batch, and that no source published results from a second plant¹. This paper adds a different question: whether the data a model needs for each new product or line could shrink from many labelled images to a handful, and what the evidence says about that.

Each new product starts with few or no defect examples. Authors from Valeo and Intel write that real defects are scarce and commercially sensitive, so public test sets usually hold only 4 to 5 defect types per product, while real production lines can show more than 20 (authors' statement, March 2025 preprint)³. A 2026 preprint on new product introduction makes the same point: labelled defect data is scarcest exactly when automated inspection is most needed (abstract)⁴. Earlier deep-learning approaches ask for much more; Aust et al. cite literature stating that several thousand images per defect class are usually needed, as the anchor paper reports (2022; the underlying literature was not checked)¹².

No independent source opened put a figure on what this labelling costs for a new line. Vendor claims of production-ready models from five to twenty images turned up in the search but were not independently confirmed, and are not used here.

What already works

A common way around defect labels is anomaly detection: the model learns what a good part looks like and flags whatever differs. These methods are often called unsupervised, but the 2025 Valeo and Intel benchmark argues that "one-class" is the more accurate term, because the models still need images known to be good³. Few-shot versions build that picture of normal from a handful of reference images, usually with a large pre-trained vision model that supplies the image features, so no further training is needed.

Benchmark results. AnomalyDINO (WACV 2025) reaches an image-level AUROC (100% would be a perfect ranking of defective above good parts; 50% is chance) of 96.6% on the MVTec AD benchmark from one reference image and 98.4% from 16, against 99.5% when all training images are used. On the harder VisA benchmark the figures are 87.4%, 94.8% and 97.6%. These were measured by the authors on public test splits at a 672-pixel setting, with the first k training images as references rather than hand-picked ones. Building the reference set from one image took about half a second, and scoring an image about 60 milliseconds, on one data-centre GPU at 448 pixels (author-measured)⁵.

In a 2023 paper, FastRecon reached 95.2% image AUROC on MVTec AD from 8 images of good parts, against 99.1% for a full-data PatchCore model. On MPDD, a harder set of metal parts photographed at varying distances and angles, 8 images gave 82.5%, against 82.1% for full-data PatchCore (author-reported)⁶. The vision-language model WinCLIP reports 91.8% AUROC on MVTec AD with text prompts alone and 93.1% with one image of a good part; on VisA the figures are 78.1% and 83.8% (abstract, 2023)⁷.

Written criteria instead of images. A 2025 preprint from the University of Erlangen-Nürnberg, PB-IAD, gave a general-purpose multimodal model (GPT-4.1) one image of a good part plus a written description of the defect types. On the cable class of MVTec AD it reached an F1 score (a balance of missed defects and false alarms) of 95.7%, and a PatchCore model needed about 100 training images to match it. On a custom set of stripped wires photographed under varied lighting, angles and colours, one-shot F1 was 91.0%, while PatchCore peaked at 83.8% with 190 images. With no reference image at all, adding written context and defect criteria raised F1 on the cable class from 84.0% to 93.1%. All figures were measured by the authors on small datasets, and the size of the wire test set was not stated in the text read⁸.

Synthetic defects. A 2026 preprint on new product introduction trained a conventional detector on generated defect images. On a private industrial dataset, adding them to a few real defects raised mAP (average precision across detection thresholds) from 78.8% to 83.3%, and transferring defects learned on another surface raised mAP from 65.0% to 85.1% on a target surface with no real defects (abstract; private dataset)⁴.

What does not exist yet. No source opened reports a few-shot inspection method running on a production line with measured false-reject and escape rates over weeks, or compared with human inspectors. The 2025 Valeo and Intel benchmark on real-world datasets set zero-shot and few-shot methods aside as out of scope³. The evidence is benchmark and small-dataset research; it says nothing about how plants in different regions differ.

The possibility

One way this could work: when a new product arrives, a quality engineer photographs a few parts known to be good and writes a short description of what counts as a defect and what variation is acceptable. The system builds its reference in seconds. For a trial period the line runs in shadow mode, in which the model flags parts but people decide. Each flagged part that a person judges good, and each hard case, could be added to the reference set. That is how a one-image start could grow toward the 16-image results in the benchmarks.

If this held, the labelling effort per new product would shift from collecting and marking many defect images to choosing reference images and writing criteria. The gain could be largest where defects are rare, products change often and each variant runs in small batches, though no source measured this.

What would have to be true

1. The few references must cover everything normal includes. Across three one-image runs, AnomalyDINO's score on the MVTec AD Capsule class had a standard deviation of 5.5 points around 90.2%, and on the VisA PCB4 class 13.7 points around 79.9%, with PCB4 reaching 99.0% at 16 images⁵. In the PB-IAD cable tests, one reference image raised recall from 73.9% to 89.1% but cut precision, the share of flagged parts that were truly defective, from 97.1% to 95.4%⁸.
2. The defects must show up as local differences from normal. AnomalyDINO detected swapped cable wires at 50.2% AUROC, no better than chance⁵. The Valeo and Intel authors explain that such logical defects, like missing or wrongly assembled components, involve something normal being absent, which cannot be found by spotting abnormal features³.
3. Imaging conditions must hold steady, or the references must be refreshed. Under simulated noise, blur, colour, shadow and camera-angle shifts, the mean image AUROC of a full-data PatchCore model fell from 91.6% to 74.4% across three datasets, and that of RD4AD from 90.7% to 48.6%³. MVTec AD 2 adds lighting-change test scenarios, and its authors report state-of-the-art methods staying below 60% average AU-PRO across its scenarios (abstract, March 2025)⁹; no source opened measured few-shot reference sets under such shifts.
4. Benchmark scores must carry over to production. The Valeo and Intel authors report that models scoring above 99.9% on MVTec AD degraded markedly on real-world data, and that some newer models fell behind older ones³. The few-shot figures above come from public benchmark sets such as MVTec AD, VisA and MPDD⁵⁶.
5. Error rates must pay off in money, not only in AUROC. On a dataset with more than 20 real-world defect types, full-data PatchCore scored 88.0% AUROC, yet correctly set aside only 16.5% of good parts when no more than 2% of defective parts could be missed³. The authors note that missing a defective part usually costs far more than a false alarm³.
6. Someone must own the written definition of a defect. The PB-IAD authors observe that whether a scratch counts as a defect is often subjective, and that a model without product context may pass a subtle first-time change, such as a batch colour shift, as normal⁸. In the AnomalyDINO tests, a reference that left out normal variation, a capsule photographed with its printed text hidden, produced false alarms on good capsules⁵.
7. The model must run within plant rules. The PB-IAD tests used a commercial model, and the authors name smaller, locally hosted models as future work⁸. AnomalyDINO's timings were measured on one NVIDIA A40 GPU, and no source opened reported timings on hardware installed at a line⁵.

What could stop it

Evidence gap. No independent trial of few-shot inspection against human inspectors was found. The anchor paper reports that the one independent human comparison used 50 practitioners and aero-engine blade images, and found a commercial deep-learning prototype at 47.4% accuracy against 63.8% for humans on borescope images, from only 20 maintenance images rather than production ones (measured in a trial, 2022)¹².

Benchmarks flatter the methods. The MVTec AD 2 authors say scores on the older MVTec AD and VisA sets have saturated in segmentation AU-PRO, with leading models separated by less than one percentage point, so those sets no longer rank methods meaningfully (abstract, March 2025)⁹. Most of the benchmark figures above come from exactly those sets.

A weak reference is a single point of failure. The AnomalyDINO authors note that results depend heavily on which reference images are chosen, and that this holds for all one-shot and few-shot methods⁵. Mixing 4% to 16% defective images into a full-data PatchCore model's training images lowered its mean AUROC from 89.2% to between 87.3% and 84.7%³. With a handful of references, a single wrongly judged good part could make up the whole reference set (reasoning, not measured).

Simpler routes may do the job. In the stable cable test, a conventional PatchCore model matched the best one-image prompt result after about 100 images of good parts⁸. Photographing 100 good parts needs no defect labels, so the saving from going down to a handful may be small next to lighting, camera and integration work. No source opened measured how setup time divides among these, or gave a price for a few-shot setup (reasoning, not measured).

Trust in a quick set-up. In the deployments the anchor paper describes, people still review flagged items or finish the inspection¹. A system set up fast from few images could invite more trust than its evidence supports; the AnomalyDINO authors themselves call for more awareness of the risks of overreliance⁵.

Signs to watch

• Few-shot entries on the public MVTec AD 2 evaluation server, which holds the hidden test answers and includes lighting-change scenarios⁹.
• A follow-up to the Valeo and Intel benchmark: its authors say zero-shot and few-shot vision-language methods may be included in future work³.
• A published production result for any few-shot or prompt-based inspection system, with false-reject rate, escape rate, period and a human baseline. None was found.
• Results from a second product or plant for any deployment named in the anchor paper¹.
• Tests of smaller, locally hosted models for prompt-based inspection, which the PB-IAD authors name as future research⁸.