The best AI models cite retracted papers, and they cannot know it

Ask a top model for the pivotal trial behind a real drug approval and it gives you the right paper and the right DOI, confidently.

In one test run, GPT-5.5, Claude Sonnet 5, and Claude Opus 4.8 all cited the ADVOCATE trial for avacopan in ANCA-associated vasculitis. Correct paper. Real DOI. Published in the New England Journal of Medicine.

It was retracted in 2026. None of the models mentioned it, because the retraction happened after their training cutoff. They had no way to know. A smarter model wouldn’t have known either.

This is the failure I wanted to measure. AI doesn’t just invent citations, it cites real, authoritative, since-retracted papers as solid evidence, and can’t tell the difference. So I built sourcecheck, a small open-source library that resolves every citation against the actual registries instead of asking another model.

The measurement that made it concrete

I asked twelve frontier and production models to answer scientific questions and cite the literature. Every citation went through sourcecheck, which resolves each one against OpenAlex and Crossref and checks it against Retraction Watch. Here’s the same check on one DOI:

from sourcecheck import Verifier, OpenAlexCrossrefBackend

v = Verifier(provenance=OpenAlexCrossrefBackend(mailto="[email protected]"))

report = v.verify([
    {"claim": "The Transformer is based solely on attention mechanisms.",
     "citation": {"doi": "10.48550/arXiv.1706.03762"}},
    {"claim": "LLMs achieve 100% factual accuracy on cited claims.",
     "citation": {"doi": "10.1234/fabricated"}},
])

for c in report.claims:
    print(c.verdict.value, "-", c.reason)

Enter fullscreen mode Exit fullscreen mode

The real paper comes back supported. The fabricated DOI comes back not_verifiable, because it doesn’t resolve to anything. There’s a CLI too, built to run as a CI gate on AI-written docs:

sourcecheck docs/summary.md --fail-on retracted,unresolved

Enter fullscreen mode Exit fullscreen mode

It’s not on PyPI yet, so for now it’s git clone https://github.com/aberaio/sourcecheck && pip install -e ..

I split the retraction questions into two kinds. Famous old retractions (the STAP stem-cell papers, the Surgisphere hydroxychloroquine study) that predate every model’s training. And prominent papers retracted in 2025 and 2026, after the cutoffs, that were reputable and citable when the models were trained.

The contrast is the whole story:

Retraction Citations to retracted papers Flagged by the model itself Famous, pre-cutoff (STAP cells, Surgisphere, amyloid-β*56…) 74 82% (61/74) Post-cutoff, reputable when the model trained 9 0% (0/9)

They’d learned the old ones. Six of the nine post-cutoff misses came from the same top-tier models that catch the old ones, and every one of them cited the retracted paper as valid. sourcecheck caught all of it, old and new, because it checks the registry every time instead of relying on what it was trained on.

Why this is the durable problem

A model gets better at not inventing DOIs as it scales. On easy, famous citations the best models are already near zero fabrication in my runs. But no amount of scaling lets a model know about a retraction that happened after it was trained. That’s structural. The gap between “the model cites it” and “the registry knows it’s dead” doesn’t close with a better model. It closes with a lookup.

That’s the case for a verification layer most demos miss, because a demo asks about things the model already knows.

Fabrication is still real, and it’s worse than the frontier suggests

The existence check, does this DOI resolve to a real paper at all, is the other half. On famous classics the best models fabricate almost nothing, 5.4% across the board. Push off the well-trodden path and it climbs, roughly doubling to 10% on questions about 2025 findings.

The gap between models is where it gets interesting for anyone picking a model for a production pipeline:

Model Citations Fabricated Fabrication rate Gemini 3.1 Pro 35 0 0.0% Claude Fable 5 10 0 0.0% Claude Opus 4.8 43 1 2.3% Claude Sonnet 5 42 1 2.4% Gemini 3 Flash 36 1 2.8% GPT-5.5 44 2 4.5% GPT-5.4 47 4 8.5% Gemini 2.5 Pro 42 6 14.3% GPT-5.4-mini 40 7 17.5% Gemini 2.5 Flash 38 7 18.4% Claude Haiku 4.5 35 9 25.7% GPT-4o-mini 37 14 37.8%

Two of the twelve hit 0% outright. The rest climb steadily to GPT-4o-mini, an older small model still in wide production deployment, which fabricated 38% of the DOIs it cited, more than one in three. Most production systems don’t run the frontier flagship on every call. They run somewhere in that middle band, where fabrication sits in the double digits.

Take away the anti-fabrication system instruction and GPT-4o-mini’s rate climbs to 43%. Prompting narrows the gap. It doesn’t close it.

You can inspect every record

There’s an interactive playground. Pick a model, pick a question, and see each citation with sourcecheck’s verdict. The DOI gets struck through if it doesn’t resolve, flagged amber if the paper’s retracted. For the post-cutoff ones there’s a note: the model had no way to know. Nothing is mocked, every citation was resolved live.

What this is, and what it is not

sourcecheck checks whether a claim is sourced, not whether it’s true. It resolves the citation and checks whether it’s been retracted. Experimentally, it also checks whether the claim actually shows up in the source. Those are the checks a language model can’t run on itself: it hallucinates DOIs, and it can’t know about a retraction that happened after its own training cutoff.

One fairness note: fabrication rate is the ranking, but the retraction result isn’t a per-model penalty. Citing the correct, since-retracted paper is often the right answer to the question. The value is the flag the model couldn’t raise on its own.

If you’re building anything that lets an AI cite sources over medical, legal, or scientific material, this is the failure mode that never shows up in a demo and always shows up in production. The board’s at mikias.io/citations, the library’s open source at github.com/aberaio/sourcecheck, and every run publishes its full per-citation audit.