At a Glance
- GPTZero detected 100 hallucinated citations across 51 papers from 4,841 NeurIPS submissions
- Mis-citations amount to 1.1% of accepted papers, statistically minor but reputationally serious
- Peer reviewers missed the fabrications amid a “submission tsunami” stricter than ever
- Why it matters: If top AI researchers can’t police their own LLM outputs, trust in academic integrity erodes
AI detection firm GPTZero scanned every paper accepted to last month’s Conference on Neural Information Processing Systems (NeurIPS) in San Diego, flagging fabricated references that slipped past multiple peer reviewers.
The Scale of the Problem
Out of 4,841 accepted papers, the startup confirmed 100 hallucinated citations inside 51 works. The total pool of references runs into the tens of thousands, so the error rate sits near 1.1%, a figure NeurIPS itself later echoed to Fortune.
- 100 bad citations
- 51 papers affected
- 4,841 total papers checked
Why Citations Matter
Citations act as academic currency. Researchers list them on CVs, grant proposals and tenure files to quantify influence. When an LLM invents a study, the phantom entry still inflates citation counts, diluting the metric for everyone.
NeurIPS states it champions “rigorous scholarly publishing in machine learning and artificial intelligence.” Each manuscript undergoes multi-person peer review, with explicit instructions to flag hallucinations.
Peer Review Under Strain
Reviewers face a workload surge GPTZero calls a “submission tsunami.” The startup’s report references a May 2025 paper, “The AI Conference Peer Review Crisis,” that documents how rising manuscript volume stretches conferences like NeurIPS “to the breaking point.”
Given the load, missing a handful of AI-generated references becomes understandable, GPTZero notes.
What NeurIPS Says
In a statement to Fortune, which first covered GPTZero’s findings, the conference stressed that “even if 1.1% of the papers have one or more incorrect references due to the use of LLMs, the content of the papers themselves [is] not necessarily invalidated.”
Irony for the AI Elite
The episode highlights an awkward truth: researchers who design state-of-the-art language models still struggle to verify the outputs they use for routine tasks like bibliography formatting.
If the field’s leading minds, reputations on the line, can’t guarantee accuracy in minor details, the broader question becomes what that portends for everyday users who place similar trust in generative AI.
Key Takeaways
- GPTZero’s audit shows AI slop can creep into even elite venues
- Statistically small numbers still chip away at trust metrics
- Peer review pipelines need reinforcements as submission counts climb
- Researchers remain responsible for double-checking every LLM line, including citations
News Of Philadelphia first reported the company’s findings.
