The Irony: Hallucinated Citations Turn Up in Papers at NeurIPS, a Top AI Conference

Even the most prestigious scientific conferences are not immune to problems in the age of generative AI. A study by the startup GPTZero found cases of "hallucinations" — fabricated citations generated by AI — in papers presented at NeurIPS, one of the leading conferences in artificial intelligence. The irony is plain: a conference devoted to cutting-edge AI advances has fallen victim to its side effects.

GPTZero, a company that specializes in detecting AI-generated text, analyzed a corpus of papers accepted at NeurIPS. The results were alarming: several papers contained references to nonexistent sources, apparently created by large language models (LLMs). This casts doubt not only on the quality of individual papers but on the entire peer-review system.

The problem of hallucinations in LLMs is well known. Models trained on huge volumes of data sometimes "invent" facts or citations to fill gaps in their knowledge or to make a text more convincing. In scientific papers, this can have serious consequences, eroding trust in research and making results harder to verify.

The discovery of AI-generated citations in NeurIPS papers raises several important questions. First, it shows that scientific papers need more thorough checks for the use of AI. Reviewers should pay closer attention to detail and double-check the sources a paper cites. Second, it highlights the importance of building tools to detect AI-generated text and building them into the scientific publishing process.

The incident also forces a rethink of the future of scientific research in the AI era. On one hand, AI can become a powerful tool for scientists, helping them analyze data, generate hypotheses and write papers. On the other, the risks of using AI have to be taken into account, and steps taken to prevent abuse.

Ultimately, the discovery of hallucinations in NeurIPS papers is a warning sign. It is a reminder that AI is only a tool, and using it demands responsibility and critical thinking. The scientific community should draw up clear rules and procedures for working with AI to guarantee the quality and reliability of scientific research.