Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseCode / SDK · ActionableX: @moreisdifferent→ original

Researcher Checks 7.1 Million Scientific Papers for Duplicate Images and Data Errors

Journals cannot keep up with checking every image and table, so a single gel photo can pass as the result of different experiments. A researcher from Metascience Observatory has downloaded 42.9 million images from 7.1 million Europe PMC papers and is searching them for duplicates. In about 500,000 data files, he has also found more than 200 papers with anomalies, including some in Nature. Scripts find the candidates, and AI only filters them. This is more reliable than asking a model to find all the errors.

AI-processed from X: @moreisdifferent; edited by Hamidun News
Researcher Checks 7.1 Million Scientific Papers for Duplicate Images and Data Errors
Source: X: @moreisdifferent. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceX: @moreisdifferent ↗
Editor-in-Chief
Key metric #1A test run over several thousand papers turned up 40 image problems
Key metric #2The open-source Forensic Metascience AI Toolkit works with Claude Code and Codex
Key metric #3XML/JATS is easier to work with than PDF, making tables easier for AI to read

Scientific journals rely on reviewers who simply cannot check every image and every table in time. As a result, the same gel or microscopy image can appear in two papers as the result of different experiments, and a data table can contain copied rows. A researcher who runs the @moreisdifferent account and the nonprofit Metascience Observatory, funded through Lightcone Commons, has set out to check such things at scale.

The scope: he has downloaded 42.9 million images from 7.1 million papers in the Europe PMC database (an open archive of biomedical publications), and is separately analyzing about 500,000 data files attached to papers.

The key to his approach: he does not ask a neural network to "find all the errors in a paper."

What Exactly Is Being Checked

The project has two strands. The first is images. The author wrote a tool that looks for improper reuse of images both within a single paper and across different works. Test runs over several thousand papers have already turned up 40 cases, which he has posted to PubPeer, a site where scientists publicly discuss papers after publication.

The second strand is data files: tables and raw measurements. Specialized tools search them for anomalies and artificially duplicated values. The original post mentions more than 100 problems found, while the project's text refers to more than 200 papers with data problems. These numbers likely reflect different points in the review and are still being refined. Many of these papers appeared in major journals: Nature, Nature Communications, Scientific Reports, Science Advances. The author is currently reviewing the findings manually.

Why Not Just Ask a Model

The author argues that AI, by default, is poor at noticing subtle problems: repeated image fragments, suspiciously similar numerical series, calculation errors. So candidate detection is done by ordinary scripts, and the model is brought in afterward to weed out false positives and rank the findings by importance. For a developer, this is a familiar pattern. Think of checking logs or financial reports: deterministic code flags suspicious spots, and an LLM decides which of them are worth showing to a human.

From this came the open-source Forensic Metascience AI Toolkit, a modular system for assembling different paper-checking workflows. The "tools agent" module gives an AI agent a set of tools for checking data and calculations. Many of the methods come from James Heathers's book An Introduction to Forensic Metascience. The whole setup can run on top of Claude Code, Codex, Grok Build, and Antigravity subscriptions. The author sees the project as a cheaper and, he hopes, higher-quality alternative to paid paper-checking services. To prove it, he is preparing benchmark sets with known errors in advance, so he can compare which tools catch which types of problems.

There is also a useful engineering detail. The author built a Python package for legally downloading papers. It can often retrieve XML/JATS (a machine-readable article format) even when no PDF is available, and AI reads tables from XML more easily than from PDF. The --fetch-everything flag collects everything tied to a paper: XML, PDF, code, appendices, and linked datasets. The package finds links to such materials through various APIs and a single LLM call that scans the paper's text.

How Findings Are Handled, and the Limitations

The author posts some findings on PubPeer. For more serious cases, he writes to the authors and journal editors before any public announcement, so they have a chance to respond. He stresses that no report should be read as an accusation of intent: data duplication can be accidental.

The limitations are clear. A human must confirm every finding, the benchmark sets for evaluation are still being prepared, and the flow of results is growing faster than one person can review it. For that reason, the author is looking for volunteers to verify findings through a dedicated web app with letter templates, programmers for the open part of the project, and colleagues with similar goals to share the work.

The practical takeaway for people building agents: if a task demands precision, a single large prompt is a bad bet. It is more reliable to build a set of narrow tools that find candidates and leave the model in the role of filter and ranker. The same principle applies to auditing reports, logs, and clinical data.

Story

Key Questions Answered

What Exactly Is Being Checked?

The project has two strands. The first is images. The author wrote a tool that looks for improper reuse of images both within a single paper and across different works. Test runs over several thousand papers have already turned up 40 cases, which he has posted to PubPeer, a site where scientists publicly discuss papers after publication. The second strand is data files: tables and raw measurements. Specialized tools search them for anomalies and artificially duplicated values.

Why Not Just Ask a Model?

The author argues that AI, by default, is poor at noticing subtle problems: repeated image fragments, suspiciously similar numerical series, calculation errors. So candidate detection is done by ordinary scripts, and the model is brought in afterward to weed out false positives and rank the findings by importance. For a developer, this is a familiar pattern. Think of checking logs or financial reports: deterministic code flags suspicious spots, and an LLM decides which of them are worth showing to a human.

How Findings Are Handled, and the Limitations?

The author posts some findings on PubPeer. For more serious cases, he writes to the authors and journal editors before any public announcement, so they have a chance to respond. He stresses that no report should be read as an accusation of intent: data duplication can be accidental. The limitations are clear. A human must confirm every finding, the benchmark sets for evaluation are still being prepared, and the flow of results is growing faster than one person can review it.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…