A Check of 600 Links Found That Up to 94% of AI Sources Remain Accessible

A broken link beneath an AI-generated response may look like damning evidence: the page did not open, so the model must have invented the source. But that logic is too convenient and often wrong. The study's author collected about 600 URLs from the responses of five AI-powered search services and checked them again roughly a month later. Depending on the service, 88% to 94% of the links opened. No confirmed fabrications were found among the inaccessible URLs: some pages denied access to the automated script, while several articles appeared to have been deleted, judging by their response codes and URL structures.

This does not exonerate AI systems. It changes the diagnosis. AI search tools face at least two distinct problems: a model may generate a plausible URL for a nonexistent page, or it may cite genuine content that disappears later. For developers building verification systems, these are fundamentally different cases.

Why a 404 Error Proves Nothing

Suppose an AI system cites a service review at example.com/blog/review. A month later, the server returns 404. There are two possible explanations: the article never existed, or the publisher deleted it. The response code alone cannot distinguish between them.

A 410 code provides more information: the site owner explicitly states that the resource has been permanently removed. This is strong evidence that the page previously existed. Codes 401 and 403 usually indicate denied access, 429 means the request limit has been exceeded, and 503 signals temporary service unavailability. None of these responses should automatically be classified as a dead link: a regular browser may open a page that the site refused to show an automated script.

A basic check must therefore account for redirects, the final address, and the server response class. The experiment used a GET request with separate connection and read timeouts, a custom User-Agent, and a limit of ten redirects. URLs returning 404 and 410 were classified as removed; responses 401, 403, 429, and 503 were classified as blocked for the bot; all other ambiguous cases were sent for manual review.

A Live Link Can Also Mislead

A 200 status confirms only that the server returned something. It does not prove that the page contains the promised article or that its text supports the AI system's conclusion. The site may have redirected an old address to its homepage, replaced the article, or retained the URL with different content.

Reliable source verification therefore has three stages: open the final address, confirm that the title and content match the citation, and then assess the reliability of the source itself. An automated link checker handles only the first stage. For a product that shows links to users, it is useful to store the date of the latest check, the final URL, the response code, and a fingerprint of the page content. This makes it possible to distinguish a temporary outage from the deletion or replacement of an article.

What Developers and Site Owners Should Do

Developers of AI search tools should not label a link as fabricated after a single failed request. They need retries, separate handling for access restrictions, and manual review of ambiguous URLs. If the answer must be auditable, the system should retain the date on which the source was accessed and a short excerpt supporting the conclusion, subject to copyright rules.

When site owners delete an article, they should preferably set up a permanent redirect to current content on the same topic. Redirecting to the homepage formally keeps the address “live,” but breaks the promise made to the reader. If no suitable replacement exists, returning 410 is more honest.

The study has a notable limitation: it is a single measurement taken roughly one month after the links were collected. It tested page availability, not the accuracy of the summaries or the quality of the sources. The conclusion is therefore narrow: a broken link alone does not prove hallucination. Verification must examine the URL's history, the page's content, and whether it actually supports the AI system's claims.