arXiv paper finds five flaws in XAI faithfulness metrics

In ten years, explainable AI has failed to deliver on its main promise: it has not taught people to trust models sensibly. That is the conclusion of the position paper arXiv:2610.12493 (cs.AI, version v1, dated 12 October 2026). The authors blame the failure on the poor performance of the faithfulness metrics used to evaluate explanations. They name five significant flaws in these metrics and propose four research directions that could make them more reliable.

Trust did not grow

The industry's original logic looked simple. If you show a person which features influenced a prediction, they will understand the model better and trust it to the right degree. In the authors' assessment, the evidence accumulated over the past decade points the other way. Explanations pushed users toward excessive trust in incorrect predictions. When the errors came to light, a backlash followed: algorithmic aversion.

As a result, XAI settled into a different role. It is primarily a debugging tool. It yields anecdotal observations, for example about which features a model actually learned. That is useful for an engineer. As a trust mechanism for an end user or a regulator, it works noticeably worse.

Where faithfulness breaks down

One of the main causes, in the authors' view, lies in the nature of explanations themselves. Methods often oversimplify a model's inner workings, which creates a mismatch between the explanation and the model. This is called unfaithfulness, or infidelity. Faithfulness here means the degree to which an explanation approximates the model.

If faithfulness could be computed rigorously, it would give a quantitative measure of one element of an explanation's reliability. But according to the paper, existing metrics make such a measurement either infeasible or computationally expensive.

Key parameters of the work:

• Format: a position paper, not an experiment or benchmark.
• Subject: faithfulness metrics for feature-importance explanations.
• Diagnosis: five significant flaws in current metrics.
• Proposal: four possible research directions.
• Core problem: measuring faithfulness is either infeasible or too computationally costly.

The published abstract does not list the five flaws or the four directions. For their content, you need to read the full text of the paper. We will not guess at them.

What practitioners should do

The practical takeaway for teams that use feature-attribution maps and similar tools is fairly sober. Treat an explanation as a hypothesis about the model's behavior, not as proof. A good-looking picture says nothing about how closely it matches the actual computation inside the network.

XAI remains useful for debugging. You can use it to look for data leaks, spurious correlations, and odd features. For products that show an explanation to a user or an auditor, it is better not to rely on a single faithfulness metric and to budget compute for verification up front. Where trust matters because of the law or the cost of errors, it is worth testing separately how people react to wrong predictions with and without an explanation. That is where, according to the evidence the authors cite, the skew toward blind trust appears.

For researchers, the signal is different. Until there is a reliable, inexpensive faithfulness metric, it is hard to compare explanation methods with one another. That means claims of the form "our method is more accurate" should be read with a caveat.