LLM Pipeline Turns Datasheets into Knowledge Graphs with 98.8% Accuracy

Researchers have published an arXiv preprint describing a pipeline that turns electronic component datasheets into hierarchical knowledge graphs. On 20 documents of varying length and with different component types, the share of faithful properties (property faithfulness) reached 98.8%. A naive single-pass approach to the same task scores 76.1%. In other words, almost a quarter of the parameters the baseline extracted from a document were not supported by the document itself.

Why a single query is not enough

Datasheets are hard for machines to read. They are long and heterogeneous, and parameters are scattered across tables, charts, and running text. The authors point out an unpleasant detail: even when a document fits entirely into the model's context window, single-pass processing misses relevant specifications. For an engineer, this is not an abstract metric. A missed limit on voltage, current, or temperature range leads to the wrong component choice and a board respin.

How the pipeline works

The pipeline runs in three stages: chunk-based property extraction, hierarchical routing, and iterative refinement. The LLM handles information extraction and semantic reasoning. It does not edit the graph directly. Instead, it generates instructions, which are validated and then applied to the graph by deterministic local code.

This design has two effects:

• each query carries less context than processing the whole document at once;
• intermediate representations stay traceable, and each property keeps a link to its source (provenance).

The output is a graph of semantically organized properties tied to the datasheet text. It can be used for downstream search and reasoning.

The numbers

Without a complete ground truth of labeled properties for all 20 documents, measuring completeness is difficult. The authors therefore introduced a metric called inter-graph completeness, which compares the semantic coverage of two graphs against each other. The results:

• property faithfulness: 98.8% for the proposed framework versus 76.1% for the single-pass baseline, meaning the share of unfaithful properties fell from 23.9% to 1.2%, roughly a 20-fold reduction;
• forward coverage: the framework fully covers 83.6% of the baseline's faithful properties;
• reverse coverage: the baseline fully covers only 10.5% of the framework's properties;
• sample: 20 datasheets with different component types and document sizes.

The second pair of figures shows that the accuracy gain was not bought at the cost of coverage. The graph contains almost everything the single-pass approach finds, and much more besides.

Where this could be useful, and the caveats

The practical value is clear for teams building search and compatibility checking on top of component databases. If properties sit in a graph with source attribution, an engineer or an agent can quickly check where a value came from and compare it with the document. For automated design verification, this matters: an error in an extracted parameter propagates to every downstream check.

The core pattern can also be reused outside electronics. The LLM proposes changes to the structure, a validator checks them, and ordinary code executes them. That way, model errors do not reach the store directly.

There are caveats, too. A sample of 20 documents is small. The completeness metric is relative: it compares two graphs against each other and does not measure coverage against a full ground truth. The abstract discloses neither the models used nor the cost or processing time per datasheet. Those figures will determine whether the approach is viable for bulk processing of catalogs with thousands of documents. For now, the result shows that staged processing with verifiable steps is more reliable than a single large query to a model.