FieldGuideRAG checks photos against reference images in a graph and beats SigLIP by up to 16 accuracy points

Researchers at the National University of Singapore have proposed FieldGuideRAG, a training-free GraphRAG method that helps a multimodal model tell apart nearly identical species of animals, plants and fungi. Instead of a text description, each node in the graph stores a single real photograph of the species. A frozen VLM compares the query image with the photo of each candidate, then follows the graph's edges to pull facts about the identified species. On seven graphs, this verification is more accurate than retrieval with SigLIP embeddings, by 16 points in the best cases.

Why a text graph can't tell lookalikes apart

The underlying problem is a familiar one: VLMs describe everyday scenes well but make mistakes when categories differ only in small details. On fungi and butterflies from iNaturalist, the tested models rarely name the correct species without a hint. Existing multimodal GraphRAG systems such as mKG-RAG, MMGraphRAG, M3KG-RAG and MegaRAG build graphs from images and documents, but they still describe every entity in words. Words convey poorly how complete the ring around an eye is, or what the streaks on a bird's breast look like. Wikipedia descriptions usually open with size and general shape, which are exactly the traits that similar species share.

A multi-image prompt that puts all candidate photos into a single query is not a way out either: VLMs are weak at comparing many pictures at once, and such a prompt has no links leading from a species to further knowledge.

How the pipeline works

The design mirrors a printed field guide: a photo of the species next to its lookalikes, a range map and notes on behavior. A node carries a name, a photo, optional text and a record of facts. Edges link a species to its relatives (genus and, on iNaturalist, family), to the latitude-longitude grid cells where the species was photographed, and to ecological facts. For the birds in CUB-200, facts come from the AVONET database: diet, migration, habitat, foraging method and body mass. The range is computed from about 50 geotagged photos per species from the iNaturalist 2021 mini training set, with no test images.

The steps are as follows. SigLIP finds the entry nodes, and kinship edges add similar species to the pool of lookalikes. For each candidate, the VLM receives two images and a question: are these the same species? The score is the probability of a "yes" answer, computed from two logits at the first generated token. This is a single forward pass without decoding, so candidates can be checked in parallel. The verification score is then combined with the probability from the range cell. The factual answer is taken from the edges of the predicted species.

The experiments ran on two frozen models, Qwen2.5-VL-7B and Qwen3-VL-8B, on CUB-200, a graph built from Wikidata photos, and four iNaturalist domains: fungi, birds, Lepidoptera and plants.

What the measurements show

On every graph, a photo on the node improves verification compared with the species name alone. Wikipedia text descriptions and CUB expert attributes give almost the same result as the name. Verifying the candidate pool is more accurate than the SigLIP retriever on all graphs for both models. On domains where the VLM barely knows the species from memory, it is also more accurate than the multi-image prompt. Text RAG improves on the zero-shot result but, on CUB-200 and the Wikidata graph, trails both methods that read photos.

The range edge adds 7 to 19 points on iNaturalist because it supplies information that embedding similarity does not contain. Answers to ecological questions via fact edges are correct, at best, for 89.5% of 28 820 questions. There is one more effect: the maximum verification score separates queries whose species was removed from the graph better than SigLIP similarity does. This is useful in practice, because a real catalog always contains unknown species.

Limitations

The results come from datasets with public catalogs, not from field conditions. A single random photo per node may turn out to be atypical, and the range edge applies only to queries with GPS coordinates, as on iNaturalist; CUB-200 has no locations. The work has also not been peer-reviewed: for now it is a preprint. The authors promise to publish the graphs, prompts and code.

For developers of multimodal RAG, the takeaway is practical: when the answer depends on subtle visual differences, a graph is better off storing reference photos and comparing them pairwise than piling up text descriptions.