Raspberry Pi 5 offline: a signed, poisoned knowledge pack passed every check

What happens to a local AI assistant that lives on a device with no network, but still has to be trustworthy? Researchers Georgios Feredzakis and Alexandros Papaspyridis tested this on a Raspberry Pi 5: a fully local RAG setup under network isolation, three sizes of Qwen2.5 (1.5B, 3B, 7B), three context sizes and 300 questions from SQuAD 2.0. The outcome: some safeguards work almost perfectly, while the most obvious one, the digital signature, turns out to be insufficient.

Five connectivity modes and knowledge packs instead of the cloud

Most generative AI deployments in IoT assume the cloud is within reach. Real devices often work intermittently, inside closed networks, or with no connectivity at all. For this setting, the authors proposed a C0–C4 connectivity scale, a taxonomy of inference-time trust mechanisms, and an update channel for disconnected modes: signed, versioned knowledge packs. The device receives a new pack, verifies it, and only then attaches it to the database from which the model pulls its context.

Where trust works and where it doesn't

The first mechanism is a gate based on the relevance scores of retrieved passages, calibrated empirically. It catches queries that have no match in the corpus almost perfectly: AUROC 0.97. It does noticeably worse with unanswerable questions inside a familiar topic, at AUROC 0.68. Such questions resemble the corpus documents, and retrieval scores cannot tell them apart from normal ones.

The second mechanism is a deterministic output validator. It turns "broken" answers into an explicit refusal, and it did so without costing the model a single correct answer. An NLI-based verification model, which the authors also tried, added little. For 7B, 96% of substantive answers to questions labeled "unanswerable" matched normalized fragments of the retrieved passages. The model does not make things up; it takes text from the context even when, formally, the context contains no answer.

A signature is not the truth

The toughest part of the study concerns updates. Signature and hash checks alone accepted 6 of 9 attacks: version rollback, replay, an expired pack, revoked keys, and incompatibility with the model. Multilayer verification rejected all 9 in roughly 4.5 ms.

But a poisoned pack with a valid signature passed every check and stripped the model of correct answers on all 30 targeted questions. The authors put it this way: integrity establishes provenance, not truth. Trust in the content has to be secured at the publisher before signing, not on the device. This test was run only on 3B, so the behavior of 7B remains an open question: in the main experiment, it refused more often to answer unanswerable questions.

Key figures from the study

• Platform: Raspberry Pi 5 with 16 GB of memory, network isolated
• Models: Qwen2.5 1.5B, 3B, 7B; three context sizes
• Data: 300 SQuAD 2.0 questions
• Retrieval-score gate: AUROC 0.97 for out-of-corpus queries, 0.68 for unanswerable in-topic queries
• Deterministic output validator: zero correct answers lost
• NLI verifier: small contribution
• 7B: 96% of substantive answers to unanswerable questions matched passage fragments
• 3B vs 7B: the same share of answers containing the gold reference, at less than half the latency and estimated energy; a 3B query takes about 22 s
• Peak resident memory at query completion: 6.3 / 9.4 / 13.6 GiB for 1.5B / 3B / 7B
• Update attacks: signature and hash let 6 of 9 through, layered verification rejected 9 of 9 in ~4.5 ms
• Poisoned signed pack: broke correct answers on 30 of 30 targets

What engineers should take from this

For offline devices, 3B is the more sensible choice over 7B: containment quality is the same, while latency and energy are half as high. The output validator is cheap and worth deploying right away. Updates need a monotonic version counter, an expiry date, key revocation and a model compatibility check; a signature alone is not enough.

Open questions remain: the authors acknowledge that the key holder and an authorized rollback to a known-good pack are not covered, that they found no ready-made update profile specifically for RAG, and that they generated the experiment's key on the device itself. Participants in the discussion suggest additionally running a fixed set of control questions before switching to a new pack. There is a window for that: embedding a pack on the Pi takes about 29 minutes.

All numbers reproduce with a single command, and the code and per-query outputs are published.