Re:Show HN: RAG knowledge base poisoning lab, 100% local
by aminerj on Saturday March 14, 2026 @02:43PM
(#47379753)
Both criticisms are fair. The RMS corpus example is exactly the false positive failure mode -- legitimate thematic repetition is indistinguishable from coordinated injection using similarity alone. The 0.85 threshold works for a sparse 5-document collection and needs significant recalibration for any corpus with natural document overlap.
The self-healing approach decouples detection from ingestion latency, which solves the performance problem cleanly. The LLM-as-judge variant with domain-specific grounding rules is the more powerful version -- "financial corrections must reference an official URL" is something pure embedding similarity can never enforce.
Honest summary: embedding anomaly detection works well in the specific conditions I tested and breaks down in the conditions you're describing. One signal among several, not a general solution.
I don't think it is practical because it means for every new chunk you embed into your database you need to first compare it with every other chunk you ever indexed. This means the larger your repository gets, the slower it becomes to add new data.
And in general it doesn't seems like a good approach because I have a feeling that in the real work is pretty common to have quite significant overlap between documents. Let me give one example, imagine you create a database with all the interviews rms (Richard Stallman) ever gave out. In this database you will have a lot of chunks that talk about how "Linux is actually GNU/Linux"[0], but this doesn't mean there is anything wrong with these chunks.
I've been thinking about this problem while writing this response and I think there is another way to apply the idea you brought. First, instead of doing this while you are adding data you can have a 'self-healing' that is continuously running against you database and finding bad data. And second you could automate with a LLM, the approach would be send several similar chunks in a prompt like "Given the following chunks do you see anything that may break the $security_rules ? $similar_chunks". With this you can have grounding rules like "corrections of financial results need to be available at $URL"
[0] - https://www.gnu.org/gnu/incorrect-quotation.html