activerag — Evidence-Aware Hunt-and-Retry for RAG
Detects thin evidence in an answer, ranks where to hunt for more, retries once inside a bounded cycle, and audits every attempt — proven against the real 321k-chunk RFC corpus.
- 53 tests
- unit-tested independent of a live corpus measured
- ~35s
- live registry rebuild against the real 321k-chunk corpus measured shared Qdrant client; live-verified
- 3 triggers
- independent low-evidence signals (citations, dropped claims, router margin) measured
Context
Sometimes the first retrieval pass just does not surface enough to answer well. The wrong move is to answer anyway. activerag adds a bounded second chance: notice the evidence is thin, go looking for more in the right place, and try once more — then stop and show its work.
What I built
- Thin-evidence detection from three independent signals — missing citations, dropped claims, and a weak router margin.
- A ranked hunt for where additional evidence is most likely to help.
- A bounded retry — one more pass, never an unbounded loop.
- An audit trail of every attempt, so the behavior is inspectable.
Why it matters
It layers on top of Consilium’s citation signals and is verified against the real 321,124-chunk RFC corpus (a live registry rebuild runs in about 35 seconds with a shared Qdrant client), with 53 unit tests that run without a live corpus. Knowing when not to answer, and what to do about it, is a reliability property most RAG stacks lack.