LLM gateway · proven cache
A semantic cache you can prove is correct.
The whole industry reports a cache hit rate. Almost nobody reports cache-hit precision— so a “90% hit rate” can quietly include wrong answers. Veritas measures the metric they omit, and gates CI on it.
What is the capital of France?
Which city is the capital of France?
same answer — safe to reuse
Is coffee good for your health?
Is coffee bad for your health?
opposite answer — must not reuse
No threshold keeps the paraphrase and rejects the negation — the negation embeds closer than the paraphrase. Both clear τ — a threshold alone would serve the negation a wrong cached answer. The two-tier guard reads intent, not distance, so it keeps one and strikes the other.
- 1.00
- cache precision (guarded)
- 0%
- false-positive rate
- 0.97
- recall at 0% FP
- 3.0×
- judge vs. threshold
Why this matters
A fast wrong answer is worse than a slow right one.
A semantic cache turns a paraphrase into a free, instant answer — but two questions can embed close yet need opposite answers. The only honest way to ship one is to measure how often it serves the wrong thing, and refuse to if it's too often.
01Measured, not asserted
An adversarial golden set of 75 pairs — paraphrases that should hit, negations and scope-flips that must not. The harness reports precision, recall, and a false-positive rate, and the CI gate fails the build if the cache would serve a wrong answer.
02A two-tier guard
A keyless deterministic check catches polarity, scope, and number flips for free; a Haiku judge catches the semantic inversions cosine can't see. Together they hold precision at 1.00 across the whole threshold curve.
03Honest failure
Provider failover commits on the first token — a failure before it is rescued by the chain; after it, the partial answer plus an error is surfaced, never a faked seamless retry. The wall is documented, not hidden.
Two-tier cache + guard
See it prove itself.
A guided tour over the real gateway: a paraphrase reused at $0, a negation correctly blocked, an outage rescued by failover — with a live metrics strip.