Labs / research
The zero-dollar dashboard
Abstract
Connect traffic to Caveman Cloud and the headline number reads $0.00, and it keeps reading $0.00 for a while. This report documents the accounting design behind that zero: a four-bucket model separating measured spend, inferred headroom, verified savings, and realized savings; a mutual-exclusion scheme that stops thirteen overlapping detectors from selling the same dollar twice; and a promotion ladder under which no estimate becomes a verified claim until an eval-gated optimizer has run in active mode on real traffic. The opening zero is the design working as intended: the headline is falsifiable, so it starts empty.
1. The projection problem
Most cost tools open with a projection. Connect your account, see what you could save; the number is large, unfalsifiable, and nobody is ever held to it. We open with a fact instead. Verified savings is a ledger of dollars that were demonstrably not spent, proven on the customer's own traffic, and on day one that ledger is empty because nothing has been proven yet.
The rule is mechanical rather than aspirational: verified_savings stays at zero until an optimizer runs in active mode on real traffic. No demo, benchmark, estimate, or opened pull request can move it. The rest of the product exists to move that number honestly.
2. Four-bucket accounting
Every report keeps four figures apart and never blurs them.
| Bucket | What it is | What can move it |
|---|---|---|
| Measured | Actual spend, priced to the cent from a dated provider catalog | Real traffic |
| Projected | Detector-estimated headroom, labeled inferred, expressed as a per-day rate | New telemetry evidence |
| Verified | Savings earned by an optimizer that ran active under an eval gate | The promotion ladder (§4) |
| Realized | The verified dollars that landed on the invoice | Arithmetic at billing time |
Two conventions in that table carry most of the honesty. When a request uses a model the pricing catalog does not know, it is priced at zero and tagged unpriced; a wrong-low number you can see beats a plausible guess you cannot audit. And a projected figure is always a per-day rate derived from a real measured column with a stated formula, never re-projected into a monthly number, because thirty times an estimate is still an estimate, just louder.
3. One dollar, one owner
Thirteen detectors read the telemetry, and several of them can see the same waste from different angles. Left alone, they would double-count it. Overlapping detectors therefore collapse into mutually exclusive families (input bloat, cache, reliability, routing), and at most one member of a family survives per agent, workflow, model, and day. The Cave Plan renderer then prints each move's prose with the same dollar band the move reports, so the words can never disagree with the number.
4. The promotion ladder
An optimization reaches the verified bucket only by walking the full rollout ladder: record, replay against fixtures, shadow, canary, then active, with guardrails that roll it back automatically on a quality regression or an error spike. When the headline finally moves, it moves because an eval-gated change ran on live traffic and held quality, and the result is recorded with a signed receipt that can be re-verified later, using the same attestation machinery CaveBench publishes under (see CaveBench methodology).
5. Discussion
A dashboard that opens at zero looks like a weakness until the number moves. Once it does, an engineering lead can hand it to a CFO without a footnote, because every dollar on it survived a test designed to keep it off. Projection-first tools cannot cross that bridge; their headline was never falsifiable to begin with. Ours could have stayed at zero forever, and that possibility is precisely what the number is worth.