---
title: "Caveman vs Not Diamond for model routing"
description: "Not Diamond focuses on choosing models and optimizing prompts. Caveman's routing work focuses on the economics of an agent request within its existing workflow, including the cost of leaving a warm ro"
canonical: https://caveman.so/compare/not-diamond
last-updated: 2026-09-07
---

# Caveman vs Not Diamond for model routing

Not Diamond focuses on choosing models and optimizing prompts. Caveman's routing work focuses on the economics of an agent request within its existing workflow, including the cost of leaving a warm route. For a local task with bloated context, Caveman also gives you a compression experiment that does not require changing the model.

Caveman Router is in private development. Evaluate it through a scoped pilot rather than planning an immediate public-service replacement. The local compression tools are available independently.

## The decision each product makes

[Not Diamond](https://docs.notdiamond.ai/docs/what-is-not-diamond) documents pretrained and custom model routing, prompt optimization, and API access for Python, TypeScript, and HTTP clients. Its model-selection work overlaps with Caveman's routing goal; the comparison should be about the workload and evidence, not whether one tool has a router at all.

| Question | Not Diamond | Caveman |
| --- | --- | --- |
| Which model should answer? | Pretrained or custom routing | Candidate selection within a declared capability and quality contract |
| Can prompts change? | Prompt-optimization product | Test compression separately from routing |
| What happens in a long agent task? | Evaluate the selected integration and policy | Cache and session economics are part of the routing evaluation |
| Can the gateway stay? | Depends on the application integration | LiteLLM integration can request a decision while LiteLLM retains inference |
| Availability | Account and API documented by Not Diamond | Router needs private access; local tools do not |

## Why session state changes the calculation

The cheapest eligible model for a new question may be expensive for the next turn of a long task. Switching can lose cached context, change tool behavior, or cause the model to repeat work. Those costs arrive after the apparently cheaper route decision.

Caveman's useful evaluation question is whether the whole remaining task gets cheaper at the required quality. A decision that keeps the current model can be the right outcome. The model price alone does not settle it.

This is an evaluation target, not a claim that every Caveman route achieves it. A pilot needs matched task evidence and clear behavior when the router cannot make a supported choice.

## Keep prompt optimization separate

A prompt optimizer can change wording, examples, or instructions. A model router changes which model receives the request. A local compressor changes eligible context. When all three move together, a passing result gives little information about which part helped.

Start with a fixed prompt and model pool. Then compare a routing candidate. Introduce prompt changes as another experiment, with the same grader. For tool-using agents, include argument validity and final side effects in the acceptance criteria.

If repeated logs dominate cost, begin with [prompt compression](/guides/prompt-compression) on the current route. You may remove waste without taking on a model-selection migration.

## What to ask for in a pilot

Use your current Not Diamond setup as one arm, a fixed strong model as another, and the Caveman candidate as a third. Keep the candidate pool and task set comparable. Include decision latency and all follow-up calls in spend.

Ask how timeouts, unsupported request fields, and unauthorized model candidates are handled. Inspect what prompt data leaves the caller for the decision. A router that works on ordinary chat may need to abstain on tools, structured output, or provider-specific features.

The [routing guide](/guides/model-routing) turns these questions into a concrete test plan.

## Choose from the measured workload

Keep Not Diamond when its existing policy meets your quality and operating needs. Try Caveman when cache-aware agent economics or a gateway-preserving decision path is the gap you need to investigate. The [switching guide](/switch/not-diamond) covers saved policy, a LiteLLM pilot, and rollback.

Sources: [Not Diamond overview](https://docs.notdiamond.ai/docs/what-is-not-diamond), [Caveman Router](/products/router), and [Caveman's LiteLLM integration](/switch/litellm). No cross-vendor cost percentage is claimed here.