---
title: "Evaluate Caveman from a Martian gateway or router"
description: "First identify which Martian feature your application uses. A fixed model through Martian Gateway and a policy that selects models are different migration problems. Save a working baseline for the exa"
canonical: https://caveman.so/switch/martian
last-updated: 2026-09-07
---

# Evaluate Caveman from a Martian gateway or router

First identify which Martian feature your application uses. A fixed model through Martian Gateway and a policy that selects models are different migration problems. Save a working baseline for the exact path before adding Caveman.

This guide supports a local compression experiment and preparation for a private-access routing pilot. It does not transfer Martian credits, credentials, or custom routing state.

## Record the request contract

Save the gateway URL, authentication method, model string, and request options. If you use LiteLLM, record the deployment alias separately from its underlying model string.

Martian's documentation shows that a LiteLLM model can include two provider-related prefixes: one for LiteLLM's transport and one for Martian's provider selection. Keep the full string in the baseline record so the mapping is reviewable.

Note whether an optimized alias is part of the request. If it can change behavior or model selection, compare it as a policy rather than labelling it a fixed-model baseline.

## Add a separate LiteLLM test alias

For a direct OpenAI-provider compression trial, you need an OpenAI key and a supported direct model ID. Keep the existing Martian alias in place and add a candidate alias to a test configuration:

```yaml
model_list:
  - model_name: caveman-direct-test
    litellm_params:
      model: openai/gpt-5.5
      api_base: http://127.0.0.1:8787/openai/v1
      api_key: os.environ/OPENAI_API_KEY
```

The model is an example; match the baseline's actual model and capabilities where possible. Do not send a Martian key or its nested provider string to the direct provider.

Run Caveman on the same host or network namespace as the test LiteLLM process:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
caveman tools config set think.mode compress
caveman start
```

The loopback listener is not a hosted gateway. Keep this experiment off production traffic until deployment and policy requirements are satisfied.

## Verify the protocol before the cost

Make a short request to check authentication and model resolution. Then run the actual tool-heavy workload. Check tool-call arguments, structured output, streaming completion, and errors if the application uses them.

On a proxy-only path, eligible non-streaming API-key calls can use supported proxy-side recovery. Streaming may pass through where that recovery is unavailable. A working request does not by itself prove that compression occurred.

If you only want to test one payload, keep the Martian provider route and run a saved tool output through Caveman first:

```bash
caveman compress < tool-output.json > candidate-view.txt
```

Recover the real emitted handle and compare its bytes with the original before wiring the view into an agent.

## Compare a routing policy separately

Caveman Router needs private access. Agree on a model pool, endpoint, and quality requirement before enabling a candidate. The [LiteLLM integration](/switch/litellm) can request a route decision while leaving inference and keys in LiteLLM.

Do not run two automatic selectors in sequence without understanding the final model identity. If Martian selects a model after another router selected an alias, the first decision may not describe the actual provider call.

Use a fixed-model arm, your current Martian arm, and the candidate on the same held-out tasks. Record all decision overhead and subsequent attempts, including failed tasks. The [routing guide](/guides/model-routing) covers cache and continuation costs.

## Inspect account and policy changes

The direct candidate does not inherit gateway credits, usage limits, logging policy, or provider fallback behavior. List which controls are required before replacing the current path. Keep model access and region restrictions explicit in every candidate pool.

Use provider usage for cost and keep Caveman's local inferred reduction separate. A lower token estimate cannot establish an account-level saving without comparable charged usage.

## Roll back

Point the caller back to the saved Martian alias and configuration. Start a fresh conversation if the trial used local recovery handles. Leave the original gateway setup and its records intact until you have decided how to retain historical evidence.

Sources: [Martian's LiteLLM guide](https://docs.withmartian.com/integrations/litellm), [Caveman's provider routes](https://docs.caveman.so/docs/proxy/providers), and [the comparison](/compare/martian).