---
title: "Evaluate Caveman from a Cloudflare AI Gateway stack"
description: "Use a local development harness to evaluate Caveman before changing an application on Cloudflare. This guide moves the optimization experiment, not your Workers deployment or DNS. Caveman's loopback p"
canonical: https://caveman.so/switch/cloudflare-ai-gateway
last-updated: 2026-09-07
---

# Evaluate Caveman from a Cloudflare AI Gateway stack

Use a local development harness to evaluate Caveman before changing an application on Cloudflare. This guide moves the optimization experiment, not your Workers deployment or DNS. Caveman's loopback proxy needs a process on the same machine as its caller.

If the objective is a hosted production migration, first confirm the Caveman deployment and private-access contract. Do not use a laptop listener as a production endpoint.

## Inventory the Cloudflare dependency

Save the gateway ID, provider route, authentication method, cache settings, rate limits, and retry or fallback configuration. Record which settings are in code and which are managed in Cloudflare.

Identify whether the application uses Workers AI or an external provider. A direct OpenAI or Anthropic test requires that provider's credential and model access. A Cloudflare token does not become an external provider key.

Choose one task with a known answer and a permitted sample of its tool output. Keep one cached and one uncached baseline if caching affects your production workload.

## Run the task locally first

Start your existing application's test harness on a development machine. Confirm it can complete the task through its current Cloudflare path. Save provider usage and the final correctness result.

Install Caveman on that machine:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
caveman setup
```

A payload-only test leaves your provider path untouched:

```bash
caveman compress < captured-tool-output.json > candidate-view.txt
```

Inspect the view and verify the actual recovery handle against the original. This is the quickest way to determine whether your payload is eligible before creating a second network path.

## Create a direct-provider candidate

For an existing OpenAI-client test harness, start Caveman explicitly in compression mode:

```bash
caveman tools config set think.mode compress
caveman start
```

Then create a separate client:

```python
import os
from openai import OpenAI

candidate = OpenAI(
    base_url="http://127.0.0.1:8787/openai/v1",
    api_key=os.environ["OPENAI_API_KEY"],
)
```

Use a destination-provider model ID that matches the baseline model where possible. If the original model exists only through Workers AI, record the model change as a separate experimental variable. Do not attribute its effect to compression.

The direct candidate does not carry Cloudflare gateway authentication headers or inherit its limits. Keep it isolated from production callers.

## Map controls before a wider move

| Cloudflare behavior | Required decision before replacing the path |
| --- | --- |
| Response caching | Recreate the key, eligibility and expiry rules, or measure with it disabled in both arms |
| Rate limiting | Identify the enforcement point and rejection behavior |
| Retries and fallback | Preserve a total attempt budget and supported fallback models |
| Logging and retention | Keep permitted fields, access, and deletion rules |
| Workers deployment | Choose a supported runtime and reachable production endpoint |

A local proxy is not a replacement for these controls. The mapping is what makes a future migration reviewable.

## Validate cache and recovery separately

First run a cold request in each arm. Then run the repeated task that represents your warm path. Record whether the response came from a gateway cache, whether the provider reported cached input, and whether Caveman transformed eligible content.

Force a recovery query for omitted detail and check the final answer. On paths without native agent tooling, proxy-side recovery depends on supported protocol and request mode. Keep any pass-through behavior in the result.

Compare complete attempts using [the measurement guide](/guides/measure-agent-cost). Include latency and failed requests, and use the provider's usage for cost. A local inferred reduction is not a Cloudflare invoice adjustment.

## Roll back

Restore the original Cloudflare client and gateway settings in the local harness. Keep production configuration unchanged until the full deployment and policy checks pass. End candidate sessions before discarding their recovery data.

Read [Cloudflare's gateway documentation](https://developers.cloudflare.com/ai-gateway/), [Caveman's provider reference](https://docs.caveman.so/docs/proxy/providers), and [the comparison](/compare/cloudflare-ai-gateway) for the scope of each layer.