---
title: "AI gateway migration: a reversible test plan for agent traffic"
description: "A gateway migration changes more than a base URL. It can change model naming, credential ownership, policy enforcement, retries, caching, streaming events, and the usage records your team relies on."
canonical: https://caveman.so/guides/gateway-migration
last-updated: 2026-09-07
---

# AI gateway migration: a reversible test plan for agent traffic

A gateway migration changes more than a base URL. It can change model naming, credential ownership, policy enforcement, retries, caching, streaming events, and the usage records your team relies on.

Start with one staging caller. Preserve the current path until the candidate has passed the request and policy checks that matter to your application.

## Save the current contract

Record the caller's endpoint, model alias, authentication source, SDK version, API protocol, headers, tool definitions, structured-output settings, and timeouts. Include gateway budgets, allowed models, provider pins, fallbacks, plugins, and cache policy.

Keep a reproducible task and expected result. Use staging credentials and disposable tools for side-effecting cases. Retain enough source records to compare the candidate's behavior.

The vendor-specific [switching index](/switch) covers the configuration boundaries for each alternative.

## Choose the scope of the candidate

| Candidate | What changes |
| --- | --- |
| New gateway deployment | Gateway runtime, configuration, and possibly credential custody |
| Local Caveman proxy | Supported request path through a loopback process |
| Caveman compression beside an existing gateway | Context path, with gateway policy preserved only where traffic still traverses it |
| Caveman Router pilot | Eligible model selection with private access |
| Caveman Platform pilot | Supported evidence collection and review workflows |

Do not treat these as one migration. A direct-provider local trial can answer a compression question while providing no evidence about replacement gateway governance.

## Start a local Caveman transport check

Install the CLI and companion binaries on the same reachable host as the test caller:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
```

Create a dedicated `caveman-test.yaml`:

```yaml
label: gateway-migration-test
mode: record
listen: 127.0.0.1:8787
providers: {}
compat: {}
```

Start it in a separate terminal on a test machine where the port is available:

```bash
caveman start --config ./caveman-test.yaml
```

Record mode keeps the request unchanged. If the port already serves another session, use an isolated environment rather than restarting shared infrastructure.

## Verify a direct-provider mount

For an OpenAI Chat Completions transport check, supply `OPENAI_API_KEY` through your secret mechanism and set `TEST_MODEL` to an actual supported model your account can use. This example makes a real provider request when you run it:

```js
// Save as gateway-check.mjs and run with Node.js.
const apiKey = process.env.OPENAI_API_KEY;
const model = process.env.TEST_MODEL;
if (!apiKey || !model) {
  throw new Error("Set OPENAI_API_KEY and TEST_MODEL first");
}

const response = await fetch(
  "http://127.0.0.1:8787/openai/v1/chat/completions",
  {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${apiKey}`,
    },
    body: JSON.stringify({
      model,
      messages: [{ role: "user", content: "Reply with: transport checked" }],
      stream: false,
    }),
  },
);
console.log(response.status, await response.text());
if (!response.ok) process.exitCode = 1;
```

This is an isolated direct-provider path. It does not carry your previous gateway's virtual-key restrictions, budget policy, or fallback configuration. Follow [provider mounts](https://docs.caveman.so/docs/proxy/providers) for other protocols and [LiteLLM switching](/switch/litellm) for its documented combined paths.

## Test the request shapes that matter

After basic transport works, exercise tools, structured output, images if used, streaming, cancellation, provider errors, and model access failures. Confirm the same model deployment and required fields reach the provider.

Check where retries occur. A stream that partially delivered output cannot always be retried like an untouched request. Keep tool side effects and duplicate calls visible.

For governed traffic, send a deliberately forbidden model request and verify refusal. A migration has failed if it quietly bypasses the policy you intended to keep.

## Enable compression only with recovery

Stop only your own test listener, then restart it with an explicit per-process mode:

```bash
CAVEMAN_MODE=compress caveman start --config ./caveman-test.yaml
```

Run a task with eligible repeated content and a required omitted detail. Confirm a supported recovery path is available. Non-streaming API-key and agent-side recovery paths have different eligibility; unsupported cases can pass through.

A successful request does not prove a transform applied. Inspect local reports and the actual task result using [the compression guide](/guides/prompt-compression).

## Reconcile usage and policy before cutover

Compare request counts, model identity, input, output, cache fields, latency, errors, and final acceptance results. Investigate duplicate spans, retries, missing usage, and price-source differences.

Keep content collection and retention explicit. A metadata-only trial cannot prove analyses that require captured message bodies.

Use [cost per completed task](/guides/measure-agent-cost), not the first request's apparent discount, to judge the economic result.

## Cut over one caller and retain rollback

Move a bounded caller or task family after all required checks pass. Preserve the prior endpoint, model string, credential source, and policy configuration. Define a trigger for restoring them if errors, latency, or quality regress.

Rollback should start fresh sessions through the saved route and verify the expected provider and policy. Remove candidate configuration only after no active request or recovery handle depends on it.

For a full gateway replacement, use [the selection guide](/guides/choose-ai-gateway) to confirm deployment, governance, and operating-cost requirements beyond the local trial.