---
title: "Test Caveman beside a Portkey gateway"
description: "Start beside Portkey if your application relies on its guardrails or routing configuration. You can test Caveman's command-output compression without removing those controls. A full gateway replacemen"
canonical: https://caveman.so/switch/portkey
last-updated: 2026-09-07
---

# Test Caveman beside a Portkey gateway

Start beside Portkey if your application relies on its guardrails or routing configuration. You can test Caveman's command-output compression without removing those controls. A full gateway replacement requires a separate decision for every policy you currently enforce.

This guide provides a working local trial and a checklist for moving one development caller. It does not claim automatic import of Portkey configurations or historical logs.

## Record what the gateway does today

Save the caller's base URL and its Portkey configuration reference. List the virtual keys, provider mapping, fallback order, caching behavior, and request or response checks that affect the task. Include settings applied in the dashboard rather than in source code.

For each guardrail, save one allowed example and one rejected example. For budgets or rate limits, record the behavior the caller expects when the limit is reached. Those examples become migration checks.

Keep credentials in your existing secret store. An exported configuration with a secret value should not become a committed fixture.

## Try compression before the gateway

Install Caveman on a development machine:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
```

For an agent that reads a noisy command, test that command through Caveman while leaving the agent's Portkey provider setup intact:

```bash
caveman shrink -- npm test
```

Use your actual test command. The wrapper keeps the command's exit status and can attach a recovery handle to an eligible smaller view. Make recovery available through the agent's documented Caveman tool setup before relying on a lossy result in an unattended workflow.

A saved output gives a repeatable byte check:

```bash
caveman shrink --file test-output.txt
caveman retrieve YOUR_RECOVERY_HANDLE > recovered.txt
cmp test-output.txt recovered.txt
```

Replace the placeholder with the handle returned by that run. If the input passes through, record that instead of trying to retrieve a nonexistent handle.

## If you need a direct-provider trial

Create a separate test caller with a direct provider credential. Portkey virtual keys and gateway headers do not become provider credentials. Resolve the provider model ID and preserve the original Portkey client as the baseline.

For the OpenAI-compatible Python client, the Caveman side uses the local OpenAI mount:

```python
import os
from openai import OpenAI

candidate = OpenAI(
    base_url="http://127.0.0.1:8787/openai/v1",
    api_key=os.environ["OPENAI_API_KEY"],
)
```

Start a local listener in compression mode before calling it:

```bash
caveman tools config set think.mode compress
caveman start
```

Keep this candidate on a development host. It does not inherit Portkey's guardrails, budget policy, virtual-key scope, or fallback settings. Do not send production traffic until every required control has an implemented and tested equivalent.

## Check ordering and failure behavior

Run the saved allowed and rejected examples. If compression happens before a guardrail, verify that the check still sees the information it needs. Test a tool failure and a provider timeout. Count actual attempts to catch duplicated retries.

Then run a real agent task. Inspect tool-call arguments, stream completion, and the final accepted result. Recovery eligibility depends on the protocol and client path; an unchanged request is not evidence that a transform succeeded.

Use [the measurement guide](/guides/measure-agent-cost) to compare total provider usage, including attempts that fail. Keep local inferred reductions separate from Portkey's usage records and any gateway fee.

## Expand only the winning path

If command-output compression helps, keep Portkey in place and roll out that narrow change. If you need Platform analysis, [discuss a Caveman pilot](/contact) and agree on the data and integration contract before moving telemetry.

There is no documented general-purpose Portkey-to-Caveman configuration importer here. Treat a wider gateway move as a policy migration, with explicit acceptance tests for each required behavior.

## Roll back

Restore the saved command hook or use the unwrapped command. For a direct-provider trial, restore the Portkey client and its configuration reference. Begin a fresh conversation if the candidate introduced recovery handles, and retain the local store until old trial sessions finish.

Sources: [Portkey gateway configs](https://portkey.ai/docs/product/ai-gateway), [Portkey guardrails](https://portkey.ai/docs/product/guardrails), [Caveman recovery](https://docs.caveman.so/docs/proxy/recoverable), and the [comparison](/compare/portkey).