---
title: "Evaluate Caveman from a Vercel AI Gateway setup"
description: "Keep your Vercel application and AI SDK code while you evaluate whether Caveman can reduce a specific agent workload. Hosting, framework choice, and model access are separate decisions. This guide cha"
canonical: https://caveman.so/switch/vercel-ai-gateway
last-updated: 2026-09-07
---

# Evaluate Caveman from a Vercel AI Gateway setup

Keep your Vercel application and AI SDK code while you evaluate whether Caveman can reduce a specific agent workload. Hosting, framework choice, and model access are separate decisions. This guide changes one development path at a time.

A local Caveman proxy cannot be reached from a hosted Vercel Function through `127.0.0.1`. Run the first experiment in a local harness. A production gateway migration needs a provisioned and supported destination.

## Save the gateway setup

Record the model string, provider preferences, fallback policy, and authentication method. Vercel deployments may use OIDC or gateway credentials; a direct provider trial needs the provider's own key instead.

Identify whether your caller uses the AI SDK's gateway model, the OpenAI-compatible API, or a coding-agent configuration. Keep that baseline callable so you can compare and restore it.

Save any coding-agent configuration changed by gateway setup. A Caveman launcher can temporarily change the provider endpoint, so verify the final route rather than assuming two setup tools compose automatically.

## First option: compress a tool result

Install the local Caveman tools:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
```

Test a command that produces the context you want to reduce:

```bash
caveman shrink -- npm test
```

This can leave the agent's existing gateway route intact. Use the agent's supported Caveman recovery tool before relying on a lossy result. For a saved fixture, retrieve the emitted handle and compare its bytes with the source.

Keep the Vercel model fixed during the experiment. If the same task now takes fewer complete model calls or less paid input, you have evidence for the context change without a gateway migration.

## Second option: a direct-provider caller

For a local client already using the OpenAI API shape, create a separate candidate:

```python
import os
from openai import OpenAI

candidate = OpenAI(
    base_url="http://127.0.0.1:8787/openai/v1",
    api_key=os.environ["OPENAI_API_KEY"],
)
```

Start Caveman before running it:

```bash
caveman tools config set think.mode compress
caveman start
```

Use the direct provider's model ID. Vercel gateway credentials and provider/model strings do not automatically map to the provider endpoint. Preserve the baseline's capability requirements, including structured output and tool calls.

This candidate does not inherit Vercel's gateway budgets or fallbacks. Keep it on an isolated development caller until each required behavior has an equivalent.

## If you use the AI SDK

You may be able to add Caveman at the framework boundary instead of changing the gateway client. The [AI SDK switching guide](/switch/vercel-ai-sdk) uses the separate experimental adapter and its exact upstream pin.

Read the manifest before installing. A newer AI SDK version is not covered just because its type names look familiar. The adapter's unsupported lifecycle and compilation states are real limits, and its presence alone does not enforce a budget.

Keep existing callbacks when composing the adapter. Verify one successful call, one failed call, and cancellation in your harness before using the resulting usage records.

## Check a full session

Start with a short connectivity request, then a tool-heavy task with a known result. Confirm model identity, tool-call arguments, stream completion, and the final output. Test a provider error to see how retries differ from the gateway baseline.

Record recovery calls and cached input. Use [the cost measurement guide](/guides/measure-agent-cost) to compare the entire task. A local report marked `inferred` does not prove a reduction in your Vercel account bill.

For proxy-only application paths, compression eligibility depends on supported recovery handling. Keep pass-through cases visible and validate the protocol mode used by the real application.

## Roll back

Restore the saved gateway client or coding-agent configuration and launch a new session. Remove only the experimental adapter or tool hook you added. Keep any recovery store while a trial conversation still refers to it.

Sources: [Vercel AI Gateway](https://vercel.com/docs/ai-gateway), [Caveman provider routes](https://docs.caveman.so/docs/proxy/providers), [Caveman adapters](https://docs.caveman.so/docs/agent-sdk/adapters), and [the comparison](/compare/vercel-ai-gateway).