---
title: "Prompt compression for AI agents: setup, recovery, and measurement"
description: "Prompt compression reduces the material a model receives. For an agent, the best target is often repeated tool output: logs, search results, repository files, or structured records that remain in the "
canonical: https://caveman.so/guides/prompt-compression
last-updated: 2026-09-07
---

# Prompt compression for AI agents: setup, recovery, and measurement

Prompt compression reduces the material a model receives. For an agent, the best target is often repeated tool output: logs, search results, repository files, or structured records that remain in the conversation for several turns.

The useful question is whether the agent finishes the same task with less total usage. This guide starts with a local fixture, verifies recovery, then moves to a checked agent session.

## Choose the right kind of reduction

| Method | Good fit | Main check |
| --- | --- | --- |
| Remove formatting or redundant structure | Repetitive structured content | Values and meaning survive |
| Select relevant sections | Long evidence with a focused question | Required detail remains available |
| Shorten generated answers with instructions | Verbose agent replies | Commands, reasoning needed by the user, and facts remain clear |
| Summarize a conversation | Long-running sessions | Decisions and unresolved work survive |
| Cache a stable prefix | Repeated identical context | Provider reports actual reuse |

These methods are not interchangeable. A shorter-answer skill does not prove model input was compressed. Provider caching can lower the price of repeated input without removing it. Read [the caching guide](/guides/prompt-caching) before changing a prefix that already gets good reuse.

## Start with a saved tool result

Use a fixture you are permitted to retain. Install the CLI and inspect the result:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
caveman shrink --file ./fixtures/tool-output.json
```

For a stdin pipeline:

```bash
caveman compress < ./fixtures/tool-output.json > tool-output.view.txt
```

Read the emitted view and report. Check what stayed, what disappeared, and whether a recovery handle was produced. A small or unsuitable input may pass through unchanged. Record that outcome rather than forcing a reduction to make the example look successful.

Do not pipe a lossy view into a consumer that expects the complete original schema. A model-facing evidence view and an application data object have different contracts.

## Verify the original before relying on the view

Caveman stores the original before publishing a lossy view. If storage fails or the configured budget is full, eligible new transforms pass through instead of invalidating existing recovery handles.

Copy the actual handle emitted by your run into the command below. `YOUR_RECOVERY_HANDLE` is a placeholder:

```bash
caveman retrieve YOUR_RECOVERY_HANDLE > recovered-original.json
cmp ./fixtures/tool-output.json recovered-original.json
```

Retrieval without a query returns the stored original byte for byte. A query requests a relevant view of the original:

```bash
caveman retrieve YOUR_RECOVERY_HANDLE "connection pool"
```

Use full retrieval when record order, exact bytes, or completeness matters. Query retrieval is another selection, not proof that every original detail was returned.

The host store persists locally. Its default payload budget is 512 MiB; existing handles are retained at capacity. WASM storage has a different lifetime because its backend is in memory. See [the recovery contract](https://docs.caveman.so/docs/proxy/recoverable) for paths and overrides.

## Give the agent a working recovery path

A handle in a prompt is not enough. The agent needs a supported tool or proxy-side mechanism to retrieve the content, and that mechanism needs access to the store.

For a supported local coding agent:

```bash
caveman doctor claude
caveman claude
```

Run a task that asks for a detail likely to be omitted from a large result. Verify the agent retrieves it and produces the correct answer. Streaming and subscription-auth paths need agent-side recovery where required; eligible non-streaming API-key paths can use supported proxy-side handling.

If the agent and store run on different hosts, resolve that boundary explicitly. A local handle does not create remote storage or grant another process access.

## Keep a direct baseline

Run the same task directly and through Caveman in separate fresh sessions. Keep the model, reasoning setting, tools, task prompt, and initial workspace fixed.

Check the final artifact independently. For a coding task, run relevant tests and inspect the patch. For extraction, compare required fields with known answers. Count every model request, including recovery, retries, and repair work.

Do not select only the content type where the compressor performs best. Include short output, long structured data, logs, and a case requiring an exact earlier detail. The [measurement guide](/guides/measure-agent-cost) explains how to summarize those results.

## Read the result correctly

| Observation | Supported conclusion |
| --- | --- |
| Smaller local token count | This representation is smaller under that tokenizer |
| Exact recovery passes | Stored original remains available |
| Agent task passes with lower input | Useful result for that task and configuration |
| Provider charges fall in a controlled comparison | Evidence about billed cost under that comparison |
| One benchmark wins | Reason to test similar work, not a universal guarantee |

Compression can lose overall if it breaks cache reuse, causes extra turns, or omits a fact the agent never retrieves. Keep those cases in the report. Apply the candidate where complete-task results support it.

## Choose an integration

Use [coding-agent setup](/guides/coding-agent-setup) for supported local launchers, [gateway migration](/guides/gateway-migration) for a controlled API-path trial, and [SDK migration](/guides/agent-sdk-migration) for agents you build yourself.

For alternative compressors, compare [Headroom](/compare/headroom), [LLMLingua](/compare/llmlingua), and [RTK](/compare/rtk). Their integration and recovery contracts matter as much as a compression ratio.