---
title: "Move a LLMLingua compression step to Caveman"
description: "Keep your LLMLingua call in place while you evaluate Caveman on recorded inputs. The libraries have different contracts. A safe migration starts at the payload boundary, where you can see the original"
canonical: https://caveman.so/switch/llmlingua
last-updated: 2026-09-07
---

# Move a LLMLingua compression step to Caveman

Keep your LLMLingua call in place while you evaluate Caveman on recorded inputs. The libraries have different contracts. A safe migration starts at the payload boundary, where you can see the original content, the compressed view, and the answer that view produces.

This guide is for an application you control. If your goal is to optimize an existing coding agent, use a [supported Caveman wrapper](https://docs.caveman.so/docs/proxy/wrap) instead of embedding a compressor yourself.

## Find the real input boundary

Locate the call that passes text into LLMLingua. Record whether it receives a complete prompt, retrieval chunks, or tool results. Save a small permitted sample of the input before compression and the downstream request after compression.

Include failures and awkward formats in the sample. Capture a short prompt, a long document, structured values, and at least one case where an omitted detail changes the correct answer. Preserve the original library version and compression settings with those fixtures.

Decide what this migration is allowed to transform. A tool result and a system instruction have different authority. Do not move a compressor across that boundary merely because both are strings.

## Install and run an offline candidate

```bash
npm install -g @caveman-ai/cli
caveman setup --install
caveman compress < tool-output.json > candidate-view.txt
```

Use a recorded tool payload that you are permitted to retain. Read the accounting output and inspect `candidate-view.txt`. The CLI delegates compression to its local engine. Missing binaries can result in pass-through, so check setup rather than treating a successful shell exit as proof of a transform.

Caveman detects supported content formats and selects a compressor. An unsupported or unhelpful transformation keeps the original. The [engine reference](https://docs.caveman.so/docs/proxy/engine) lists format and build constraints.

## Add the recovery contract

When a lossy view is produced, the original is stored locally and a recovery handle identifies it. Keep that handle with the view. Use the actual emitted value for this check:

```bash
caveman retrieve YOUR_RECOVERY_HANDLE > recovered.json
cmp tool-output.json recovered.json
```

Then test recovery from the same runtime and storage environment that the agent will use. A handle on a developer laptop is not accessible to a remote worker by magic. A container restart without a retained volume can remove the store even if your application keeps the handle.

Your integration needs to expose recovery as an agent tool or use a proxy path that handles it. Do not replace the LLMLingua return string with a Caveman view while omitting this step. The model would see less information without the mechanism that makes the change recoverable.

## Evaluate both candidates

Feed the original, LLMLingua view, and Caveman view to the same downstream task. Grade the final answer with the same evaluator. Keep the compression model's resource use in the LLMLingua arm and recovery turns in the Caveman arm.

Measure total elapsed time as well as model usage. A CPU-bound or GPU-bound compression step can move the bottleneck even when it reduces downstream input. Record no-op cases and any increased cost. The [cost guide](/guides/measure-agent-cost) covers paired runs and incomplete usage.

If Caveman helps tool results but not long prose, route only the tool results through it. You can preserve your existing document pipeline. There is no benefit in forcing unlike payloads through one compressor for administrative neatness.

## Change one application path

Put the candidate behind a configuration flag that selects the compressor before a new task starts. Keep the original content available throughout that task. Do not alternate marker formats in the middle of a conversation unless the runtime can resolve both.

Before enabling the flag beyond a test caller, check streaming, timeout behavior, large inputs, and a failed storage write. Verify the application's error handling when recovery is unavailable. A local byte check proves storage fidelity; the full task check proves that the agent can use it.

## Roll back

Turn the candidate flag off for new tasks and restore the recorded LLMLingua settings. Allow in-flight Caveman tasks to finish with their original store, or explicitly end them before removing that store. Keep the experiment's source fixtures so a later compressor update can be compared with the same evidence.

Read the [architectural comparison](/compare/llmlingua) and [Microsoft's LLMLingua documentation](https://github.com/microsoft/LLMLingua) alongside the [Caveman recovery contract](https://docs.caveman.so/docs/proxy/recoverable).