---
title: "Evaluate Caveman beside LangSmith without breaking agent workflows"
description: "Treat LangSmith observability, prompt dependencies, and LangGraph execution as separate migration items. Start by moving one investigation or optimization workflow. Caveman Platform is in private deve"
canonical: https://caveman.so/switch/langsmith
last-updated: 2026-09-07
---

# Evaluate Caveman beside LangSmith without breaking agent workflows

Treat LangSmith observability, prompt dependencies, and LangGraph execution as separate migration items. Start by moving one investigation or optimization workflow. Caveman Platform is in private development, so agree on supported ingestion and evaluation scope before changing production collection.

If reducing agent input is the immediate goal, test Caveman's local tools while keeping LangSmith. That trial needs no migration of historical runs.

## Locate dependencies beyond the exporter

Find tracing initialization, environment settings, decorated functions, callback handlers, prompt fetches, dataset jobs, and links from CI or issue trackers. Record the project and run identifiers used by your team.

Check whether application behavior depends on a remote prompt or configuration. Preserve that dependency until its replacement can resolve the same version and variables. Removing tracing and replacing prompt delivery are different changes.

Use [LangSmith's documentation](https://docs.langchain.com/langsmith/observability) for your SDK and integration. Do not assume every LangChain or LangGraph event becomes an equivalent span in another product.

## Freeze graph execution during the trial

Keep the same graph code, checkpoint store, thread identifiers, tool definitions, and interrupt behavior. Save a representative checkpoint through your application's supported process.

Include a run that pauses for human input and resumes. If the candidate loses the relationship between the original call and the resumed run, a total grouped by session may undercount work or mix separate attempts.

Never rerun a production tool with external side effects merely to reproduce telemetry. Use staging credentials and disposable fixtures for migration checks.

## Preserve evaluation meaning

Copy or retain access to the inputs and expected outcomes you are authorized to use. Record the prompt version, model settings, grader code, and score interpretation. A numeric score without its grader is not a portable test.

Separate a fixed acceptance suite from cases used to tune the candidate. Include tool selection, final answer quality, unwanted side effects, and total cost. Judge the task after all retries and resumed steps finish.

The [evaluation guide](/guides/agent-evaluations) gives a practical structure. Keep the original LangSmith run links alongside candidate results so reviewers can inspect both.

## Run a compression trial independently

For a supported local coding-agent session:

```bash
npm install -g @caveman-ai/cli
caveman setup --install
caveman doctor codex
caveman codex
```

Use a checked task with repeated tool output. Compare with a fresh direct session and verify recovery of an omitted detail. This command launches a coding agent; it does not wrap an arbitrary LangGraph server or migrate LangSmith data.

For an application integration, follow [the LangGraph guide](/switch/langgraph). Its adapter status and runtime boundaries matter more than the similarity of the APIs.

## Validate the Platform pilot

Agree on an ingestion path with Caveman and send one staging workflow. Compare model identity, token usage, parent relationships, status, timing, and attribution against the source records. Confirm how missing usage, streaming, retries, and interrupted runs appear.

Check redaction at the source and retention at the destination. If captured payloads are unavailable, record which analyses cannot run. Do not infer semantic content from metadata alone.

Move one consumer, such as a weekly cost investigation, after that mapping works. Keep release evaluations in LangSmith until the candidate has demonstrated equivalent acceptance behavior on the cases your team relies on.

## Roll back without losing the task

Restore the saved tracing configuration and client initialization. Graph code and checkpoints should still be usable because they were held constant. Retain old run links through the review and retention period.

Remove duplicate instrumentation and unused credentials only after consumers have moved. Reconcile overlapping telemetry before comparing aggregate spend.

Read [the comparison](/compare/langsmith) for when Caveman is a useful addition and when keeping LangSmith is the simpler choice.