Follow the whole request.
Keep trace and parent-span IDs together. Inspect the LLM call alongside the request, auth, guardrail, and service spans that LiteLLM emits.
Caveman for LiteLLM
Keep LiteLLM.
Add Caveman.
Bring your LiteLLM traces into Caveman. Understand usage, investigate failures, and turn captured conversations into evaluations. Your gateway stays where it is.
Caveman Platform · in private development
Same endpointSame provider keysOne inference gateway
01 / Your existing traffic path
Your app still
calls LiteLLM.
LiteLLM keeps provider calls, virtual keys, budgets, retries, and fallbacks. Its OpenTelemetry exporter sends traces to Caveman in the background.
Your appLiteLLMYour providers
LiteLLM traces OpenTelemetry Caveman
Already using an observability backend? Add Caveman to your OpenTelemetry Collector’s existing export pipeline. Keep your current destination.
02 / From a trace to the next improvement
See what each call used.
Read models, tokens, timing, errors, and LiteLLM-reported cost. External observations keep their source and accounting basis.
Make the failure testable.
Opt into redacted message and tool-content capture. Open retained content, build evaluation datasets and scenarios, and give investigations concrete evidence.
03 / Connect once
Start with traces.
Add routing when ready.
Use native LiteLLM OpenTelemetry for visibility. Add the optional Caveman callback when you want model decisions inside your existing gateway.
1. Create a project trace key.
The setup page gives you the exporter settings for your Caveman project. Provider credentials stay in LiteLLM.
2. Enable the exporter.
Set the environment where LiteLLM starts, or fan out your collector. Run your existing config and send a request through your existing endpoint.
3. Open the trace.
Check receipt of real LiteLLM traces in Caveman. Follow a span into its content, linked evaluations, and investigation context.
Metadata-only by default. Content capture requires explicit storage and purpose consent and follows redaction, access, and retention controls. The routing callback supports portable text chat completions; other requests keep their original model. Trace export does not apply gateway compression or establish verified savings.
Make every token count.
You chose your gateway.
Keep it.
Bring your LiteLLM deployment and the workflows you want to improve. Connect Caveman around the stack you already operate.