A flight recorder
for AI agents.
Every production run leaves a vestige. Vestige captures it, freezes it as a byte-exact regression test, and replays any failure deterministically — so a prompt change can never quietly break your agent again.
This is a live trace — not a mock.
Works with the tools you already use
OpenAI and Anthropic are auto-instrumented directly. Framework calls are captured through the HTTP and DB layer underneath — no per-framework plugin needed.
The insight
“Agents are non-deterministic, so you can't test them” is wrong. Your code is deterministic. The non-determinism lives in the LLM and tool responses. Freeze those, and replay becomes exact — a real regression test, not a probabilistic score.
The problem
Agents fail in ways normal software doesn't
Unit tests don't cover LLM calls. Integration tests can't replay production state. Nothing tells you what changed when a model upgrade breaks behavior.
ERR · SILENT
Failures with no stack trace
A prompt tweak changes behavior. Nothing throws. No alert fires. A customer notices a week later, and by then the run that caused it is long gone.
ERR · LOST
Impossible to reproduce
You re-run it locally and it works. The exact API responses, the exact context, the exact state your agent saw in production — vanished the moment the request finished.
ERR · BLIND
No diff when behavior drifts
You upgrade the model or swap a tool. Something feels off. You have no baseline to compare against and no way to see what actually changed, token by token.
The loop
Walk one trace through its whole life
Record it, freeze it, break it, and watch the gate catch the regression. Click through — or use the arrow keys.
REC · 01
Instrument once, capture everything
Two lines of SDK init wrap your agent. Every LLM call, tool invocation, and outbound HTTP request becomes a span in one trace — model, tokens, cost, latency, and the full prompt and completion.
- No proxy, no sidecar — runs in-process
- OpenAI + Anthropic auto-instrumented
- Transitive HTTP and DB calls captured too
How replay works
Capture every input. Serve it back on demand.
At record time the SDK intercepts every external call your agent makes — model requests, tool responses, HTTP, DB — and writes each one to content-addressed storage. At replay time it runs your code again in a sandbox and serves those exact bytes instead of hitting the network.
Nothing is mocked and nothing is guessed. The agent can't tell it isn't in production — which is exactly why the output is identical unless your code changed it.
- 64 KB
- inline threshold before payloads offload to R2
- 0
- model calls during a replay — deterministic and free
- sha256
- content hash keys every fixture, deduped across runs
- < 90s
- to replay a 50-span trace in a fresh sandbox
import vestige from openai import OpenAI vestige.init(api_key="vs_live_…", project_id="proj_…") # Every call below is now captured — and replayable. client = OpenAI() reply = client.chat.completions.create(model="gpt-4o", messages=[…])
Record → replay pipeline
SDK capture
every external I/O
content-hash
sha256
R2 fixtures
frozen bytes
sandbox replay
your latest code
diff
token-level
Under the hood
Everything the black box captures
One recorder, the full picture of a run — captured richly enough that replaying it is byte-exact, not best-effort.
The full trace, every span
Each LLM call, tool, and HTTP request is a span with its own latency, tokens, and cost.
Calls your tools make, too
We follow the HTTP and DB calls underneath your tools — so nothing is missing at replay time.
Content-hashed fixtures
Every external response is frozen and keyed by its content hash, then deduped across runs.
Byte-exact token diff
When a replay diverges, a word-level diff pinpoints the exact tokens that changed.
Replay costs nothing
Fixtures are served from storage, never the model. A full CI suite spends zero on tokens.
Sandboxed and isolated
Each replay runs your code in a fresh, network-isolated container that can only see fixtures.
Where we're alone
Not just another trace viewer
Langfuse and Braintrust explore and score traces. Respan routes them through a gateway and scores them. Vestige stops the regression from shipping — and never sits in your traffic.
| Capability | Vestige | Langfuse | Braintrust | Respan |
|---|---|---|---|---|
| Full trace capture (spans, tokens, cost) | ✓ | ✓ | ✓ | ✓ |
| Deterministic replay from frozen fixtures | ✓ | — | — | — |
| Byte-exact regression test from a real run | ✓ | — | — | — |
| PR-gated CI check that blocks the merge | ✓ | — | — | — |
| Word-level output diff across runs | ✓ | — | — | — |
| Zero LLM cost to run a test | ✓ | — | — | — |
| Stays out of your request path (no proxy) | ✓ | ✓ | ✓ | — |
Built for teams shipping agents to real users
Coding agents
The most-traced agents on earth. Every keystroke can regress; gate each one.
Support agents
Tone and policy drift are invisible until a customer screenshots them. Freeze the good reply.
Research & RAG
Retrieval and synthesis change with every model bump. Diff the answer, not just the score.
Questions, answered
- How is this different from Langfuse or Braintrust?
- They are trace viewers and eval frameworks — and Vestige does those things too. The difference is replay-driven CI. We capture real production responses as fixtures, replay your agent against them deterministically, and block the PR if behavior drifts. An eval scores an output with a model; a Vestige test asserts the output is byte-for-byte what it was. No competitor gates a merge on that today.
- What if my agent is non-deterministic?
- That's the point. Replay is deterministic because the fixture is the recorded response — not a fresh call to the model. Your agent sees the exact bytes it saw in production. The model's randomness is frozen out; only your code can change the result.
- Does replay cost money in CI?
- No. Replay never calls a real model. It serves frozen fixtures directly. A full CI run across dozens of tests costs zero dollars in model fees — you pay only for the compute to run your own code.
- What if the change is intentional and the test should update?
- Re-promote. Open the new passing trace, click Promote to test, and it replaces the baseline. One click — no fixture to regenerate by hand, no test file to edit.
- What does it support today?
- The Python SDK auto-instruments OpenAI and Anthropic via their official clients, plus any HTTP or DB call made underneath — so LangChain, LlamaIndex, CrewAI and AutoGen work through their transitive calls. A TypeScript SDK for the Vercel AI SDK is in progress.
Private beta
Put a black box in your agent
We onboard design partners by hand. Tell us what you're building and a founder will get you set up — usually within a few days.
Book a 15-minute demo callor leave your details and we'll reach out: