If your agent already exports traces to Langfuse, Arize Phoenix, or LangSmith, Coval can pull those traces into the trace viewer, run trace-based metrics (timings, LLM judges, custom metrics), and populate the transition heatmap — without re-instrumenting your agent. Connect your account once in Settings → Integrations, and Coval handles the rest after each simulation.
How it works
The flow is the same for every provider:
- Your agent sends traces to your platform as it does today.
- When a Coval simulation finishes, Coval fetches the traces that fall inside the simulation’s time window.
- They’re normalized to OpenTelemetry spans and written to the same ClickHouse-backed trace store that native OTLP ingestion uses.
- The trace viewer, trace metrics, and transition heatmap work against the imported spans exactly as they do for native OTLP traces.
Imported spans are tagged with service.name set to the source (langfuse, arize, or langsmith) so they’re easy to distinguish in the viewer. If a simulation already has native OTLP traces, the import is skipped to avoid duplicate spans.
Correlation keeps imports precise. Set simulation_output_id (or session_id / coval_simulation_output_id) in your trace metadata so Coval matches traces to the right simulation — essential when your agent handles concurrent calls. Each provider’s exact keys and code are in its tab below.
Langfuse
Arize Phoenix
LangSmith
Prerequisites
- A Langfuse project with at least one completed trace
- Langfuse Public Key and Secret Key from your project settings
Connect
- Open Settings → Integrations in Coval and expand the Langfuse Integration panel.
- Paste your Public Key and Secret Key.
- Set Host if you self-host Langfuse; leave the default
https://us.cloud.langfuse.com for Langfuse Cloud.
- Save.
CorrelationCoval matches Langfuse trace metadata on simulation_output_id, session_id, or coval_simulation_output_id. If none is set, Coval falls back to the simulation’s time window and imports every trace in it — set a key to stay precise under concurrent calls.from langfuse import Langfuse
langfuse = Langfuse()
with langfuse.start_as_current_span(
name="turn",
metadata={"simulation_output_id": simulation_output_id},
) as span:
...
service.name = langfuse and the original Langfuse attributes preserved under langfuse.*. OTel GenAI attributes (gen_ai.request.model, gen_ai.usage.*) are emitted for LLM generations so trace metrics work out of the box.Limits
- Import runs once per simulation, synchronously, within a 30-second budget.
- Up to 500 traces per simulation window (100/page × 5 pages).
Troubleshooting| Symptom | Likely cause |
|---|
| No spans, correct time window | Check the Langfuse Integration card in Settings; re-save if the Configured chip is missing. |
| Spans don’t match the simulation | Set metadata.simulation_output_id on your traces. |
401 Unauthorized | Keys were rotated. Click Replace key in Settings. |
Arize AX cloud support is on the roadmap. AX’s /v2/spans endpoint indexes 15–30 minutes after OTLP ingest, which doesn’t fit Coval’s per-simulation import window. For now, Coval imports from Phoenix, which exposes spans the moment they land.
- A self-hosted Phoenix deployment reachable from Coval’s workers, or a Phoenix Cloud account, with a project name you write spans to
Connect
- Open Settings → Integrations in Coval and expand the Arize Integration panel.
- Fill in the fields and save.
| Field | Required | Notes |
|---|
| Phoenix Host | Yes | e.g. https://app.phoenix.arize.com for Cloud, or your self-hosted base URL. |
| Project Name | Yes | The Phoenix project your agent writes to. |
| API Key | Optional | Required for Phoenix Cloud, ignored for unauthenticated self-hosted Phoenix. |
simulation_output_id, session_id, session.id, or coval_simulation_output_id (and the same keys inside a JSON metadata attribute). Set one on your root span.from opentelemetry import trace as otel_trace
tracer = otel_trace.get_tracer("my-agent")
with tracer.start_as_current_span("conversation") as span:
span.set_attribute("simulation_output_id", simulation_output_id)
...
span.set_attribute(SpanAttributes.SESSION_ID, simulation_output_id).Coval fails closed when a correlation hint is set but no spans match — it returns an empty result rather than importing everything in the window, avoiding cross-contamination between concurrent simulations.VerifyOpen the result and click View Traces. Imported spans appear with service.name = arize. LLM spans are normalized to span_name = "llm", and common token-count attributes are aliased to OTel GenAI conventions:| Arize / OpenInference | OTel GenAI alias |
|---|
llm.token_count.prompt | gen_ai.usage.input_tokens |
llm.token_count.completion | gen_ai.usage.output_tokens |
input.value | input |
output.value | output |
- Import runs once per simulation, synchronously, within a 90-second budget (with a brief retry-on-empty).
- Up to 5,000 spans per simulation (1,000/page × 5 pages).
- Self-hosted Phoenix must be reachable from Coval’s workers — expose it via a public ingress if it’s on a private network.
Troubleshooting| Symptom | Likely cause |
|---|
| No spans, correct time window | Check the Arize Integration card; confirm Project Name matches what your agent writes to. |
| Spans in Phoenix but not Coval | Set simulation_output_id (or session_id / session.id) on your root span. |
401 Unauthorized | API key rotated. Click Replace key. |
Connection refused (self-hosted) | Phoenix isn’t publicly reachable from Coval’s workers. |
Prerequisites
- A LangSmith account with a project your agent writes runs to
- A LangSmith API key (Settings → API Keys,
lsv2_pt_...)
Connect
- Open Settings → Integrations in Coval and expand the LangSmith Integration panel.
- Fill in the fields and save.
| Field | Required | Notes |
|---|
| API Key | Yes | LangSmith API key (lsv2_pt_...). Workspace-scoped is fine. |
| Project Name | Yes | The LangSmith project (session name) your agent writes to. |
| Host | Yes | Defaults to https://api.smith.langchain.com. Use the EU or AWS host for those regions. |
simulation_output_id (or session_id / coval_simulation_output_id) in the run’s metadata. When the root run carries it, Coval also pulls in its child runs in the same trace — you don’t need to tag every span.from langsmith import traceable
@traceable(metadata={"simulation_output_id": simulation_output_id})
def handle_call(payload):
...
/otel/v1/traces? LangSmith only maps attributes prefixed with langsmith.metadata. into the run metadata that /runs/query filters on:span.set_attribute("langsmith.metadata.simulation_output_id", simulation_output_id)
service.name = langsmith. LLM runs are normalized to span_name = "llm", and token counts are extracted from LangSmith’s various locations and exposed as OTel GenAI conventions:| LangSmith field | OTel GenAI alias |
|---|
outputs.usage_metadata.input_tokens (or prompt_tokens) | gen_ai.usage.input_tokens |
outputs.usage_metadata.output_tokens (or completion_tokens) | gen_ai.usage.output_tokens |
outputs.usage_metadata.total_tokens | gen_ai.usage.total_tokens |
extra.metadata.ls_model_name | gen_ai.request.model |
inputs / outputs | input / output |
- Import runs once per simulation, synchronously, within a 90-second budget (with a brief retry-on-empty).
- Up to 500 runs per simulation (100/page × 5 pages).
- Uses the
/runs/query endpoint (available on all plans); bulk S3 export is not used.
Troubleshooting| Symptom | Likely cause |
|---|
| No spans, correct time window | Check the LangSmith Integration card; confirm Project Name matches your agent’s project. |
| Spans in LangSmith but not Coval | Set metadata.simulation_output_id on the root run. |
401 Unauthorized | API key rotated or revoked. Click Replace key. |
404 on /sessions | Wrong Host for your region — try the EU or AWS host. |
See also
