Prerequisites
- A Coval account with an API key (manage your keys)
- A simulation output ID (for simulations) or a conversation ID (for conversations)
- Python 3.8+ with the OpenTelemetry SDK installed
Configuration
Configure the OpenTelemetry tracer provider to export spans to Coval’s trace ingestion endpoint:The
timeout parameter must be set to 30 seconds to ensure spans are exported reliably. We are working on reducing this requirement in a future update.Getting the Simulation Output ID
TheX-Simulation-Id header must be set to the simulation output ID for the specific call you’re tracing. The simulation output ID is a per-call identifier — different from the run ID. Here’s how to obtain it at runtime.
Inbound voice agents
When Coval places an inbound call, it passes the simulation output ID as a SIP header:X-Coval-Simulation-Id. Read this header when the call arrives and use it to configure your OTLP exporter.
Twilio Programmable Voice (PSTN) — Standard Twilio phone numbers route over the public telephone network, which strips SIP headers. Use the
pre_call_webhook_url agent config instead: Coval will POST the simulation ID to your agent before dialing. See the Twilio ConversationRelay guide.LiveKit agents
For a LiveKit agent, Coval calls your token-generation endpoint before joining the room. Add the simulation output ID to that request:1
Configure the token request header
In the LiveKit agent configuration, expand Advanced Configuration and set Custom Headers (JSON) to:Coval resolves
{{simulation_id}} to the simulation output ID for each call.2
Carry the ID into the agent job
Read
X-Coval-Simulation-Id in the token endpoint and include its value in the dispatch metadata or other per-room context your LiveKit agent reads when it starts.3
Configure the exporter
Read the ID from the LiveKit job context and use it as the
X-Simulation-Id header on the OTLP exporter:Outbound voice agents
Coval’s outbound trigger POST can include the simulation output ID in the request payload. Addsimulation_output_id to your trigger_call_payload configuration in your template, then read it when your webhook receives the trigger and use it to configure the exporter.
WebSocket agents
For direct and HTTP-first WebSocket voice agents and Chat WebSocket agents, pass the simulation output ID in the WebSocket upgrade headers.1
Add the correlation header
In the agent configuration, set Custom Headers to:Coval resolves
{{simulation_id}} separately for each simulation before opening the connection.2
Read it and configure the exporter
Read
X-Coval-Simulation-Id from the WebSocket upgrade request, then configure the OTLP exporter with it as X-Simulation-Id:{"coval_simulation_id": "{{simulation_id}}"} in Initialization JSON and read the ID from the first frame instead.
{{simulation_output_id}} is the canonical placeholder; {{simulation_id}} is accepted as a legacy alias. Voice WebSocket agents can also put it in HTTP-first request fields or TwiML voice-webhook fields. Chat WebSocket agents support it in Custom Headers and Initialization JSON.Tracing for Conversations
For conversations (post-hoc call evaluation), there is no Coval-initiated call, so there is no simulation output ID available at call time. Instead, you use a conversation ID to associate traces with a conversation. The conversation ID is only available after the call ends and you submit the transcript to Coval — which means you can’t configure the OTLP exporter up front. The solution is to buffer spans in memory during the call, then flush them once you have the ID.1
Buffer spans during the call
Use
InMemorySpanExporter (included in opentelemetry-sdk) to hold spans locally during the call instead of exporting them in real time.2
Submit the conversation after the call ends
Post the transcript (and optionally audio) to See
POST /v1/conversations:submit. The response contains the conversation_id you need for trace export.POST /v1/conversations:submit for the full request schema including optional audio, metadata, and metrics fields.3
Export the buffered spans
Create an OTLP exporter with
X-Conversation-Id and flush the buffered spans to Coval.Full conversation tracing example
Uploading Traces via the Dashboard
You can also upload traces directly from the Coval dashboard without using the SDK. In the Conversations page, click Upload to Conversations and:- Add your audio file or transcript as usual
- In the Traces (Optional) section, select your OTLP traces JSON file (must contain a
resourceSpansarray) - Click Upload — the conversation and traces are submitted together
Payload Limits & Batching
A single export request to/v1/traces has a size limit. Large buffered exports — most commonly the end-of-call flush in the conversation flow above — can exceed it and fail with 413 Request Entity Too Large.
Keep each export request under roughly 3–4 MB. Treat this as a practical target, not a fixed contract: stay comfortably below it rather than tuning to an exact boundary.
Splitting spans across requests
You can split one call’s spans across multiple export requests. Every request carrying the sameX-Conversation-Id (or X-Simulation-Id) is merged server-side into a single trace, reconstructed from each span’s parent/child relationships. There is no ordering requirement between requests.
The simplest way to stay under the limit is BatchSpanProcessor with a bounded batch size, which chunks exports for you:
max_export_batch_size if your spans carry large attributes such as full transcripts or prompts.
Spans can arrive before
POST /v1/conversations:submit has finished registering the conversation. They are still attributed correctly and reconcile automatically — no special handling needed on your side.Instrumenting Your Agent
Once the tracer is configured, wrap operations in spans to capture trace data:Shutdown — Call
provider.shutdown() when your agent exits. With SimpleSpanProcessor, spans are exported synchronously as each span ends (not buffered), so they are already in Coval before shutdown is called. Shutdown is still good practice for clean resource teardown.Span Naming Conventions
Coval’s trace viewer applies semantic colors and labels to well-known span names. Using these names gives a richer experience in the UI and enables built-in trace metrics.
Any span name works — spans with names not listed above will still appear in the UI with auto-assigned colors. Use
service.name in your Resource to group spans by service.
For complete working implementations, see the voice agent examples on GitHub — Vapi, Pipecat, and LiveKit agents that emit the full span schema.
Instrumenting STT Spans
To use the STT Word Error Rate metric (or its Audio Upload variant), your agent must emitstt spans with a transcript attribute containing the transcribed text. This is what allows Coval to compare your agent’s STT output against a reference transcript. Coval also accepts the older stt.transcription alias for compatibility, but transcript is the canonical attribute for new integrations. We also recommend attaching stt.confidence when your STT provider exposes a per-utterance confidence score.
Here is an example using the Pipecat framework:
PipelineTask(..., enable_tracing=True), Pipecat still emits the standard stt span, and the subclass adds stt.confidence onto that same span:
"stt" and include the transcript attribute with the transcribed text. stt.confidence is optional, but when present it should be a 0.0-1.0 score for the final utterance.
Instrumenting LLM Spans
Includellm.finish_reason on llm spans so you can tell why the model stopped generating. This is especially useful when debugging responses that were silently cut off because llm.finish_reason=length.
Here is a Pipecat example that enriches the built-in traced llm span:
llm span after the provider response finishes:
stop, length, tool_calls, and content_filter.
Provider Fallback Spans
Many voice agents use a provider fallback chain for STT — for example, Deepgram → Google → Azure. Without per-provider spans, a singlestt span only shows the final result; there is no visibility into which provider served the call, how long each attempt took, or why a fallback triggered.
The convention is to create one stt.provider.<name> child span per provider attempt, nested inside the parent stt span:
Span attributes
Code example
Full Example
Using Span Attributes in Custom Metrics
Any numeric span attribute your agent emits can be measured using a Custom Trace Metric (METRIC_CUSTOM_TRACE). This lets you track latency, token counts, or any other numeric value from your traces without writing custom evaluation code.
To create a custom trace metric, specify:
- Span Name — the
span_nameof the spans to aggregate (e.g.llm,tts, or any custom span you create) - Metric Attribute — the span attribute key containing the numeric value (e.g.
metrics.ttfb,token_count) - Aggregation Method — how to aggregate across turns:
average,median,p90,p95,p99,max,min,sum,count,error_rate, orsuccess_rate