WebSocket (Voice)

Overview

WebSocket voice agents stream audio over a single persistent WebSocket connection. Coval can exchange raw binary PCM frames or JSON envelopes that wrap base64-encoded PCM, MP3, or G.711 mu-law audio, plus configured non-audio events (cart updates, session signals) the agent emits. Use this connection type for voice agents that:

Stream audio over WebSocket rather than SIP, WebRTC, or HTTP.
Receive Coval audio at a fixed sample rate and return PCM, MP3, or G.711 mu-law audio.
Optionally send structured side-events, such as cart updates or session status messages, alongside audio.

For text-only WebSocket agents (token-by-token chat) see Chat WebSocket.

Connection modes

Mode	When to use
Direct	The agent exposes a stable `wss://` URL Coval can dial directly.
HTTP-first	The agent requires an HTTP setup call to provision a per-session WebSocket URL before the audio stream begins.
TwiML webhook	The agent exposes a Twilio-compatible voice webhook that returns a bidirectional `<Connect><Stream>` instruction.

In HTTP-first mode, Coval makes the configured HTTP request, extracts the WebSocket URL using websocket_url_response_path, and opens the audio WebSocket against that URL. In TwiML webhook mode, Coval calls the voice webhook with Twilio-style call parameters, reads the WebSocket URL and custom parameters from the returned TwiML, then starts a Twilio-shaped media stream. See TwiML voice-webhook setup.

Authentication

WebSocket voice agents authenticate during the WebSocket upgrade.

Authorization header — set authorization_header to the auth value Coval should send during the WebSocket upgrade. Values like Bearer <ACCESS_TOKEN> and Basic <BASE64_CREDENTIALS> are sent as the Authorization header value. Values like X-API-Key <KEY> are sent as the X-API-Key header.
Query-string token — when the agent only supports browser-style auth, encode the token directly in the endpoint, for example wss://example.com/ws?token=....
Custom headers — custom_headers accepts additional upgrade headers. In the UI, add header name/value rows. Through the API, send metadata.custom_headers as a JSON object or as a JSON-encoded object string, for example {"X-Foo":"bar"} or "{\"X-Foo\":\"bar\"}".

The UI masks the authorization header field. Do not pass JSON in authorization_header; use custom_headers for additional named headers. Tokens included directly in the endpoint query string may be visible anywhere URLs are logged, so prefer authorization_header when the agent supports it.

Audio transport

Audio can be exchanged as raw PCM bytes or as JSON envelopes containing a base64-encoded audio payload. The default JSON shape is audio_chunk / data; the JSON audio preset uses audio_message / audio_bytes, and the Twilio Media Streams preset uses media / media.payload. These JSON shapes are configurable per agent, and setting send_audio_template to exactly {{audio_data}} makes outbound audio raw bytes instead. The JSON audio preset uses:

Codec: PCM (linear)
Sample rate: 16 000 Hz
Bit depth: 16-bit
Endianness: little-endian
Channels: 1 (mono)
Recommended frame duration for peer implementations: 20-100 ms

Message frames can be bidirectional. Use audio_message_type_value to identify the agent frames that contain inbound audio, and use send_audio_template to shape Coval-originated audio frames. For the JSON audio preset, Coval sends audio_message frames with sender: "USER" and the agent should send its own audio_message frames with sender: "AI".

Audio format fields

Field	Default	Purpose
`endpoint`	– (required)	`wss://` URL Coval connects to in direct mode. Plain `ws://`, `http://`, and `https://` endpoints are rejected for direct WebSocket connections.
`connection_mode`	`direct`	`direct`, `http_first`, or `twiml_webhook`.
`initialization_json`	empty	Optional JSON object Coval sends after the WebSocket upgrade and before any ready-message wait.
`send_audio_template`	`{"type":"audio_chunk","data":"{{audio_data}}"}`	Outbound JSON template. Must contain `{{audio_data}}`. Setting it to literally `{{audio_data}}` sends raw PCM bytes (no JSON wrapping).
`message_type_path`	`type`	Dot-notation path to the field that names the message kind.
`audio_message_type_value`	`audio_chunk`	Value that identifies an inbound audio frame. Use `*` to treat every JSON message as audio.
`audio_data_path`	`data`	Dot-notation path to the base64 audio payload inside an inbound frame.
`audio_encoding`	`pcm`	JSON audio payload encoding: `pcm`, `mp3`, or `ulaw`. MP3 is supported for inbound audio. `ulaw` sends and receives G.711 mu-law audio.
`receive_audio_channels`	`2`	`1` for mono inbound JSON PCM, `2` to keep the legacy stereo-to-mono averaging behavior.
`send_sample_rate_hertz`	`16000`	Outbound sample rate Coval sends to the agent. Allowed: 8 000, 16 000, 24 000, 48 000.
`receive_sample_rate_hertz`	`48000`	Sample rate the agent sends. Allowed: 8 000, 16 000, 24 000, 48 000.
`pipeline_sample_rate_hertz`	`16000`	Coval processing rate; must stay 16 000.
`pace_inbound_binary_audio`	inferred	Pace inbound binary PCM in real time so resampling and metrics see realistic timing. Defaults on when outbound audio is configured for raw PCM bytes and off for JSON templates.

The default receive rate is higher than the send rate because many voice integrations return higher-rate audio while receiving 16 kHz audio from Coval. Paths use dot notation for nested fields, for example payload.audio.data. Match send_sample_rate_hertz / receive_sample_rate_hertz to the agent’s actual stream format; mismatched sample rates can cause speed, pitch, or quality issues.

HTTP-first setup fields

These fields apply when connection_mode is http_first:

Field	Default	Purpose
`http_url`	– (required)	`https://` setup endpoint Coval calls before opening the WebSocket.
`http_method`	`POST`	Setup request method. Allowed: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, `OPTIONS`.
`http_request_body`	`{}`	JSON object body for the setup request.
`http_headers`	`{}`	JSON object of headers for the setup request.
`websocket_url_response_path`	– (required)	Dot-notation path to the WebSocket URL in the setup response, for example `data.websocket_url`.
`authorization_header`	empty	Auth value for the WebSocket upgrade after setup. This is separate from `http_headers`.
`custom_headers`	`{}`	Additional headers for the WebSocket upgrade after setup.

TwiML voice-webhook setup fields

These fields apply when connection_mode is twiml_webhook:

Field	Default	Purpose
`voice_url`	– (required)	HTTP or HTTPS voice webhook Coval calls to obtain TwiML.
`voice_http_method`	`POST`	`POST` sends form-encoded fields; `GET` sends the same fields as query parameters.
`voice_http_headers`	`{}`	Headers for authenticating or routing the voice-webhook request.
`voice_form_fields`	`{}`	Additional fields sent with the Twilio-style call parameters.
`twilio_account_sid`	empty	Static value sent as `AccountSid`.
`twilio_from_number`	empty	Static agent-level value sent as `From`.
`twilio_to_number`	empty	Static agent-level value sent as `To`.
`authorization_header`	empty	Auth value for the WebSocket upgrade after Coval parses the TwiML.
`custom_headers`	`{}`	Additional headers for the WebSocket upgrade.

twilio_from_number and twilio_to_number are fixed on the agent configuration. They are not currently resolved from test-case inputs or agent attributes. TwiML webhook mode requires audio_encoding: ulaw and 8 000 Hz send and receive sample rates. Coval applies those values when they are omitted, but rejects explicit values that do not match the Twilio Media Streams format.

Coval does not currently add an X-Twilio-Signature header to the voice-webhook request. The endpoint must accept unsigned requests or use another authentication mechanism through voice_http_headers.

Handshake

Field	Default	Purpose
`handshake_ready_message_type`	`session_ready` in direct mode; empty in HTTP-first and TwiML webhook modes	Set to an empty string to skip the ready-message wait.
`handshake_requires_session_id`	`true` in direct mode; `false` in HTTP-first and TwiML webhook modes	When `true`, the ready message must include `session_id`.
`handshake_timeout_seconds`	`30`	Seconds Coval waits for the ready message.

With the default message_type_path of type, a direct-mode ready message looks like:

{
  "type": "session_ready",
  "session_id": "abc123"
}

If you customize message_type_path, Coval uses that same path to find the ready-message type.

Per-simulation placeholders

Coval resolves the following placeholders once for each WebSocket voice simulation:

Placeholder	Value	Available in
`{{run_id}}`	Coval’s parent run ID	Connection setup fields
`{{simulation_output_id}}`	Coval’s simulation output ID	Connection setup fields
`{{simulation_id}}`	Legacy alias for `{{simulation_output_id}}`	Connection setup fields
`{{stream_sid}}`	`MZ` followed by 32 lowercase hexadecimal characters	Connection setup fields and send templates
`{{call_sid}}`	`CA` followed by 32 lowercase hexadecimal characters	Connection setup fields and send templates

Connection setup fields are initialization_json in direct and HTTP-first modes, custom_headers in every mode, http_request_body and http_headers in HTTP-first mode, and voice_form_fields and voice_http_headers in TwiML webhook mode. Send templates are send_audio_template, send_media_template, and send_dtmf_template. The generated stream and call SIDs are fresh for every simulation. They are Twilio-shaped transport identifiers, not aliases for Coval’s simulation output ID.

CallSid is CA followed by 32 hexadecimal characters. It is not a bare 32-character correlation ID, and the API does not create or return a separate correlation_id.

For WebSocket simulations, Coval returns the same generated value in both completed-result lookup paths:

Get run: run.results.call_sids[simulation_output_id]
Get simulation: simulation.call_sid

run.results.call_sids contains one entry per WebSocket simulation output that has a generated CallSid. Historical outputs and other transports are omitted from that map. With the Python SDK, read the value from response.run.results.call_sids[simulation_output_id] after the run reaches COMPLETED. To correlate an external interaction with Coval in both directions, persist the run ID, simulation output ID, and CallSid when the connection starts. For a direct or HTTP-first WebSocket connection, include them in initialization_json:

{
  "event": "start",
  "start": {
    "callSid": "{{call_sid}}",
    "customParameters": {
      "coval_run_id": "{{run_id}}",
      "coval_simulation_id": "{{simulation_output_id}}"
    }
  }
}

For TwiML webhook mode, send the Coval IDs as additional voice-webhook fields:

{
  "voice_form_fields": {
    "CovalRunId": "{{run_id}}",
    "CovalSimulationId": "{{simulation_output_id}}"
  }
}

Coval also sends the generated CallSid as a standard Twilio-style webhook field. The webhook can store that three-ID mapping before it returns TwiML. If the WebSocket server also needs the Coval IDs, return them as <Parameter> values under <Stream>; Coval includes returned parameters in the generated start.customParameters object.

Non-audio event capture

Many voice agents emit side-events alongside the audio stream — cart updates, transcript fragments, session telemetry. By default, Coval ignores non-audio JSON messages. To tell Coval which message types to accept, set:

{ "non_audio_event_message_types": ["system_notify"] }

For each matching message, Coval reads:

event_type — the value at message_type_path (for example system_notify).
event_name — the optional event field from the payload (for example ocb:cart-updated).
payload — the full parsed JSON message.

For example, when message_type_path is action and non_audio_event_message_types includes system_notify, this inbound message is accepted as a non-audio event:

{
  "action": "system_notify",
  "event": "ocb:cart-updated",
  "payload": {
    "items": [
      {
        "name": "latte",
        "quantity": 1,
        "modifiers": ["oat milk"],
        "price": 4.5
      }
    ],
    "subtotal": 4.5,
    "total": 4.5
  }
}

Coval does not emit these events to your agent. It only receives configured message types and ignores unconfigured non-audio JSON messages. Accepted event messages are stored with the simulation transcript as websocket_event entries. Transcript-based metrics, including LLM judge metrics, see JSON that includes event_type, event_name, and the full payload, so they can evaluate structured side-channel data such as cart contents, selected menu items, modifiers, quantities, and prices alongside the spoken conversation.

Media (image) frames

Voice WebSocket simulations can attach images from a test case mid-conversation. send_media_template controls the outbound shape:

{
  "type": "media",
  "name": "{{media_name}}",
  "mime_type": "{{mime_type}}",
  "data": "{{media_data}}"
}

Template rules:

{{media_data}} is required.
{{media_name}} and {{mime_type}} are optional placeholders.
If the template is exactly {{media_data}}, Coval sends raw bytes.
Otherwise, Coval base64-encodes the image and substitutes it into your JSON template.

See Test Sets — image attachment for the test-case side.

DTMF frames

send_dtmf_template controls how Coval sends keypad digits when a persona uses DTMF. The template must contain {{digit}}. Coval sends one frame per digit. The Twilio Media Streams preset uses:

{
  "event": "dtmf",
  "streamSid": "{{stream_sid}}",
  "dtmf": {
    "track": "inbound_track",
    "digit": "{{digit}}"
  }
}

Examples

Initialization payload:

{
  "action": "start_session",
  "session_type": "simulation",
  "metadata": {
    "source": "coval",
    "test_mode": true
  }
}

Custom WebSocket upgrade headers:

{
  "X-Client-ID": "coval-simulation",
  "X-API-Version": "2024-01",
  "X-Environment": "production"
}

JSON audio preset

The agent UI ships a JSON audio preset that fills the metadata for JSON audio WebSocket agents. It sets:

{
  "connection_mode": "direct",
  "websocket_compat_profile": "json_audio",
  "initialization_json": "",
  "handshake_ready_message_type": "",
  "handshake_requires_session_id": false,
  "send_sample_rate_hertz": 16000,
  "receive_sample_rate_hertz": 16000,
  "send_audio_template": "{\"action\":\"audio_message\",\"payload\":{},\"audio_bytes\":\"{{audio_data}}\",\"sender\":\"USER\"}",
  "message_type_path": "action",
  "audio_message_type_value": "audio_message",
  "audio_data_path": "audio_bytes",
  "audio_encoding": "pcm",
  "receive_audio_channels": 1,
  "non_audio_event_message_types": ["system_notify"]
}

Set authorization_header to Bearer <ACCESS_TOKEN> after picking the preset if the agent requires auth (most production endpoints do).

Twilio Media Streams preset

Use the Twilio Media Streams configuration preset when you already know the wss:// endpoint and want Coval to connect directly. In the agent editor:

Choose WebSocket as the connection type.
Leave Connection Mode set to Direct WebSocket.
Set Configuration Preset to Twilio Media Streams.
Enter the WebSocket endpoint and any custom parameters in Initialization JSON.

The preset configures G.711 mu-law at 8 000 Hz, disables the ready-message handshake, and fills the Twilio-shaped start, media, and DTMF templates. Its generated identifier placeholders are resolved separately for every simulation:

{
  "connection_mode": "direct",
  "initialization_json": {
    "event": "start",
    "streamSid": "{{stream_sid}}",
    "start": {
      "streamSid": "{{stream_sid}}",
      "callSid": "{{call_sid}}",
      "tracks": ["inbound"],
      "customParameters": {},
      "mediaFormat": {
        "encoding": "audio/x-mulaw",
        "sampleRate": 8000,
        "channels": 1
      }
    }
  },
  "handshake_ready_message_type": "",
  "handshake_requires_session_id": false,
  "send_sample_rate_hertz": 8000,
  "receive_sample_rate_hertz": 8000,
  "send_audio_template": "{\"event\":\"media\",\"streamSid\":\"{{stream_sid}}\",\"media\":{\"payload\":\"{{audio_data}}\"}}",
  "send_dtmf_template": "{\"event\":\"dtmf\",\"streamSid\":\"{{stream_sid}}\",\"dtmf\":{\"track\":\"inbound_track\",\"digit\":\"{{digit}}\"}}",
  "message_type_path": "event",
  "audio_message_type_value": "media",
  "audio_data_path": "media.payload",
  "audio_encoding": "ulaw",
  "receive_audio_channels": 1
}

Edit start.customParameters after selecting the preset when your endpoint needs fixed connection metadata. The preset does not make a Twilio voice-webhook request; use TwiML webhook mode when the endpoint and custom parameters should come from your voice webhook.

TwiML voice-webhook setup

TwiML webhook mode follows the voice-webhook portion of a Twilio Media Streams connection:

Coval generates a CallSid in the form CA plus 32 hexadecimal characters.
Coval calls voice_url with CallSid, AccountSid, From, To, Direction=inbound, CallStatus=ringing, and any voice_form_fields.
The webhook returns TwiML containing a bidirectional <Connect><Stream> element.
Coval opens the returned wss:// URL and sends a connected event followed by a start event with sequenceNumber: "1". The start.customParameters object contains the TwiML <Parameter> values.
Coval streams G.711 mu-law audio at 8 000 Hz using the configured media and DTMF templates.

The webhook must return the bidirectional form:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="wss://voice.example.com/media">
      <Parameter name="instance_name" value="production" />
      <Parameter name="voice_service_id" value="service-123" />
    </Stream>
  </Connect>
</Response>

<Start><Stream> is a one-way audio fork and is rejected. The <Stream> URL must use wss://.

Configure in the agent editor

Create or edit a voice agent and choose WebSocket as the connection type.
Under Connection Mode, choose TwiML Voice Webhook.
Enter the Voice Webhook URL and choose GET or POST.
Add any voice-webhook headers, form fields, Twilio Account SID, and From / To values your endpoint expects.
If the returned WebSocket URL requires authentication, configure Authorization Header or Custom Headers under WebSocket Connection.

Selecting TwiML Voice Webhook applies the Twilio Media Streams audio, media, and DTMF settings, including G.711 mu-law audio at 8 000 Hz and no ready-message wait.

Configure through the Agents API

You can also configure TwiML webhook mode through the Agents API, which is useful for automated agent setup. The following request creates a complete Twilio-shaped WebSocket voice agent. Replace the example URLs, numbers, and custom fields with values for your endpoint:

curl --request POST 'https://api.coval.dev/v1/agents' \
  --header 'x-api-key: <COVAL_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "display_name": "TwiML WebSocket Voice Agent",
    "model_type": "MODEL_TYPE_WEBSOCKET",
    "metadata": {
      "connection_mode": "twiml_webhook",
      "voice_url": "https://voice.example.com/twiml",
      "voice_http_method": "POST",
      "voice_http_headers": {
        "Authorization": "Bearer <VOICE_WEBHOOK_TOKEN>"
      },
      "voice_form_fields": {
        "gateway_mode": "nats"
      },
      "twilio_account_sid": "AC00000000000000000000000000000000",
      "twilio_from_number": "+15551234567",
      "twilio_to_number": "+15557654321",
      "handshake_ready_message_type": "",
      "handshake_requires_session_id": false,
      "send_sample_rate_hertz": 8000,
      "receive_sample_rate_hertz": 8000,
      "send_audio_template": "{\"event\":\"media\",\"streamSid\":\"{{stream_sid}}\",\"media\":{\"payload\":\"{{audio_data}}\"}}",
      "send_dtmf_template": "{\"event\":\"dtmf\",\"streamSid\":\"{{stream_sid}}\",\"dtmf\":{\"track\":\"inbound_track\",\"digit\":\"{{digit}}\"}}",
      "message_type_path": "event",
      "audio_message_type_value": "media",
      "audio_data_path": "media.payload",
      "audio_encoding": "ulaw",
      "receive_audio_channels": 1
    }
  }'

initialization_json and metadata.endpoint are not used in this mode. Coval builds the initialization events from the returned TwiML and connects to its <Stream> URL.

When updating an existing agent, the Agents API replaces the complete metadata object you send. Retrieve the current agent first, merge these TwiML fields with any metadata you need to preserve, and then submit the full metadata object in the update request.

Setup

Prepare the agent endpoint. Confirm wss:// is reachable, audio format matches the configuration above, and decide whether the agent requires Bearer auth.
Create the agent in Coval. Open the Agents page in your Coval org, choose WebSocket as the connection type, and select the connection mode that matches your endpoint. For a voice webhook, choose TwiML Voice Webhook and fill in the webhook fields. You can also use the Agents API for automated setup.
Smoke test. Build a small test set with a single voice persona and run a simulation. The transcript should show alternating turns, the result page should expose usable audio, and any configured side-events should be available to transcript-based metrics.

How simulations work

Coval connects directly, performs HTTP-first setup, or calls the configured TwiML voice webhook to obtain the WebSocket URL.
Coval sends the configured initialization payload. In TwiML webhook mode, it sends the generated connected and start events instead.
If handshake_ready_message_type is set, Coval waits for the ready message before sending audio.
Coval streams persona audio outward using send_audio_template at the configured sample rate: raw bytes for {{audio_data}}, or JSON text frames for any JSON template.
Inbound binary frames or matching JSON audio frames are decoded and resampled if needed.
Inbound non-audio JSON messages whose type is in non_audio_event_message_types are accepted; unconfigured non-audio messages are ignored.
When the persona finishes, Coval closes the WebSocket cleanly.

Sending Traces

Direct and HTTP-first WebSocket agents can send traces to Coval. Set Custom Headers to {"X-Coval-Simulation-Id": "{{simulation_id}}"}. Read that header during the WebSocket upgrade and use its value as the X-Simulation-Id header when exporting OTLP spans. See WebSocket agents for the full setup.

Troubleshooting

Empty transcript with audio frames flowing. Check that audio_message_type_value matches the agent’s field, that audio_data_path points at the base64 payload, and that audio_encoding matches the wire format. Inbound audio sounds half-speed or distorted. Confirm receive_audio_channels. JSON PCM that arrives mono should be configured with receive_audio_channels: 1; the historical default 2 averages two channels and halves the apparent rate when the source is mono. Cart events / status messages look ignored. Add the action value to non_audio_event_message_types. Without it, Coval ignores non-audio JSON messages. Auth failures during handshake. Verify the authorization_header value, or move the token to a ?token=... query string when the agent only supports browser-style auth. TwiML response is rejected. Confirm the voice webhook returns valid XML with <Response><Connect><Stream url="wss://...">. A <Start><Stream> response is one-way and cannot carry the simulated user’s audio back to the agent. Twilio-shaped stream connects but no audio is exchanged. Confirm mu-law at 8 000 Hz, leave the ready-message handshake disabled, and verify the media paths are event and media.payload. The agent editor applies these transport settings when you select TwiML Voice Webhook. For API configuration, include the Twilio media and DTMF templates shown above. Connection refused locally. Tunnel the agent’s ws:// server through ngrok or Cloudflare Tunnel and use the resulting wss:// URL as the agent endpoint.

ngrok http 8080
# Use the generated wss:// URL as the agent endpoint.

If your tunnel provider shows an https:// URL, use the corresponding wss:// URL in Coval. Update the agent configuration when the tunnel URL changes, or use a reserved tunnel domain for a stable endpoint. Unreadable audio or media payloads. For JSON audio/media templates, Coval substitutes base64 data into {{audio_data}} / {{media_data}}; for raw templates, the agent must expect raw PCM or media bytes. Verify the JSON is valid, the configured message fields match the agent payload, audio_encoding is correct, and send_media_template includes {{media_name}} / {{mime_type}} when the agent needs file metadata. Timeouts or no response. Confirm the agent keeps the WebSocket open for the whole conversation, processes incoming audio frames without blocking, sends audio responses in the configured shape, and logs initialization / ready messages while testing.

Best practices

Pick the JSON audio or Twilio Media Streams preset instead of hand-filling fields when one matches your endpoint. It keeps the metadata canonical for the agent shape.
Mirror the agent’s sample rate exactly in send_sample_rate_hertz / receive_sample_rate_hertz. Resampling is supported but degrades audio.
Capture the side-events you care about by adding their action values to non_audio_event_message_types. Don’t silently rely on the agent emitting them.
Keep the agent’s WebSocket handler long-lived and avoid closing the connection while the simulation is active.
Log initialization payloads, ready messages, and payload parsing errors during initial setup.
Rotate Bearer tokens on a schedule; Coval re-reads the value at every connection setup.

Simulate

Evaluate

Observe

Review

Cookbooks

Sofia

Reference

Overview

Connection modes

Authentication

Audio transport

Audio format fields

HTTP-first setup fields

TwiML voice-webhook setup fields

Handshake

Per-simulation placeholders

Non-audio event capture

Media (image) frames

DTMF frames

Examples

JSON audio preset

Twilio Media Streams preset

TwiML voice-webhook setup

Configure in the agent editor

Configure through the Agents API

Setup

How simulations work

Sending Traces

Troubleshooting

Best practices

​Overview

​Connection modes

​Authentication

​Audio transport

​Audio format fields

​HTTP-first setup fields

​TwiML voice-webhook setup fields

​Handshake

​Per-simulation placeholders

​Non-audio event capture

​Media (image) frames

​DTMF frames

​Examples

​JSON audio preset

​Twilio Media Streams preset

​TwiML voice-webhook setup

​Configure in the agent editor

​Configure through the Agents API

​Setup

​How simulations work

​Sending Traces

​Troubleshooting

​Best practices

Overview

Connection modes

Authentication

Audio transport

Audio format fields

HTTP-first setup fields

TwiML voice-webhook setup fields

Handshake

Per-simulation placeholders

Non-audio event capture

Media (image) frames

DTMF frames

Examples

JSON audio preset

Twilio Media Streams preset

TwiML voice-webhook setup

Configure in the agent editor

Configure through the Agents API

Setup

How simulations work

Sending Traces

Troubleshooting

Best practices