WebSocket (Voice)

​

Overview

• Stream audio over WebSocket rather than SIP, WebRTC, or HTTP.
• Receive Coval’s Linear PCM audio at a fixed sample rate and return PCM or MP3 audio.
• Optionally send structured side-events, such as cart updates or session status messages, alongside audio.

​

Connection modes

​

Authentication

• Authorization header — set authorization_header to the auth value Coval should send during the WebSocket upgrade. Values like Bearer <ACCESS_TOKEN> and Basic <BASE64_CREDENTIALS> are sent as the Authorization header value. Values like X-API-Key <KEY> are sent as the X-API-Key header.
• Query-string token — when the agent only supports browser-style auth, encode the token directly in the endpoint, for example wss://example.com/ws?token=....
• Custom headers — custom_headers accepts additional upgrade headers. In the UI, add header name/value rows. Through the API, send metadata.custom_headers as a JSON object or as a JSON-encoded object string, for example {"X-Foo":"bar"} or "{\"X-Foo\":\"bar\"}".

​

Audio transport

• Codec: PCM (linear)
• Sample rate: 16 000 Hz
• Bit depth: 16-bit
• Endianness: little-endian
• Channels: 1 (mono)
• Recommended frame duration for peer implementations: 20-100 ms

​

Audio format fields

​

HTTP-first setup fields

​

Handshake

{
  "type": "session_ready",
  "session_id": "abc123"
}

​

Non-audio event capture

{ "non_audio_event_message_types": ["system_notify"] }

• event_type — the value at message_type_path (for example system_notify).
• event_name — the optional event field from the payload (for example ocb:cart-updated).
• payload — the full parsed JSON message.

{
  "action": "system_notify",
  "event": "ocb:cart-updated",
  "payload": {
    "items": [
      {
        "name": "latte",
        "quantity": 1,
        "modifiers": ["oat milk"],
        "price": 4.5
      }
    ],
    "subtotal": 4.5,
    "total": 4.5
  }
}

​

Media (image) frames

{
  "type": "media",
  "name": "{{media_name}}",
  "mime_type": "{{mime_type}}",
  "data": "{{media_data}}"
}

• {{media_data}} is required.
• {{media_name}} and {{mime_type}} are optional placeholders.
• If the template is exactly {{media_data}}, Coval sends raw bytes.
• Otherwise, Coval base64-encodes the image and substitutes it into your JSON template.

​

Examples

{
  "action": "start_session",
  "session_type": "simulation",
  "metadata": {
    "source": "coval",
    "test_mode": true
  }
}

{
  "X-Client-ID": "coval-simulation",
  "X-API-Version": "2024-01",
  "X-Environment": "production"
}

​

JSON audio preset

{
  "connection_mode": "direct",
  "websocket_compat_profile": "json_audio",
  "initialization_json": "",
  "handshake_ready_message_type": "",
  "handshake_requires_session_id": false,
  "send_sample_rate_hertz": 16000,
  "receive_sample_rate_hertz": 16000,
  "send_audio_template": "{\"action\":\"audio_message\",\"payload\":{},\"audio_bytes\":\"{{audio_data}}\",\"sender\":\"USER\"}",
  "message_type_path": "action",
  "audio_message_type_value": "audio_message",
  "audio_data_path": "audio_bytes",
  "audio_encoding": "pcm",
  "receive_audio_channels": 1,
  "non_audio_event_message_types": ["system_notify"]
}

​

Setup

1. Prepare the agent endpoint. Confirm wss:// is reachable, audio format matches the configuration above, and decide whether the agent requires Bearer auth.
2. Create the agent in Coval. Open the Agents page in your Coval org, choose WebSocket as the connection type, and either fill the fields manually or apply the JSON audio preset.
3. Smoke test. Build a small test set with a single voice persona and run a simulation. The transcript should show alternating turns, the result page should expose usable audio, and any configured side-events should be available to transcript-based metrics.

​

How simulations work

1. Coval performs any HTTP-first setup, then opens the WebSocket with any configured Bearer token or custom headers.
2. If handshake_ready_message_type is set, Coval waits for the ready message before sending audio.
3. Coval streams persona audio outward using send_audio_template at the configured sample rate: raw PCM bytes for {{audio_data}}, or JSON text frames for any JSON template.
4. Inbound binary frames or matching JSON audio frames are decoded and resampled if needed.
5. Inbound non-audio JSON messages whose type is in non_audio_event_message_types are accepted; unconfigured non-audio messages are ignored.
6. When the persona finishes, Coval closes the WebSocket cleanly.

​

Troubleshooting

ngrok http 8080
# Use the generated wss:// URL as the agent endpoint.

​

Best practices

1. Pick the JSON audio preset (or a similar named preset) instead of hand-filling fields when one exists. It keeps the metadata canonical for the agent shape.
2. Mirror the agent’s sample rate exactly in send_sample_rate_hertz / receive_sample_rate_hertz. Resampling is supported but degrades audio.
3. Capture the side-events you care about by adding their action values to non_audio_event_message_types. Don’t silently rely on the agent emitting them.
4. Keep the agent’s WebSocket handler long-lived and avoid closing the connection while the simulation is active.
5. Log initialization payloads, ready messages, and payload parsing errors during initial setup.
6. Rotate Bearer tokens on a schedule; Coval re-reads the value at every connection setup.