Skip to main content
POST

Authorizations

​
X-API-Key
string
header
required

API key for authentication

Headers

​
X-Coval-Workspace-Id
string

Workspace that scopes this operation. When omitted, the organization's active default workspace is used when one can be resolved.

Required string length: 1 - 26

Body

application/json

Create metric request

​
metric_name
string
required

Display name

Required string length: 1 - 200
​
description
string
required

Metric description

Required string length: 1 - 1000
​
metric_type
enum<string>
required

Metric evaluation type.

  • METRIC_AGENT_JUDGE - Agentic binary evaluation with bounded evidence tools
  • METRIC_AGENT_JUDGE_CATEGORICAL - Agentic categorical evaluation with bounded evidence tools
  • METRIC_AGENT_JUDGE_NUMERICAL - Agentic numerical evaluation with bounded evidence tools
  • METRIC_LLM_BINARY - Yes/no LLM evaluation
  • METRIC_CATEGORICAL - Multi-class classification
  • METRIC_NUMERICAL_LLM_JUDGE - Numerical scoring (1-N)
  • METRIC_AUDIO_LLM_BINARY - Audio-based yes/no
  • METRIC_AUDIO_LLM_CATEGORICAL - Audio-based classification
  • METRIC_AUDIO_LLM_NUMERICAL - Audio-based scoring
  • METRIC_TOOLCALL - Tool/function call evaluation
  • METRIC_METADATA_FIELD - Extract metadata field
  • METRIC_TRANSCRIPT_REGEX - Regex pattern matching
  • METRIC_PAUSE_ANALYSIS - Speech pause detection
  • METRIC_SQL_FLOAT - Custom SQL query over the simulation's data (set sql_query)
  • METRIC_COMPOSITE_EVALUATION - Judges a list of criteria independently and reports how many were met (set criteria_source, and criteria_path or criteria to match)
Available options:
METRIC_LLM_BINARY,
METRIC_AGENT_JUDGE,
METRIC_AGENT_JUDGE_CATEGORICAL,
METRIC_CATEGORICAL,
METRIC_AUDIO_LLM_BINARY,
METRIC_AUDIO_LLM_CATEGORICAL,
METRIC_TOOLCALL,
METRIC_METADATA_FIELD,
METRIC_AGENT_JUDGE_NUMERICAL,
METRIC_NUMERICAL_LLM_JUDGE,
METRIC_AUDIO_LLM_NUMERICAL,
METRIC_TRANSCRIPT_REGEX,
METRIC_USER_FRUSTRATION,
METRIC_AGENT_REFUSAL,
METRIC_PAUSE_ANALYSIS,
METRIC_SQL_FLOAT,
METRIC_COMPOSITE_EVALUATION,
METRIC_LLM_TTFB,
METRIC_TTS_TTFB,
METRIC_STT_TTFB,
METRIC_LLM_TOKEN_USAGE,
METRIC_TOOL_CALL_RATE,
METRIC_TRACE_STT_WER,
METRIC_TRACE_AUDIO_UPLOAD_STT_WER,
METRIC_CUSTOM_TRACE,
METRIC_VOICE_QUALITY,
METRIC_PHONEME_STRETCH,
METRIC_SYLLABLE_RATE,
METRIC_LOOP_DETECTION,
METRIC_CODEC_ARTIFACT,
METRIC_CLIPPING_ARTIFACT,
METRIC_DROPOUT_ARTIFACT,
METRIC_TIMBRE_DRIFT,
METRIC_SPEECH_ARTIFACT_ANOMALY,
METRIC_PITCH_VARIABILITY,
METRIC_NON_EXPRESSIVE_PAUSES,
METRIC_ABRUPT_PITCH_CHANGES,
METRIC_TERMINAL_CONTOUR,
METRIC_LUFS,
METRIC_VOLUME,
METRIC_VOCAL_FRY,
METRIC_DERIVED_STRICT_LOGIC,
METRIC_DERIVED_AGGREGATE,
METRIC_IVR_FLOW_ADHERENCE,
METRIC_CUSTOM_AGENT_FAILS_TO_RESPOND,
METRIC_CUSTOM_AGENT_NEEDS_REPROMPTING,
METRIC_CUSTOM_AUDIO_FREQUENCY,
METRIC_CUSTOM_AUDIO_SENTIMENT,
METRIC_CUSTOM_END_REASON,
METRIC_MATCH_EXPECTED_OUTPUT,
METRIC_SPEAKING_TIME_PERCENTAGE,
METRIC_SPECTROGRAM_PITCH_ANALYSIS,
METRIC_VOLUME_PITCH_MISALIGNMENT,
METRIC_WORDS_PER_MESSAGE_WITH_THRESHOLD
​
judge_mode
enum<string>
default:STANDARD

LLM Judge execution mode. Agentic mode is available for text LLM Judges only.

Available options:
STANDARD,
AGENTIC
​
prompt
string

LLM evaluation prompt. Required for LLM-based metrics.

​
enabled_tools
enum<string>[] | null

Agentic LLM Judge tools. Omit to use V1 defaults; pass an empty list to disable all tools.

Evidence tool available to an Agent Judge metric.

Available options:
get_transcript,
search_transcript,
search_transcript_regex,
get_trace_spans,
query_simulation_frames,
get_run_context,
get_evidence_manifest,
get_simulation_outcome,
get_dependency_metric_outputs
​
dependency_metric_ids
string[] | null

Agent Judge metric dependencies. Omit for none; dependencies must be selected explicitly in runs.

Maximum array length: 10
Minimum string length: 1
​
categories
string[]

Categories for classification. Required for categorical metrics.

Required array length: 2 - 50 elements
​
min_value
number

Minimum score. Required for numerical metrics.

​
max_value
number

Maximum score. Required for numerical metrics.

​
metadata_field_type
enum<string>

Field type. Required for METRIC_METADATA_FIELD.

Available options:
STRING,
NUMBER,
BOOLEAN
​
metadata_field_key
string

Metadata key. Required for METRIC_METADATA_FIELD.

​
regex_pattern
string

Regex pattern. Required for METRIC_TRANSCRIPT_REGEX.

​
role
enum<string>

Speaker role filter. Aliases user and assistant are normalized on write.

Available options:
agent,
persona,
user,
assistant
​
min_pause_duration_seconds
number

Min pause duration in seconds. Required for METRIC_PAUSE_ANALYSIS.

Required range: x >= 0.5
​
max_silence_duration_seconds
number
Required range: x > 0
​
min_silence_gap_seconds
number
Required range: x > 0
​
frequency_threshold
number
Required range: x > 0
​
direction
enum<string>
Available options:
above,
below
​
success_sentiments
enum<string>[]
Minimum array length: 1
Available options:
Neutral,
Happy,
Angry,
Sad
​
percent_above
number
Required range: 0 <= x <= 100
​
success_end_reasons
enum<string>[]
Minimum array length: 1
Available options:
UNKNOWN,
IDLE_TIMEOUT,
DURATION_LIMIT,
PERSONA_DISCONNECTED,
AGENT_DISCONNECTED,
PIPELINE_ERROR,
REPETITION_LOOP,
AUDIO_UPLOAD_PLAYBACK_COMPLETED,
SCRIPT_COMPLETED,
SCRIPT_DIVERGED
​
observation_name
string
Minimum string length: 1
​
expected_body
Minimum string length: 1
​
match_path
string

Optional dot path; brackets must contain an integer index or comma-separated key=value filters.

Minimum string length: 1
​
min_volume_change_for_pitch_misalignment
number
Required range: x > 0
​
threshold
integer
Required range: x >= 0
​
operator
enum<string>
Available options:
<,
<=,
>,
>=,
==,
!=
​
ivr_flow
object

IVR flow definition (required for METRIC_IVR_FLOW_ADHERENCE). The same flow tree the flow builder in the Coval UI produces; binding it here makes the metric runnable without a UI step.

Example:
​
sql_query
string

SQL query run against the simulation's data. Required for METRIC_SQL_FLOAT. The query returns one row per timestamp with a numeric value and a start_offset_milliseconds; set aggregation_method (SUM, AVERAGE, MIN, MAX, or COUNT; default AVERAGE) to reduce those rows to a single value, and unit for the reported unit. SQL metric units use exact result-unit identifiers: s for seconds, ms for milliseconds, count, percent, or another supported result unit. Do not use seconds or milliseconds. Omit unit for a unitless result; an unsupported SQL unit is rejected before the metric is saved.

Maximum string length: 50000
Example:

"SELECT start_offset_milliseconds, latency_milliseconds AS value FROM events"

​
aggregation_method
string | null

Aggregation method for custom trace values, or SUM, AVERAGE, MIN, MAX, or COUNT for METRIC_SQL_FLOAT (default AVERAGE).

​
unit
string | null

Optional display unit. For METRIC_SQL_FLOAT, use a result-unit identifier such as s, ms, count, or percent; null selects a unitless result.

Maximum string length: 32
​
criteria_source
enum<string>

Where the metric reads its criteria from. Required for METRIC_COMPOSITE_EVALUATION. test_case reads a field off each test case (most commonly its expected behaviors), test_case_attribute reads from the test case's attributes, and metric_metadata uses the fixed list in criteria.

Available options:
test_case,
test_case_attribute,
metric_metadata
Example:

"test_case"

​
criteria_path
string

Path to the criteria on the source. Required when criteria_source is test_case or test_case_attribute.

Maximum string length: 200
Example:

"expected_behaviors"

​
criteria
string[]

Literal list of criteria. Required when criteria_source is metric_metadata.

Example:
​
reporting_method
enum<string>

How per-criterion verdicts aggregate into the metric's value. Each criterion is judged independently as met or not met; percentage_of_criteria_met scores the fraction met among the criteria the judge could evaluate.

Available options:
percentage_of_criteria_met,
count_of_criteria_met,
all_criteria_met
Example:

"percentage_of_criteria_met"

​
base_prompt_template
string

Custom prompt template used to evaluate each criterion. Include a {criterion} placeholder; if it is omitted the criterion is appended to the prompt.

Maximum string length: 50000
​
include_traces
boolean

Inject OTel trace context into the LLM judge prompt during evaluation. Supported for LLM judge metric types and METRIC_COMPOSITE_EVALUATION (METRIC_LLM_BINARY, METRIC_CATEGORICAL, METRIC_NUMERICAL_LLM_JUDGE, METRIC_AUDIO_LLM_BINARY, METRIC_AUDIO_LLM_CATEGORICAL, METRIC_AUDIO_LLM_NUMERICAL, METRIC_COMPOSITE_EVALUATION).

​
audio_filter
object | null

Restrict an audio metric to part of the recording, for example only the caller with {"include": [{"type": "speaker", "speakers": ["persona"]}]}. Omit or send null to use the metric's default scope. Supported for METRIC_ABRUPT_PITCH_CHANGES, METRIC_AUDIO_LLM_BINARY, METRIC_AUDIO_LLM_CATEGORICAL, METRIC_AUDIO_LLM_NUMERICAL, METRIC_CLIPPING_ARTIFACT, METRIC_CODEC_ARTIFACT, METRIC_CUSTOM_AGENT_NEEDS_REPROMPTING, METRIC_CUSTOM_AUDIO_FREQUENCY, METRIC_CUSTOM_AUDIO_SENTIMENT, METRIC_DROPOUT_ARTIFACT, METRIC_LOOP_DETECTION, METRIC_LUFS, METRIC_NON_EXPRESSIVE_PAUSES, METRIC_PAUSE_ANALYSIS, METRIC_PHONEME_STRETCH, METRIC_PITCH_VARIABILITY, METRIC_SPEAKING_TIME_PERCENTAGE, METRIC_SPECTROGRAM_PITCH_ANALYSIS, METRIC_SPEECH_ARTIFACT_ANOMALY, METRIC_SYLLABLE_RATE, METRIC_TERMINAL_CONTOUR, METRIC_TIMBRE_DRIFT, METRIC_VOCAL_FRY, METRIC_VOICE_QUALITY, METRIC_VOLUME, METRIC_VOLUME_PITCH_MISALIGNMENT; rejected with 400 for any other type.

Example:
​
runtime_config
object

Override the LLM model used for metric evaluation. If omitted, the platform default model is used. Use GET /v1/models/metric to list available models. Not supported for audio metric types (METRIC_AUDIO_LLM_BINARY, METRIC_AUDIO_LLM_CATEGORICAL, METRIC_AUDIO_LLM_NUMERICAL), which always use the platform-default audio model.

​
target_condition
object

Target condition for metric evaluation

Example:
​
tags
string[] | null

Tags to associate with this metric. Null or omitted creates the metric with no tags. Pass [] for an empty tag list.

Example:
​
case_insensitive
boolean | null

Apply case-insensitive matching (default: false)

​
detection_preset
enum<string> | null

METRIC_ABRUPT_PITCH_CHANGES / METRIC_NON_EXPRESSIVE_PAUSES / METRIC_VOCAL_FRY preset (recommended). 'strict' flags more, 'normal' is the default, 'lenient' flags fewer. Omit to use 'normal'.

Available options:
strict,
normal,
lenient
​
harmonics_to_noise_ratio_threshold_offset_db
number | null

Advanced (dB): offset below baseline HNR under which a frame is fry (less negative flags more).

Required range: -25 <= x < 0
​
jitter_threshold_multiplier
number | null

Advanced: multiple of baseline jitter above which a frame is fry (lower flags more).

Required range: x > 1
​
loud_threshold_db
number | null

Advanced (dBFS): level at/above which audio is flagged loud. Must exceed soft_threshold_db.

Required range: -60 <= x <= 0
​
low_pitch_threshold_multiplier
number | null

Advanced: pitch fraction of baseline below which a frame is fry (higher flags more).

Required range: 0 < x < 1
​
mad_z_score_threshold
number | null

Advanced: override the preset's MAD modified z-score threshold.

Required range: x > 0
​
match_mode
enum<string> | null

Match mode: 'presence' (default) returns 1.0 if found, 'absence' returns 1.0 if NOT found

Available options:
presence,
absence
​
metric_attribute
string | null

Span attribute key to measure (required for METRIC_CUSTOM_TRACE)

Maximum string length: 200
​
metric_metadata
object | null

Derived metric config. For METRIC_DERIVED_STRICT_LOGIC: {derived_type, conditions, join_operator, met_value, not_met_value}. For METRIC_DERIVED_AGGREGATE: {derived_type, aggregation_type, parent_metric_ids, count_where_target?}.

​
min_fry_segment_seconds
number | null

Advanced (s): minimum kept fry-run duration (lower flags more).

Required range: 0 <= x <= 1
​
pause_detection_preset
enum<string> | null

Anomaly detection preset (recommended). 'strict' flags more pauses (MAD threshold 2.0), 'normal' is balanced (3.0, default), 'lenient' is conservative (4.0). Omit to use 'normal'.

Available options:
strict,
normal,
lenient
​
pitch_change_threshold_hz
number | null

Advanced (Hz): pitch movement near a pause above which the pause counts as expressive.

Required range: x > 0
​
position
enum<string> | null

Position constraint: 'any' (default), 'first', or 'last' message of the role

Available options:
any,
first,
last
​
significant_changes_threshold_hz
number | null

Advanced (Hz): pitch jump above which a change counts as abrupt (lower flags more).

Required range: x > 0
​
soft_threshold_db
number | null

Advanced (dBFS): level at/below which audio is flagged soft.

Required range: -60 <= x <= 0
​
span_name
string | null

OTel span name to query (required for METRIC_CUSTOM_TRACE)

Maximum string length: 200
​
threshold_preset
enum<string> | null

METRIC_VOLUME threshold preset (recommended). 'strict' flags more level outliers, 'normal' is the default, 'lenient' flags fewer. Omit to use 'normal'.

Available options:
strict,
normal,
lenient
​
value_source
string | null

Source of the aggregated value for METRIC_CUSTOM_TRACE: 'attribute' (default) reads the configured span attribute; 'duration' aggregates the span's own duration in seconds.

Response

Metric created

​
metric
object
required

Metric resource