Skip to main content
POST

Authorizations

​
X-API-Key
string
header
required

API key for authentication

Body

application/json

Create metric request

​
metric_name
string
required

Display name

Required string length: 1 - 200
​
description
string
required

Metric description

Required string length: 1 - 1000
​
metric_type
enum<string>
required

Metric evaluation type.

  • METRIC_LLM_BINARY - Yes/no LLM evaluation
  • METRIC_CATEGORICAL - Multi-class classification
  • METRIC_NUMERICAL_LLM_JUDGE - Numerical scoring (1-N)
  • METRIC_AUDIO_LLM_BINARY - Audio-based yes/no
  • METRIC_AUDIO_LLM_CATEGORICAL - Audio-based classification
  • METRIC_AUDIO_LLM_NUMERICAL - Audio-based scoring
  • METRIC_TOOLCALL - Tool/function call evaluation
  • METRIC_METADATA_FIELD - Extract metadata field
  • METRIC_TRANSCRIPT_REGEX - Regex pattern matching
  • METRIC_PAUSE_ANALYSIS - Speech pause detection
  • METRIC_SQL_FLOAT - Custom SQL query over the simulation's data (set sql_query)
  • METRIC_COMPOSITE_EVALUATION - Judges a list of criteria independently and reports how many were met (set criteria_source, and criteria_path or criteria to match)
Available options:
METRIC_LLM_BINARY,
METRIC_CATEGORICAL,
METRIC_NUMERICAL_LLM_JUDGE,
METRIC_AUDIO_LLM_BINARY,
METRIC_AUDIO_LLM_CATEGORICAL,
METRIC_AUDIO_LLM_NUMERICAL,
METRIC_TOOLCALL,
METRIC_METADATA_FIELD,
METRIC_TRANSCRIPT_REGEX,
METRIC_PAUSE_ANALYSIS,
METRIC_SQL_FLOAT,
METRIC_COMPOSITE_EVALUATION,
METRIC_CUSTOM_AGENT_FAILS_TO_RESPOND,
METRIC_CUSTOM_AGENT_NEEDS_REPROMPTING,
METRIC_CUSTOM_AUDIO_FREQUENCY,
METRIC_CUSTOM_AUDIO_SENTIMENT,
METRIC_CUSTOM_END_REASON,
METRIC_MATCH_EXPECTED_OUTPUT,
METRIC_SPEAKING_TIME_PERCENTAGE,
METRIC_SPECTROGRAM_PITCH_ANALYSIS,
METRIC_VOLUME_PITCH_MISALIGNMENT,
METRIC_WORDS_PER_MESSAGE_WITH_THRESHOLD
​
prompt
string

LLM evaluation prompt. Required for LLM-based metrics.

​
categories
string[]

Categories for classification. Required for categorical metrics.

Required array length: 2 - 50 elements
​
min_value
number

Minimum score. Required for numerical metrics.

​
max_value
number

Maximum score. Required for numerical metrics.

​
metadata_field_type
enum<string>

Field type. Required for METRIC_METADATA_FIELD.

Available options:
STRING,
NUMBER,
BOOLEAN
​
metadata_field_key
string

Metadata key. Required for METRIC_METADATA_FIELD.

​
regex_pattern
string

Regex pattern. Required for METRIC_TRANSCRIPT_REGEX.

​
role
enum<string>

Speaker role filter. Aliases user and assistant are normalized on write.

Available options:
agent,
persona,
user,
assistant
​
min_pause_duration_seconds
number

Min pause duration in seconds. Required for METRIC_PAUSE_ANALYSIS.

Required range: x >= 0.5
​
max_silence_duration_seconds
number
Required range: x > 0
​
min_silence_gap_seconds
number
Required range: x > 0
​
frequency_threshold
number
Required range: x > 0
​
direction
enum<string>
Available options:
above,
below
​
success_sentiments
enum<string>[]
Minimum array length: 1
Available options:
Neutral,
Happy,
Angry,
Sad
​
percent_above
number
Required range: 0 <= x <= 100
​
success_end_reasons
enum<string>[]
Minimum array length: 1
Available options:
UNKNOWN,
IDLE_TIMEOUT,
PERSONA_DISCONNECTED,
AGENT_DISCONNECTED,
PIPELINE_ERROR,
REPETITION_LOOP,
AUDIO_UPLOAD_PLAYBACK_COMPLETED,
SCRIPT_COMPLETED,
SCRIPT_DIVERGED
​
observation_name
string
Minimum string length: 1
​
expected_body
Minimum string length: 1
​
match_path
string

Optional dot path; brackets must contain an integer index or comma-separated key=value filters.

Minimum string length: 1
​
min_volume_change_for_pitch_misalignment
number
Required range: x > 0
​
threshold
integer
Required range: x >= 0
​
operator
enum<string>
Available options:
<,
<=,
>,
>=,
==,
!=
​
sql_query
string

SQL query run against the simulation's data. Required for METRIC_SQL_FLOAT. The query returns one row per timestamp with a numeric value and a start_offset_milliseconds; set aggregation_method (SUM, AVERAGE, MIN, MAX, or COUNT; default AVERAGE) to reduce those rows to a single value, and unit for the reported unit.

Maximum string length: 50000
Example:

"SELECT start_offset_milliseconds, latency_milliseconds AS value FROM events"

​
criteria_source
enum<string>

Where the metric reads its criteria from. Required for METRIC_COMPOSITE_EVALUATION. test_case reads a field off each test case (most commonly its expected behaviors), test_case_attribute reads from the test case's attributes, and metric_metadata uses the fixed list in criteria.

Available options:
test_case,
test_case_attribute,
metric_metadata
Example:

"test_case"

​
criteria_path
string

Path to the criteria on the source. Required when criteria_source is test_case or test_case_attribute.

Maximum string length: 200
Example:

"expected_behaviors"

​
criteria
string[]

Literal list of criteria. Required when criteria_source is metric_metadata.

Example:
​
reporting_method
enum<string>

How per-criterion verdicts aggregate into the metric's value. Each criterion is judged independently as met or not met; percentage_of_criteria_met scores the fraction met among the criteria the judge could evaluate.

Available options:
percentage_of_criteria_met,
count_of_criteria_met,
all_criteria_met
Example:

"percentage_of_criteria_met"

​
base_prompt_template
string

Custom prompt template used to evaluate each criterion. Include a {criterion} placeholder; if it is omitted the criterion is appended to the prompt.

Maximum string length: 50000
​
include_traces
boolean

Inject OTel trace context into the LLM judge prompt during evaluation. Supported for LLM judge metric types and METRIC_COMPOSITE_EVALUATION (METRIC_LLM_BINARY, METRIC_CATEGORICAL, METRIC_NUMERICAL_LLM_JUDGE, METRIC_AUDIO_LLM_BINARY, METRIC_AUDIO_LLM_CATEGORICAL, METRIC_AUDIO_LLM_NUMERICAL, METRIC_COMPOSITE_EVALUATION).

​
runtime_config
object

Override the LLM model used for metric evaluation. If omitted, the platform default model is used. Use GET /v1/models/metric to list available models. Not supported for audio metric types (METRIC_AUDIO_LLM_BINARY, METRIC_AUDIO_LLM_CATEGORICAL, METRIC_AUDIO_LLM_NUMERICAL), which always use the platform-default audio model.

​
target_condition
object

Target condition for metric evaluation

Example:
​
tags
string[] | null

Tags to associate with this metric. Null or omitted creates the metric with no tags. Pass [] for an empty tag list.

Example:

Response

Metric created

​
metric
object
required

Metric resource