Metrics

create and update also accept --input-json (inline JSON, @file, or - for stdin); explicit flags override fields from the JSON. See Input JSON.

List Metrics

coval metrics list [OPTIONS]

Option	Type	Default	Description
`--filter`	string	—	Filter expression (supports `metric_type`, `metric_name`, `create_time`)
`--page-size`	number	50	Results per page (1-100)
`--order-by`	string	`-create_time`	Sort field, prefix with `-` for descending
`--include-builtin`	flag	—	Include built-in metrics (e.g. Turn Count, Audio Duration)

Output columns: ID, NAME, TYPE, CREATED

coval metrics list

Get Metric

coval metrics get <metric_id>

Argument	Type	Required	Description
`metric_id`	string	Yes	The metric ID

coval metrics get met_abc123

Create Metric

coval metrics create [OPTIONS]

Common options

Option	Type	Required	Description
`--name`	string	Yes	Metric name (1-200 characters)
`--description`	string	Yes	Metric description (1-1000 characters)
`--type`	string	Yes	Metric type (see below)
`--target-condition`	JSON	No	Pass/fail target, e.g. `'{"comparison_operator":"gte","target_float":0.8}'` (numeric operators `eq`, `neq`, `gt`, `gte`, `lt`, `lte` with `target_float`; `in`/`nin` with `target_values`)
`--runtime-config`	JSON	No	Override the judge model, e.g. `'{"model_version":"<model id>"}'`. List model IDs with `coval models metric`. Not supported for audio metric types
`--tags`	string	No	Comma-separated tag names; pass an empty value to clear all tags

Type-specific options

Each metric type requires the options marked in the Metric Types table. Options are grouped here by the type they configure.

Option	Type	Applies to	Description
`--prompt`	string	`llm-binary`, `categorical`, `numerical`, audio variants, `toolcall`	LLM evaluation prompt
`--categories`	string	`categorical`, `audio-categorical`	Comma-separated category values (min 2, max 50)
`--min-value`	number	`numerical`, `audio-numerical`	Minimum value
`--max-value`	number	`numerical`, `audio-numerical`	Maximum value
`--expected-body`	string	`toolcall`	Expected request body, as a JSON value or a plain string
`--match-path`	string	`toolcall`	Dot path into the payload to match on
`--metadata-field-type`	string	`metadata`	Field type: `STRING`, `NUMBER`, or `BOOLEAN`
`--metadata-field-key`	string	`metadata`	Metadata field key to extract
`--regex-pattern`	string	`regex`	Regex pattern for transcript matching
`--role`	string	`regex`	Transcript role to evaluate (`agent` or `user`)
`--match-mode`	string	`regex`	`presence` (default) or `absence` — absence returns 1 if the pattern is NOT found
`--position`	string	`regex`	`any` (default), `first`, or `last` message of the role
`--case-insensitive`	boolean	`regex`	`true` or `false` — case-insensitive matching
`--min-pause-duration`	number	`pause`	Minimum pause duration in seconds (>= 0.5)
`--max-silence-duration-seconds`	number	`pause`	Maximum silence duration in seconds
`--min-silence-gap-seconds`	number	`pause`	Minimum silence gap in seconds
`--criteria-source`	string	`composite`	Where per-criterion checks come from: `test_case` or `metric_metadata`
`--criteria-path`	string	`composite`	Test-case field holding the criteria list (required when `--criteria-source test_case`, e.g. `expected_behaviors`)
`--criteria`	string	`composite`	Comma-separated static criteria (required when `--criteria-source metric_metadata`)
`--reporting-method`	string	`composite`	How to score: `percentage_of_criteria_met`, `count_of_criteria_met`, or `all_criteria_met`
`--base-prompt-template`	string	`composite`	Optional base prompt template for the judge
`--include-traces`	boolean	`composite`	`true` or `false` — include traces in the evaluation
`--ivr-flow`	JSON	`ivr-flow-adherence`	IVR flow graph, e.g. `{"start_node":"welcome","nodes":[{"id":"welcome","prompt":"Press 1"}]}`

The following options configure other metric behaviours the API accepts; descriptions are taken from coval metrics create --help.

Option	Type	Description
`--threshold`	number	Numeric threshold to compare against
`--operator`	string	Comparison operator (`<`, `<=`, `>`, `>=`, `==`, `!=`)
`--frequency-threshold`	number	Frequency threshold
`--direction`	string	Threshold direction (`above` or `below`)
`--percent-above`	number	Percentage of the conversation that must be above the threshold (0-100)
`--success-sentiments`	string	Comma-separated passing sentiments (`Neutral`, `Happy`, `Angry`, `Sad`)
`--success-end-reasons`	string	Comma-separated simulation end reasons that count as success
`--observation-name`	string	Observation name
`--min-volume-change-for-pitch-misalignment`	number	Minimum volume change for pitch misalignment
`--sql-query`	string	SQL query defining a SQL metric

Metric Types

Type	Description	Required options
`llm-binary`	Binary (yes/no) LLM judgment	`--prompt`
`categorical`	Categorical LLM judgment with defined options	`--prompt`, `--categories`
`numerical`	Numerical score from LLM judgment	`--prompt`, `--min-value`, `--max-value`
`audio-binary`	Binary audio analysis	`--prompt`
`audio-categorical`	Categorical audio analysis	`--prompt`, `--categories`
`audio-numerical`	Numerical audio analysis	`--prompt`, `--min-value`, `--max-value`
`toolcall`	Tool call verification	`--prompt` (plus optional `--expected-body`, `--match-path`)
`metadata`	Extract a metadata field value	`--metadata-field-type`, `--metadata-field-key`
`regex`	Match transcript against a regex pattern	`--regex-pattern`
`pause`	Analyze pause durations in audio	`--min-pause-duration`
`composite`	Judge each criterion separately and aggregate	`--criteria-source test_case --criteria-path <field>` or `--criteria-source metric_metadata --criteria "a,b,c"`
`ivr-flow-adherence`	Check whether the caller follows an IVR call graph	`--ivr-flow`

Examples

# LLM Binary
coval metrics create \
  --name "Issue Resolved" \
  --description "Did the agent resolve the customer issue?" \
  --type llm-binary \
  --prompt "Was the customer's issue fully resolved?"

# Categorical
coval metrics create \
  --name "Sentiment" \
  --description "Customer sentiment during the call" \
  --type categorical \
  --categories "positive,neutral,negative" \
  --prompt "What was the customer's overall sentiment?"

# Numerical
coval metrics create \
  --name "Professionalism Score" \
  --description "Rate the agent's professionalism" \
  --type numerical \
  --min-value 1 \
  --max-value 10 \
  --prompt "Rate the agent's professionalism on a scale of 1-10"

# Audio Binary
coval metrics create \
  --name "Background Noise" \
  --description "Is there excessive background noise?" \
  --type audio-binary \
  --prompt "Is there excessive background noise in the audio?"

# Audio Numerical
coval metrics create \
  --name "Audio Clarity" \
  --description "Rate the audio clarity" \
  --type audio-numerical \
  --min-value 1 \
  --max-value 5 \
  --prompt "Rate the audio clarity on a scale of 1-5"

# Tool Call
coval metrics create \
  --name "Lookup Order Called" \
  --description "Did the agent call lookup_order with the right order ID?" \
  --type toolcall \
  --prompt "Was lookup_order called with the order ID the customer gave?" \
  --match-path "arguments.order_id" \
  --expected-body '"ORD-1234"'

# Metadata
coval metrics create \
  --name "Response Time" \
  --description "Extract the response time from metadata" \
  --type metadata \
  --metadata-field-type NUMBER \
  --metadata-field-key "response_time_ms"

# Regex — basic pattern match
coval metrics create \
  --name "Greeting Check" \
  --description "Did the agent greet the customer?" \
  --type regex \
  --regex-pattern "(hello|hi|welcome|good morning)" \
  --role "agent" \
  --case-insensitive true

# Regex — compliance (absence mode)
coval metrics create \
  --name "No Unauthorized Promises" \
  --description "Agent must not make unauthorized promises" \
  --type regex \
  --regex-pattern "(guarantee|promise|definitely)" \
  --role "agent" \
  --match-mode "absence" \
  --case-insensitive true

# Regex — first message disclosure
coval metrics create \
  --name "Recording Disclosure" \
  --description "Agent must state recording disclosure in first message" \
  --type regex \
  --regex-pattern "this call may be recorded" \
  --role "agent" \
  --position "first" \
  --case-insensitive true

# Pause
coval metrics create \
  --name "Long Pauses" \
  --description "Detect pauses longer than 3 seconds" \
  --type pause \
  --min-pause-duration 3.0

# Composite — one verdict per expected behavior on the test case
coval metrics create \
  --name "Expected Behaviors Met" \
  --description "Every expected behavior on the test case is observed" \
  --type composite \
  --criteria-source test_case \
  --criteria-path expected_behaviors \
  --reporting-method all_criteria_met

# IVR flow adherence
coval metrics create \
  --name "Billing Menu Adherence" \
  --description "Caller follows the billing menu path" \
  --type ivr-flow-adherence \
  --ivr-flow '{"start_node":"welcome","nodes":[{"id":"welcome","prompt":"Press 1 for billing, 2 for support","auto_advance_to":"billing"}]}'

Duplicate Metric

Create a copy of an existing metric.

coval metrics duplicate <metric_id>

Argument	Type	Required	Description
`metric_id`	string	Yes	ID of the metric to duplicate

coval metrics duplicate met_abc123

Update Metric

coval metrics update <metric_id> [OPTIONS]

Argument	Type	Required	Description
`metric_id`	string	Yes	The metric ID to update

update accepts the same options as Create Metric; omitted options are left unchanged. Flags can only set a value. To clear a field, send an explicit JSON null through --input-json:

# Change the prompt
coval metrics update met_abc123 --prompt "Updated evaluation prompt"

# Clear the IVR flow graph
coval metrics update met_abc123 --input-json '{"ivr_flow":null}'

# Revert to the platform default judge model
coval metrics update met_abc123 --input-json '{"runtime_config":null}'

Delete Metric

coval metrics delete <metric_id>

Argument	Type	Required	Description
`metric_id`	string	Yes	The metric ID to delete

Test Metric

Run a metric against one or more simulation outputs to test its behavior. This is an asynchronous operation — it returns immediately with metric output IDs that you can use to track the result.

coval metrics test <metric_id> [OPTIONS]

Argument	Type	Required	Description
`metric_id`	string	Yes	The metric ID to test

Option	Type	Required	Description
`--simulation-output-ids`	string	Yes	Comma-separated simulation output IDs to run the metric against (1-100)
`--simulation-output-id`	string	No	A single simulation output ID. Deprecated by the API — use `--simulation-output-ids`
`--dev-id`	string	No	Developer identifier for debugging

coval metrics test met_abc123 --simulation-output-ids simout_def456,simout_ghi789

Metric Versions

List the version history of a metric definition.

coval metrics versions <metric_id>

Argument	Type	Required	Description
`metric_id`	string	Yes	The metric ID

Output columns: Version ID, #, Change Type, Label

coval metrics versions met_abc123

Judge Models

List the models that can be used in --runtime-config when creating or updating a metric.

coval models metric

Output columns: Model ID, Display Name, Provider, Tier

coval models metric --format json | jq '.[].model_id'

Baselines

Baselines learn a metric’s normal value over time and flag anomalous deviations. All baseline commands are nested under coval metrics baselines <metric_id>.

List Baselines

coval metrics baselines <metric_id> list

Output columns: ID, Display Name, Status, Direction, Obs, Baseline

Get Baseline

coval metrics baselines <metric_id> get <baseline_id>

Create Baseline

coval metrics baselines <metric_id> create [OPTIONS]

Option	Type	Description
`--agent-id`	string	Agent ID. Omit for an org-wide baseline
`--test-set-id`	string	Test set ID. Omit for uploaded conversations
`--persona-id`	string	Persona ID. Omit to cover all personas
`--display-name`	string	User-facing label
`--sigma-threshold`	number	Sigmas from the baseline that constitute an anomaly (default 2.0)
`--direction`	string	Which deviations count as anomalous: `BOTH` (default), `ABOVE`, or `BELOW`
`--detection-method`	string	Detection algorithm (default `EWMA`)

coval metrics baselines met_abc123 create \
  --agent-id ag_abc123 \
  --display-name "Support agent — issue resolved" \
  --sigma-threshold 2.5 \
  --direction BELOW

Update Baseline

coval metrics baselines <metric_id> update <baseline_id> [OPTIONS]

Option	Type	Description
`--display-name`	string	New label
`--sigma-threshold`	number	New anomaly threshold in sigmas
`--direction`	string	`BOTH`, `ABOVE`, or `BELOW`
`--status`	string	`ACTIVE`, `STANDBY`, or `ARCHIVED`

coval metrics baselines met_abc123 update 01HXYZ1234567890ABCDEF --status ACTIVE

Delete Baseline

coval metrics baselines <metric_id> delete <baseline_id>

Thresholds

A threshold turns a metric’s value into a pass/fail target. Each metric has at most one threshold, so get and update take no threshold ID. All threshold commands are nested under coval metrics thresholds <metric_id>.

List Thresholds

coval metrics thresholds <metric_id> list

Output columns: Name, Operator, Upper, Lower, Source

Get Threshold

coval metrics thresholds <metric_id> get

Create Threshold

coval metrics thresholds <metric_id> create [OPTIONS]

Option	Type	Description
`--comparison-operator`	string	`eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`, `bt`, or `nbt`
`--target-float-upper`	number	Target value (upper bound for `bt`/`nbt`, the single value for the other numeric operators)
`--target-float-lower`	number	Lower bound for the range operators (`bt`/`nbt`)
`--target-values`	string	Comma-separated target values for the list operators (`in`/`nin`)

# Pass when the score is at least 0.8
coval metrics thresholds met_abc123 create --comparison-operator gte --target-float-upper 0.8

# Pass when the category is one of two values
coval metrics thresholds met_def456 create --comparison-operator in --target-values "positive,neutral"

Update Threshold

coval metrics thresholds <metric_id> update [OPTIONS]

Accepts the same options as create. Omitted options are left unchanged.

coval metrics thresholds met_abc123 update --target-float-upper 0.9

Delete Threshold

coval metrics thresholds <metric_id> delete <threshold_id>

Getting Started

Resources

List Metrics

Get Metric

Create Metric

Common options

Type-specific options

Metric Types

Examples

Duplicate Metric

Update Metric

Delete Metric

Test Metric

Metric Versions

Judge Models

Baselines

List Baselines

Get Baseline

Create Baseline

Update Baseline

Delete Baseline

Thresholds

List Thresholds

Get Threshold

Create Threshold

Update Threshold

Delete Threshold

​List Metrics

​Get Metric

​Create Metric

​Common options

​Type-specific options

​Metric Types

​Examples

​Duplicate Metric

​Update Metric

​Delete Metric

​Test Metric

​Metric Versions

​Judge Models

​Baselines

​List Baselines

​Get Baseline

​Create Baseline

​Update Baseline

​Delete Baseline

​Thresholds

​List Thresholds

​Get Threshold

​Create Threshold

​Update Threshold

​Delete Threshold

List Metrics

Get Metric

Create Metric

Common options

Type-specific options

Metric Types

Examples

Duplicate Metric

Update Metric

Delete Metric

Test Metric

Metric Versions

Judge Models

Baselines

List Baselines

Get Baseline

Create Baseline

Update Baseline

Delete Baseline

Thresholds

List Thresholds

Get Threshold

Create Threshold

Update Threshold

Delete Threshold