> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Metrics

> Manage evaluation metrics with the Coval CLI

`create` and `update` also accept `--input-json` (inline JSON, `@file`, or `-` for stdin); explicit flags override fields from the JSON. See [Input JSON](https://docs.coval.ai/cli/agent-mode.md#input-json).

## List Metrics

```bash theme={null}
coval metrics list [OPTIONS]
```

| Option | Type | Default | Description |
| - | - | - | - |
| `--filter` | string | — | Filter expression (supports `metric_type`, `metric_name`, `create_time`) |
| `--page-size` | number | 50 | Results per page (1-100) |
| `--order-by` | string | `-create_time` | Sort field, prefix with `-` for descending |
| `--include-builtin` | flag | — | Include built-in metrics (e.g. Turn Count, Audio Duration) |

**Output columns:** ID, NAME, TYPE, CREATED

```bash theme={null}
coval metrics list
```

## Get Metric

```bash theme={null}
coval metrics get <metric_id>
```

| Argument | Type | Required | Description |
| - | - | - | - |
| `metric_id` | string | **Yes** | The metric ID |

```bash theme={null}
coval metrics get met_abc123
```

## Create Metric

```bash theme={null}
coval metrics create [OPTIONS]
```

### Common options

| Option | Type | Required | Description |
| - | - | - | - |
| `--name` | string | **Yes** | Metric name (1-200 characters) |
| `--description` | string | **Yes** | Metric description (1-1000 characters) |
| `--type` | string | **Yes** | Metric type (see below) |
| `--target-condition` | JSON | No | Pass/fail target, e.g. `'{"comparison_operator":"gte","target_float":0.8}'` (numeric operators `eq`, `neq`, `gt`, `gte`, `lt`, `lte` with `target_float`; `in`/`nin` with `target_values`) |
| `--runtime-config` | JSON | No | Override the judge model, e.g. `'{"model_version":"<model id>"}'`. List model IDs with `coval models metric`. Not supported for audio metric types |
| `--tags` | string | No | Comma-separated tag names; pass an empty value to clear all tags |

### Type-specific options

Each metric type requires the options marked in the [Metric Types](#metric-types) table. Options are grouped here by the type they configure.

| Option | Type | Applies to | Description |
| - | - | - | - |
| `--prompt` | string | `llm-binary`, `categorical`, `numerical`, audio variants, `toolcall` | LLM evaluation prompt |
| `--categories` | string | `categorical`, `audio-categorical` | Comma-separated category values (min 2, max 50) |
| `--min-value` | number | `numerical`, `audio-numerical` | Minimum value |
| `--max-value` | number | `numerical`, `audio-numerical` | Maximum value |
| `--audio-filter` | JSON | `audio-binary`, `audio-categorical`, `audio-numerical` | JSON string limiting audio analysis to part of the recording, e.g. `'{"include":[{"type":"speaker","speakers":["persona"]}]}'` |
| `--expected-body` | string | `toolcall` | Expected request body, as a JSON value or a plain string |
| `--match-path` | string | `toolcall` | Dot path into the payload to match on |
| `--metadata-field-type` | string | `metadata` | Field type: `STRING`, `NUMBER`, or `BOOLEAN` |
| `--metadata-field-key` | string | `metadata` | Metadata field key to extract |
| `--regex-pattern` | string | `regex` | Regex pattern for transcript matching |
| `--role` | string | `regex` | Transcript role to evaluate (`agent` or `user`) |
| `--match-mode` | string | `regex` | `presence` (default) or `absence` — absence returns 1 if the pattern is NOT found |
| `--position` | string | `regex` | `any` (default), `first`, or `last` message of the role |
| `--case-insensitive` | boolean | `regex` | `true` or `false` — case-insensitive matching |
| `--min-pause-duration` | number | `pause` | Minimum pause duration in seconds (>= 0.5) |
| `--max-silence-duration-seconds` | number | `pause` | Maximum silence duration in seconds |
| `--min-silence-gap-seconds` | number | `pause` | Minimum silence gap in seconds |
| `--criteria-source` | string | `composite` | Where per-criterion checks come from: `test_case` or `metric_metadata` |
| `--criteria-path` | string | `composite` | Test-case field holding the criteria list (required when `--criteria-source test_case`, e.g. `expected_behaviors`) |
| `--criteria` | string | `composite` | Comma-separated static criteria (required when `--criteria-source metric_metadata`) |
| `--reporting-method` | string | `composite` | How to score: `percentage_of_criteria_met`, `count_of_criteria_met`, or `all_criteria_met` |
| `--base-prompt-template` | string | `composite` | Optional base prompt template for the judge |
| `--include-traces` | boolean | `composite` | `true` or `false` — include traces in the evaluation |
| `--ivr-flow` | JSON | `ivr-flow-adherence` | IVR flow graph, e.g. `{"start_node":"welcome","nodes":[{"id":"welcome","prompt":"Press 1"}]}` |

The following options configure other metric behaviours the API accepts; descriptions are taken from `coval metrics create --help`.

| Option | Type | Description |
| - | - | - |
| `--threshold` | number | Numeric threshold to compare against |
| `--operator` | string | Comparison operator (`<`, `<=`, `>`, `>=`, `==`, `!=`) |
| `--frequency-threshold` | number | Frequency threshold |
| `--direction` | string | Threshold direction (`above` or `below`) |
| `--percent-above` | number | Percentage of the conversation that must be above the threshold (0-100) |
| `--success-sentiments` | string | Comma-separated passing sentiments (`Neutral`, `Happy`, `Angry`, `Sad`) |
| `--success-end-reasons` | string | Comma-separated simulation end reasons that count as success |
| `--observation-name` | string | Observation name |
| `--min-volume-change-for-pitch-misalignment` | number | Minimum volume change for pitch misalignment |
| `--sql-query` | string | SQL query defining a SQL metric |

### Additional fields through JSON

CLI 0.9.0 and later accept these additional metric fields on `create` and `update` through `--input-json` (`audio_filter` and `dependency_metric_ids` were added in CLI 0.9.1):

| Fields | Type | Purpose |
| - | - | - |
| `judge_mode` | string | Judge mode |
| `enabled_tools` | array of strings | Agent Judge evidence tools; an empty array disables all tools |
| `dependency_metric_ids` | array of strings | Metrics that an Agent Judge metric depends on; an empty array clears them |
| `aggregation_method`, `unit` | string | Aggregation and display unit |
| `value_source`, `span_name`, `metric_attribute` | string | Custom-trace value selection |
| `metric_metadata` | JSON | Metric metadata |
| `audio_filter` | JSON | Limits an audio metric to part of the recording (same payload as `--audio-filter`) |
| `detection_preset`, `pause_detection_preset`, `threshold_preset` | string | Audio detection and threshold presets |
| `harmonics_to_noise_ratio_threshold_offset_db`, `jitter_threshold_multiplier`, `loud_threshold_db`, `low_pitch_threshold_multiplier`, `mad_z_score_threshold`, `min_fry_segment_seconds`, `pitch_change_threshold_hz`, `significant_changes_threshold_hz`, `soft_threshold_db` | number | Audio analysis thresholds |

Use fields appropriate to the metric type; JSON field names use underscores rather than flag-style hyphens. On update, `aggregation_method: null` resets aggregation to the default and `unit: null` clears the display unit. Omitting either field leaves it unchanged.

```bash theme={null}
coval metrics update met_abc123 --input-json '{"aggregation_method":null,"unit":null}'
```

### Metric Types

| Type | Description | Required options |
| - | - | - |
| `llm-binary` | Binary (yes/no) LLM judgment | `--prompt` |
| `categorical` | Categorical LLM judgment with defined options | `--prompt`, `--categories` |
| `numerical` | Numerical score from LLM judgment | `--prompt`, `--min-value`, `--max-value` |
| `audio-binary` | Binary audio analysis | `--prompt` |
| `audio-categorical` | Categorical audio analysis | `--prompt`, `--categories` |
| `audio-numerical` | Numerical audio analysis | `--prompt`, `--min-value`, `--max-value` |
| `toolcall` | Tool call verification | `--prompt` (plus optional `--expected-body`, `--match-path`) |
| `metadata` | Extract a metadata field value | `--metadata-field-type`, `--metadata-field-key` |
| `regex` | Match transcript against a regex pattern | `--regex-pattern` |
| `pause` | Analyze pause durations in audio | `--min-pause-duration` |
| `composite` | Judge each criterion separately and aggregate | `--criteria-source test_case --criteria-path <field>` or `--criteria-source metric_metadata --criteria "a,b,c"` |
| `ivr-flow-adherence` | Check whether the caller follows an IVR call graph | `--ivr-flow` |

### Examples

```bash theme={null}
# LLM Binary
coval metrics create \
  --name "Issue Resolved" \
  --description "Did the agent resolve the customer issue?" \
  --type llm-binary \
  --prompt "Was the customer's issue fully resolved?"

# Categorical
coval metrics create \
  --name "Sentiment" \
  --description "Customer sentiment during the call" \
  --type categorical \
  --categories "positive,neutral,negative" \
  --prompt "What was the customer's overall sentiment?"

# Numerical
coval metrics create \
  --name "Professionalism Score" \
  --description "Rate the agent's professionalism" \
  --type numerical \
  --min-value 1 \
  --max-value 10 \
  --prompt "Rate the agent's professionalism on a scale of 1-10"

# Audio Binary
coval metrics create \
  --name "Background Noise" \
  --description "Is there excessive background noise?" \
  --type audio-binary \
  --prompt "Is there excessive background noise in the audio?"

# Audio Binary — restricted to the caller's audio
coval metrics create \
  --name "Caller Politeness" \
  --description "The caller stays polite" \
  --type audio-binary \
  --prompt "Was the caller polite?" \
  --audio-filter '{"include":[{"type":"speaker","speakers":["persona"]}]}'

# Audio Numerical
coval metrics create \
  --name "Audio Clarity" \
  --description "Rate the audio clarity" \
  --type audio-numerical \
  --min-value 1 \
  --max-value 5 \
  --prompt "Rate the audio clarity on a scale of 1-5"

# Tool Call
coval metrics create \
  --name "Lookup Order Called" \
  --description "Did the agent call lookup_order with the right order ID?" \
  --type toolcall \
  --prompt "Was lookup_order called with the order ID the customer gave?" \
  --match-path "arguments.order_id" \
  --expected-body '"ORD-1234"'

# Metadata
coval metrics create \
  --name "Response Time" \
  --description "Extract the response time from metadata" \
  --type metadata \
  --metadata-field-type NUMBER \
  --metadata-field-key "response_time_ms"

# Regex — basic pattern match
coval metrics create \
  --name "Greeting Check" \
  --description "Did the agent greet the customer?" \
  --type regex \
  --regex-pattern "(hello|hi|welcome|good morning)" \
  --role "agent" \
  --case-insensitive true

# Regex — compliance (absence mode)
coval metrics create \
  --name "No Unauthorized Promises" \
  --description "Agent must not make unauthorized promises" \
  --type regex \
  --regex-pattern "(guarantee|promise|definitely)" \
  --role "agent" \
  --match-mode "absence" \
  --case-insensitive true

# Regex — first message disclosure
coval metrics create \
  --name "Recording Disclosure" \
  --description "Agent must state recording disclosure in first message" \
  --type regex \
  --regex-pattern "this call may be recorded" \
  --role "agent" \
  --position "first" \
  --case-insensitive true

# Pause
coval metrics create \
  --name "Long Pauses" \
  --description "Detect pauses longer than 3 seconds" \
  --type pause \
  --min-pause-duration 3.0

# Composite — one verdict per expected behavior on the test case
coval metrics create \
  --name "Expected Behaviors Met" \
  --description "Every expected behavior on the test case is observed" \
  --type composite \
  --criteria-source test_case \
  --criteria-path expected_behaviors \
  --reporting-method all_criteria_met

# IVR flow adherence
coval metrics create \
  --name "Billing Menu Adherence" \
  --description "Caller follows the billing menu path" \
  --type ivr-flow-adherence \
  --ivr-flow '{"start_node":"welcome","nodes":[{"id":"welcome","prompt":"Press 1 for billing, 2 for support","auto_advance_to":"billing"}]}'
```

## Duplicate Metric

Create a copy of an existing metric.

```bash theme={null}
coval metrics duplicate <metric_id>
```

| Argument | Type | Required | Description |
| - | - | - | - |
| `metric_id` | string | **Yes** | ID of the metric to duplicate |

```bash theme={null}
coval metrics duplicate met_abc123
```

## Update Metric

```bash theme={null}
coval metrics update <metric_id> [OPTIONS]
```

| Argument | Type | Required | Description |
| - | - | - | - |
| `metric_id` | string | **Yes** | The metric ID to update |

`update` accepts the same options as [Create Metric](#create-metric); omitted options are left unchanged. Flags can only set a value. To clear `audio_filter`, `ivr_flow`, `aggregation_method`, or `unit`, send an explicit JSON `null` through `--input-json`; to clear `dependency_metric_ids`, send `[]` (or `null`):

```bash theme={null}
# Change the prompt
coval metrics update met_abc123 --prompt "Updated evaluation prompt"

# Clear the audio filter
coval metrics update met_abc123 --input-json '{"audio_filter":null}'

# Clear the IVR flow graph
coval metrics update met_abc123 --input-json '{"ivr_flow":null}'

# Clear saved Agent Judge metric dependencies
coval metrics update met_abc123 --input-json '{"dependency_metric_ids":[]}'
```

## Delete Metric

```bash theme={null}
coval metrics delete <metric_id>
```

| Argument | Type | Required | Description |
| - | - | - | - |
| `metric_id` | string | **Yes** | The metric ID to delete |

## Test Metric

Run a metric against one or more simulation outputs to test its behavior. This is an asynchronous operation — it returns immediately with metric output IDs that you can use to track the result.

```bash theme={null}
coval metrics test <metric_id> [OPTIONS]
```

| Argument | Type | Required | Description |
| - | - | - | - |
| `metric_id` | string | **Yes** | The metric ID to test |

| Option | Type | Required | Description |
| - | - | - | - |
| `--simulation-output-ids` | string | **Yes** | Comma-separated simulation output IDs to run the metric against (1-100) |
| `--simulation-output-id` | string | No | A single simulation output ID. **Deprecated** by the API — use `--simulation-output-ids` |
| `--dev-id` | string | No | Developer identifier for debugging |

```bash theme={null}
coval metrics test met_abc123 --simulation-output-ids simout_def456,simout_ghi789
```

## Metric Versions

List the version history of a metric definition.

```bash theme={null}
coval metrics versions <metric_id>
```

| Argument | Type | Required | Description |
| - | - | - | - |
| `metric_id` | string | **Yes** | The metric ID |

**Output columns:** Version ID, #, Change Type, Label

```bash theme={null}
coval metrics versions met_abc123
```

## Judge Models

List the models that can be used in `--runtime-config` when creating or updating a metric.

```bash theme={null}
coval models metric
```

**Output columns:** Model ID, Display Name, Provider, Tier

```bash theme={null}
coval models metric --format json | jq '.[].model_id'
```

***

## Baselines

Baselines learn a metric's normal value over time and flag anomalous deviations. All baseline commands are nested under `coval metrics baselines <metric_id>`.

### List Baselines

```bash theme={null}
coval metrics baselines <metric_id> list
```

**Output columns:** ID, Display Name, Status, Direction, Obs, Baseline

### Get Baseline

```bash theme={null}
coval metrics baselines <metric_id> get <baseline_id>
```

### Create Baseline

```bash theme={null}
coval metrics baselines <metric_id> create [OPTIONS]
```

| Option | Type | Description |
| - | - | - |
| `--agent-id` | string | Agent ID. Omit for an org-wide baseline |
| `--test-set-id` | string | Test set ID. Omit for uploaded conversations |
| `--persona-id` | string | Persona ID. Omit to cover all personas |
| `--display-name` | string | User-facing label |
| `--sigma-threshold` | number | Sigmas from the baseline that constitute an anomaly (default 2.0) |
| `--direction` | string | Which deviations count as anomalous: `BOTH` (default), `ABOVE`, or `BELOW` |
| `--detection-method` | string | Detection algorithm (default `EWMA`) |

```bash theme={null}
coval metrics baselines met_abc123 create \
  --agent-id ag_abc123 \
  --display-name "Support agent — issue resolved" \
  --sigma-threshold 2.5 \
  --direction BELOW
```

### Update Baseline

```bash theme={null}
coval metrics baselines <metric_id> update <baseline_id> [OPTIONS]
```

| Option | Type | Description |
| - | - | - |
| `--display-name` | string | New label |
| `--sigma-threshold` | number | New anomaly threshold in sigmas |
| `--direction` | string | `BOTH`, `ABOVE`, or `BELOW` |
| `--status` | string | `ACTIVE`, `STANDBY`, or `ARCHIVED` |

```bash theme={null}
coval metrics baselines met_abc123 update 01HXYZ1234567890ABCDEF --status ACTIVE
```

### Delete Baseline

```bash theme={null}
coval metrics baselines <metric_id> delete <baseline_id>
```

## Thresholds

A threshold turns a metric's value into a pass/fail target. Each metric has at most one threshold, so `get` and `update` take no threshold ID. All threshold commands are nested under `coval metrics thresholds <metric_id>`.

### List Thresholds

```bash theme={null}
coval metrics thresholds <metric_id> list
```

**Output columns:** Name, Operator, Upper, Lower, Source

### Get Threshold

```bash theme={null}
coval metrics thresholds <metric_id> get
```

### Create Threshold

```bash theme={null}
coval metrics thresholds <metric_id> create [OPTIONS]
```

| Option | Type | Description |
| - | - | - |
| `--comparison-operator` | string | `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`, `bt`, or `nbt` |
| `--target-float-upper` | number | Target value (upper bound for `bt`/`nbt`, the single value for the other numeric operators) |
| `--target-float-lower` | number | Lower bound for the range operators (`bt`/`nbt`) |
| `--target-values` | string | Comma-separated target values for the list operators (`in`/`nin`) |

```bash theme={null}
# Pass when the score is at least 0.8
coval metrics thresholds met_abc123 create --comparison-operator gte --target-float-upper 0.8

# Pass when the category is one of two values
coval metrics thresholds met_def456 create --comparison-operator in --target-values "positive,neutral"
```

### Update Threshold

```bash theme={null}
coval metrics thresholds <metric_id> update [OPTIONS]
```

Accepts the same options as create. Omitted options are left unchanged.

```bash theme={null}
coval metrics thresholds met_abc123 update --target-float-upper 0.9
```

### Delete Threshold

```bash theme={null}
coval metrics thresholds <metric_id> delete <threshold_id>
```

## Python Metric Drafts

CLI 0.10.0 and later provide `python-metric-drafts` commands. They work only when Python metrics are enabled for your Coval environment; otherwise the API returns `403` with `Python metrics are not enabled.`

Author metrics in Python with a draft, preview, and publication workflow. A draft defines a Python `evaluate(conversation)` function; a preview runs it against a simulation output before publication.

### Create Draft

```bash theme={null}
coval python-metric-drafts create [OPTIONS]
```

The API requires `display_name`, `source`, and `output_type`. Supply them as flags or through `--input-json` (inline JSON, `@file`, or `-` for stdin). No individual flag is required when JSON supplies the field. Explicit flags override JSON fields; input requirement flags override the corresponding properties in `--requirements` or JSON input. See [Input JSON](https://docs.coval.ai/cli/agent-mode.md#input-json).

| Option | Type | Required | Description |
| - | - | - | - |
| `--display-name` | string | No | Metric display name (1-200 characters) |
| `--source` | string | No | Python source defining `evaluate(conversation)`. Conflicts with `--source-file` |
| `--source-file` | path | No | Path to a `.py` file defining `evaluate(conversation)`. Conflicts with `--source` |
| `--output-type` | string | No | Scalar return type: `number` or `boolean` |
| `--target-metric-id` | string | No | Existing metric to version-bump |
| `--target-version-id` | string | No | Existing metric version to supersede |
| `--description` | string | No | What this metric measures |
| `--unit` | string | No | Optional numeric unit; omit for unitless |
| `--requirements` | JSON object | No | Required conversation inputs, supplied as a JSON string |
| `--require-audio` | boolean | No | Require audio in the evaluation input (default: `false`) |
| `--require-traces` | boolean | No | Require traces in the evaluation input (default: `false`) |
| `--frame-table` | string | No | Frame table required by the evaluation input (e.g., `turns`). Repeat for multiple |
| `--input-json` | JSON | No | Provide draft fields as inline JSON, `@file`, or `-` for stdin |

```bash theme={null}
coval python-metric-drafts create \
  --display-name "Issue Resolved" \
  --description "Whether the agent resolved the issue" \
  --source-file ./issue_resolved.py \
  --output-type boolean
```

### Get Draft

```bash theme={null}
coval python-metric-drafts get <draft_id>
```

Returns the draft, its current version, and fingerprint. The fingerprint identifies the exact definition approved by a preview. A draft is immutable: changing the source or input/output settings requires a new draft and new previews.

### Create Preview

Run the draft against exactly one simulation output. Preview evaluation is asynchronous.

```bash theme={null}
coval python-metric-drafts previews <draft_id> create --simulation-output-ids <simulation_output_id>
```

| Argument / Option | Type | Required | Description |
| - | - | - | - |
| `draft_id` | string | **Yes** | The draft ID |
| `--simulation-output-ids` | string | **Yes** | Exactly one simulation output ID |

### Get Preview

```bash theme={null}
coval python-metric-drafts previews <draft_id> get <preview_id>
```

Poll until the preview reaches a terminal state and inspect `status`, `value`, `explanation`, and `error`. In [Agent Mode](https://docs.coval.ai/cli/agent-mode.md), a `COMPLETED` status offers publication as the next action; a queued preview carrying an unconfirmed-admission warning may still run, so keep polling it before retrying. A terminal `SKIPPED` preview retains its diagnostic without offering polling or publication.

A successful value can be numeric zero or boolean false; neither value alone indicates a failed preview. Numeric results must be finite; aggregation and timestamped subvalues are not supported by this workflow.

### Cancel Preview

```bash theme={null}
coval python-metric-drafts previews <draft_id> cancel <preview_id>
```

Cancels a pending preview. An already running evaluation cannot be cancelled retroactively.

### Publish Draft

Publish the approved draft version. Requires the preview ID and the exact draft fingerprint from the draft create/get response (`draft.fingerprint`).

```bash theme={null}
coval python-metric-drafts publish <draft_id> --preview-ids <preview_id> --fingerprint <draft_fingerprint>
```

| Argument / Option | Type | Required | Description |
| - | - | - | - |
| `draft_id` | string | **Yes** | The draft ID |
| `--preview-ids` | string | **Yes** | One to ten successful preview IDs approving the definition, separated by commas |
| `--fingerprint` | string | **Yes** | The exact 64-character lowercase hexadecimal fingerprint at the previewed version, from the draft create/get response |

Publication returns the resulting `metric_id` and `version_id`.

```bash theme={null}
# Full authoring workflow
coval python-metric-drafts create --display-name "Issue Resolved" \
  --source-file ./issue_resolved.py --output-type boolean
coval python-metric-drafts previews draft_123 create --simulation-output-ids simout_456
coval python-metric-drafts previews draft_123 get preview_789
coval python-metric-drafts publish draft_123 \
  --preview-ids preview_789 \
  --fingerprint <draft_fingerprint_from_create_or_get>
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.