Test Cases

create and update also accept --input-json (inline JSON, @file, or - for stdin); explicit flags override fields from the JSON. See Input JSON.

List Test Cases

coval test-cases list [OPTIONS]

Option	Type	Default	Description
`--filter`	string	—	Filter expression (e.g., `test_set_id=abc12345`)
`--test-set-id`	string	—	Filter by test set ID (8-char ID)
`--page-size`	number	50	Results per page (1-100)
`--order-by`	string	`-create_time`	Sort field, prefix with `-` for descending

Output columns: ID, INPUT, TYPE, TEST SET, CREATED

# List all test cases
coval test-cases list

# Filter by test set
coval test-cases list --test-set-id ts_abc123

Get Test Case

coval test-cases get <test_case_id>

Argument	Type	Required	Description
`test_case_id`	string	Yes	The test case ID

coval test-cases get tc_abc123

Create Test Case

Create a single test case or bulk import from stdin.

coval test-cases create [OPTIONS]

Option	Type	Required	Description
`--test-set-id`	string	Yes	Test set to add the case to (8-char ID)
`--input`	string	No	Test case input text
`--expected-behavior`	string	No	Expected behavior; repeat the flag for multiple behaviors
`--expected`	string	No	Single expected output text. Deprecated — prefer `--expected-behavior`
`--description`	string	No	Human-readable description
`--input-type`	string	No	Input type: `SCENARIO` (default), `TRANSCRIPT`, `IVR`, `AUDIO`, `MANUAL`, or `SCRIPT`
`--stdin`	flag	No	Read one JSON object per line from stdin for bulk creation

The input must come from exactly one source: --input, --input-json, or --stdin. --stdin cannot be combined with the other per-case flags.

Single Test Case

coval test-cases create \
  --test-set-id ts_abc123 \
  --input "I need help with my order" \
  --expected-behavior "Asks for the order number" \
  --expected-behavior "Offers to look up the order status" \
  --description "Basic order help request"

Bulk Import from Stdin

Pass --stdin to read one JSON object per line. Each line is a full create request (the same fields --input-json accepts); test_set_id comes from the flag.

echo '{"input_str": "I need a refund", "expected_behaviors": ["Confirms the refund policy"], "description": "Refund request"}
{"input_str": "Where is my order?", "expected_behaviors": ["Asks for the order number"], "description": "Order tracking"}' \
  | coval test-cases create --test-set-id ts_abc123 --stdin

Or import from a file:

cat test_cases.jsonl | coval test-cases create --test-set-id ts_abc123 --stdin

Each line must be valid JSON with the following fields:

Field	Type	Required	Description
`input_str`	string	Yes	Input text
`expected_behaviors`	string[]	No	Expected behaviors (preferred)
`expected_output_str`	string	No	Single expected output text (legacy)
`expected_output_json`	object	No	Structured expected output
`description`	string	No	Description
`input_type`	string	No	`SCENARIO` (default), `TRANSCRIPT`, `IVR`, `AUDIO`, `MANUAL`, or `SCRIPT`
`script_turns`	array	Conditional	Ordered persona turns; required when `input_type` is `SCRIPT` (see below)
`simulation_metadata_input`	object	No	Simulation metadata for the case
`metric_input`	object	No	Input passed to metrics
`user_notes`	string	No	Free-text notes

SCRIPT test cases

A SCRIPT test case replaces the persona’s improvised turns with a fixed sequence. Provide script_turns at the top level of the request; each entry is one of:

Shape	Meaning
`"Hello, I'd like to check my balance."`	Spoken text (a bare string)
`{"type": "text", "text": "..."}`	Spoken text, explicit form
`{"type": "dtmf", "digits": "2"}`	Keypad presses (digits 0-9, `*`, `#`, and phone punctuation)
`{"type": "skip"}`	The persona stays silent for one turn

coval test-cases create \
  --test-set-id ts_abc123 \
  --input-type SCRIPT \
  --input "Caller navigates the billing IVR" \
  --input-json '{
    "script_turns": [
      "Hi, I have a question about my bill.",
      {"type": "dtmf", "digits": "2"},
      {"type": "skip"},
      "Yes, that is the right account."
    ]
  }'

simulation_metadata_input.script_turns is the legacy form and is still accepted, but new integrations should use the top-level script_turns field. Changing input_type away from SCRIPT clears the stored turns.

Update Test Case

coval test-cases update <test_case_id> [OPTIONS]

Argument	Type	Required	Description
`test_case_id`	string	Yes	The test case ID to update

Option	Type	Description
`--input`	string	New input text
`--expected-behavior`	string	New expected behavior; repeat the flag to replace the list with multiple behaviors
`--expected`	string	New single expected output text. Deprecated — prefer `--expected-behavior`
`--description`	string	New description
`--input-type`	string	New input type

coval test-cases update tc_abc123 --input "Updated input text"

# Replace the expected behaviors
coval test-cases update tc_abc123 \
  --expected-behavior "Greets the caller" \
  --expected-behavior "Verifies identity before sharing account details"

Delete Test Case

coval test-cases delete <test_case_id>

Argument	Type	Required	Description
`test_case_id`	string	Yes	The test case ID to delete

Media Upload URL

Request a presigned URL for uploading a media file to a test case.

coval test-cases media-upload-url <test_case_id> --filename <name> [OPTIONS]

Argument	Type	Required	Description
`test_case_id`	string	Yes	The test case ID (8-char)

Option	Type	Required	Description
`--filename`	string	Yes	Name of the file to upload
`--mime-type`	string	No	MIME type of the file (default `application/octet-stream`)

coval test-cases media-upload-url tc_abc123 --filename greeting.wav --mime-type audio/wav

Getting Started

Resources

List Test Cases

Get Test Case

Create Test Case

Single Test Case

Bulk Import from Stdin

SCRIPT test cases

Update Test Case

Delete Test Case

Media Upload URL

​List Test Cases

​Get Test Case

​Create Test Case

​Single Test Case

​Bulk Import from Stdin

​SCRIPT test cases

​Update Test Case

​Delete Test Case

​Media Upload URL

List Test Cases

Get Test Case

Create Test Case

Single Test Case

Bulk Import from Stdin

SCRIPT test cases

Update Test Case

Delete Test Case

Media Upload URL