Tools Reference - Coval Documentation

Sofia

consult_sofia

Delegate a read-only evaluation question to Sofia. Sofia can use Coval product knowledge and inspect the authenticated organization’s runs, simulations, conversations, metrics, agents, personas, test sets, and dashboards. It cannot create, update, run, or delete resources.

Parameter	Type	Required	Description
`prompt`	string	Yes	Question or task for Sofia, up to 12,000 characters
`conversation`	object[]	No	Up to 12 prior messages needed for context
`session_id`	string	No	Stable, non-secret caller session identifier, up to 128 letters, numbers, `.`, `_`, `:`, or `-` characters

Each conversation item must contain exactly the message content needed for the current request:

Field	Type	Required	Limits
`role`	string	Yes	`user` or `assistant`
`content`	string	Yes	Non-empty text, up to 12,000 characters

Do not include API keys, access tokens, or other secrets in conversation context or session_id. For service monitoring and evaluation, Coval records the prompt, the optional conversation messages supplied to this tool, and Sofia’s response. Messages elsewhere in the parent Claude, Codex, or other client session are not sent to Coval unless the client includes them in this tool call. Use direct tools for deterministic reads and writes. Use consult_sofia when the task benefits from evaluation expertise or synthesis across several Coval resources.

Run Management

list_runs

List evaluation runs with filtering and pagination.

Parameter	Type	Required	Description
`page_size`	number	No	Results per page (1-100, default: 50)
`page_token`	string	No	Pagination token from previous response
`order_by`	string	No	Sort order (e.g., `-create_time` for newest first)
`filter`	string	No	Filter expression (e.g., `status="COMPLETED"`)

get_run

Get detailed information about a specific run.

Parameter	Type	Required	Description
`run_id`	string	Yes	The unique run ID

Returns run details including status, progress, and metrics (if completed).

create_run

Launch a new evaluation run.

Parameter	Type	Required	Description
`agent_id`	string	Yes	Agent ID from `list_agents`
`persona_id`	string	Yes	Persona ID from `list_personas`
`test_set_id`	string	Yes	Test set ID from `list_test_sets`
`metric_ids`	string[]	No	Specific metrics to evaluate
`options.iteration_count`	number	No	Iterations per test case (1-10, default: 1)
`options.concurrency`	number	No	Parallel simulations (1-5, default: 1)
`tags`	string[]	No	Up to 20 tags, 200 characters each
`metadata`	object	No	Custom metadata for tracking

Agent Management

list_agents

List all configured agents.

Parameter	Type	Required	Description
`page_size`	number	No	Results per page (1-100, default: 50)
`page_token`	string	No	Pagination token
`order_by`	string	No	Sort order
`filter`	string	No	Filter by `model_type`, `display_name`, etc.

get_agent

Get detailed configuration for a specific agent.

Parameter	Type	Required	Description
`agent_id`	string	Yes	Agent ID from `list_agents`

create_agent

Create a new agent configuration.

Parameter	Type	Required	Description
`display_name`	string	Yes	Human-readable name (1-200 chars)
`model_type`	string	Yes	Agent type (see below)
`phone_number`	string	No	E.164 format for voice agents
`endpoint`	string	No	Webhook or WebSocket URL
`prompt`	string	No	System prompt/instructions
`metadata`	object	No	Custom metadata

Model Types:

MODEL_TYPE_VOICE - Inbound voice
MODEL_TYPE_OUTBOUND_VOICE - Outbound voice
MODEL_TYPE_CHAT - Chat/text
MODEL_TYPE_SMS - SMS messaging
MODEL_TYPE_WEBSOCKET - WebSocket

update_agent

Update an existing agent configuration.

Parameter	Type	Required	Description
`agent_id`	string	Yes	Agent to update
`display_name`	string	No	New name
`phone_number`	string	No	New phone number
`endpoint`	string	No	New endpoint URL
`prompt`	string	No	New system prompt
`metadata`	object	No	New metadata

Test Set Management

list_test_sets

List all test sets available for evaluation.

Parameter	Type	Required	Description
`page_size`	number	No	Results per page (1-100, default: 50)
`page_token`	string	No	Pagination token
`order_by`	string	No	Sort order
`filter`	string	No	Filter expression

get_test_set

Get detailed information about a test set.

Parameter	Type	Required	Description
`test_set_id`	string	Yes	Test set ID from `list_test_sets`

create_test_set

Create a new test set.

Parameter	Type	Required	Description
`display_name`	string	Yes	Test set name (1-100 chars)
`slug`	string	No	URL-friendly ID (auto-generated if omitted)
`description`	string	No	Test set description
`test_set_type`	string	No	`DEFAULT`, `SCENARIO`, `TRANSCRIPT`, or `WORKFLOW`
`test_set_metadata`	object	No	Configuration metadata
`parameters`	object	No	Test parameterization

Test Case Management

list_test_cases

List test cases with optional filtering by test set.

Parameter	Type	Required	Description
`page_size`	number	No	Results per page (1-100, default: 50)
`page_token`	string	No	Pagination token
`order_by`	string	No	Sort order
`filter`	string	No	Filter expression, for example `test_set_id="abc12345"`

get_test_case

Get detailed information about a test case.

Parameter	Type	Required	Description
`test_case_id`	string	Yes	Test case ID from `list_test_cases`

create_test_case

Create a new test case in a test set.

Parameter	Type	Required	Description
`test_set_id`	string	Yes	Eight-character test set ID
`input_str`	string	Yes	Scenario or input presented to the agent
`expected_behaviors`	string[]	No	Behaviors expected from the agent
`description`	string	No	Test case description
`simulation_metadata_input`	object	No	Context passed to the simulation environment
`metric_input`	object	No	Custom metric inputs
`user_notes`	string	No	Internal notes

update_test_case

Update an existing test case.

Parameter	Type	Required	Description
`test_case_id`	string	Yes	Test case to update
`input_str`	string	No	Updated scenario or input
`expected_behaviors`	string[]	No	Updated expected behaviors
`description`	string	No	New description
`simulation_metadata_input`	object	No	Updated simulation context
`metric_input`	object	No	Updated metric inputs
`user_notes`	string	No	Updated internal notes

Metrics

list_metrics

List available evaluation metrics.

Parameter	Type	Required	Description
`page_size`	number	No	Results per page (1-100, default: 50)
`page_token`	string	No	Pagination token
`order_by`	string	No	Sort order
`filter`	string	No	Filter expression
`include_builtin`	boolean	No	Include built-in metrics

get_metric

Get detailed configuration for a specific metric.

Parameter	Type	Required	Description
`metric_id`	string	Yes	Metric ID from `list_metrics`

Personas

list_personas

List available simulated personas for testing.

Parameter	Type	Required	Description
`page_size`	number	No	Results per page (1-100, default: 50)
`page_token`	string	No	Pagination token
`order_by`	string	No	Sort order
`filter`	string	No	Filter expression

get_persona

Get detailed configuration for a specific persona.

Parameter	Type	Required	Description
`persona_id`	string	Yes	Persona ID from `list_personas`

Returns persona configuration including voice settings, language, and behavior.

​Sofia

​consult_sofia

​Run Management

​list_runs

​get_run

​create_run

​Agent Management

​list_agents

​get_agent

​create_agent

​update_agent

​Test Set Management

​list_test_sets

​get_test_set

​create_test_set

​Test Case Management

​list_test_cases

​get_test_case

​create_test_case

​update_test_case

​Metrics

​list_metrics

​get_metric

​Personas

​list_personas

​get_persona