Changelog

Install Coval from the ChatGPT plugin listing or Claude connector directory to inspect evaluation data, manage supported resources, launch supported runs, and consult Sofia from the assistant where you already work.

You can now rerun metrics across many simulations at once through the v1 API, with a new rerun-metrics endpoint, so you do not have to trigger them one at a time.

Human Review projects now support a required manager, membership history, and a membership audit log, so you can see who has access to a project and when it changed.

Reviewers can now mark an assignment review complete directly, making it clearer when a set of labels is finished.

Sofia, the in-app agent, now streams its responses as they generate, with speed controls, so answers start appearing right away.

Test run names now display correctly in the runs list.

Reviewers can now label conversations blind, without seeing existing or AI-generated labels, so human judgments stay independent. Blind mode can be set per metric.

Human Review project insights add a per-labeler assignee tab and a per-labeler-per-day activity plot, so you can see who labeled what and how labeling is progressing.

Metric configurations are now validated as you set them up, so misconfigured metrics are caught early instead of failing at run time.

Reports now remember your simulation-level scope, so a report keeps the exact set of simulations you selected.

Sofia, the in-app agent, can now take Human Review actions and summarize behavioral outcomes from your runs.

Deleted test sets, personas, and agents can now be restored from a Recently Deleted view, so an accidental delete is no longer permanent.

Reports built from a Human Review project can now filter to just the human-labeled simulations, so the report reflects only the conversations your team reviewed.

Default metric rules can now match more than one metadata condition at once, combined with AND, so you can target metrics more precisely.

Creating a new agent now opens a type picker for voice, text, and advanced setups instead of defaulting straight to voice.

Sofia, the in-app agent, can now generate charts and visualizations in its answers when you ask about your results.

You can now add tags to a metric directly from the metric page, without going back to the list.

Simulation results now include the caller’s source and destination phone numbers, in the app and through the API.

Scatter plots in Multi-Run Analysis add an exclude-outliers toggle that dims Tukey IQR outliers and drops them from the trend line, plus a legend toggle and a point-count caption for shared reports.

A test set’s last-edited time now updates correctly after edits made through the API.

Deleted simulations and runs no longer appear in report rows.

Sofia is an in-app agent that helps you set up tests, create metrics, and dig into results through chat, without leaving Coval.

Chat agents now resolve test case and agent template variables in custom headers, request payloads, and initialization payloads, so you can drive per-scenario values into your endpoint.

Metrics that do not apply to a call now skip with a clear reason instead of a generic failure, and LLM judge metrics retry automatically on truncated or off-list responses, so you see fewer spurious failures.

Metrics now keep a version history, so you can see how a metric changed over time and review prior versions.

A library of ready-made LLM judge metric templates is now available when you create a metric, and the metric gallery is reorganized into clearer categories.

Runs now include an AI-generated summary of what happened, available in beta. Mark a summary helpful or not to help it improve.

Human Review now has project-scoped pages, so you can open a project’s overview and assignments on their own pages and share a direct link to them.

Coval now validates Pipecat, LiveKit, and WebSocket agent connections when you set them up, so configuration problems surface before you run.

You can now upload your own background sounds for simulations, so agents can be tested against the exact ambient conditions they will face in production.

Telephony call recordings in MP3 format now upload reliably, including low-sample-rate recordings that were previously rejected.

A new metric that flags whether an agent sounds monotone or expressive across a call.

A new metric that measures perceived loudness across a call, so you can catch audio that is too quiet or too loud.

Create and delete dashboards programmatically through the API and CLI.

Chat agents now support SSE streaming and a configurable response format.

Dynamic metrics can now reference per-conversation metadata directly in the prompt template.

When your organization is running evaluations beyond your concurrency limits, we will store your data but flag the evaluations as an error. This lets you rerun them later, once you are operating within your concurrency limits.

Metric descriptions auto-populate and the name auto-fills when you create a metric.

Categorical audio metrics now flag a clear, descriptive error when no categories are configured, so misconfigurations surface right away.

A built-in metric that checks whether a call follows the intended IVR navigation path, with no custom scoring logic required.

Define and simulate branching IVR call flows visually, directly in Coval.

Choose how per-turn scores roll up across a multi-turn trace, for example worst-case or first-occurrence.

Apply custom tags to metrics, run templates, and test sets to organize and filter your library.

New voices are available in the persona picker, ready to use with no setup.

Custom sign-in now works across all configured authentication methods.

Password-protected shared runs now open reliably across different saved password formats.

Large reports that used to struggle to load now open reliably even at scale, thanks to a change in how we load them.

The metric picker now uses grouped, nested categories, so the right metric is faster to find and apply.

There is no longer a 1000-character limit on metadata for uploaded conversations.

Auto-generated metrics are now reliably tagged LLM Judge, so AI-scored and rule-based metrics are easy to tell apart.

Speech anomaly, volume variance, and audio sentiment metrics now scope to the channel and time range you configure.

Metrics now do a better job of handling a wider variety of metadata inputs.