Choose a metric

The Metric Library is organized by how each metric works. This page flips that around: start from what you want to measure and jump straight to the metrics that measure it.

Just starting out? A solid baseline for most agents is Latency, End Reason, and one custom Binary LLM Judge for your specific success criteria. Add more as you learn where your agent struggles.

Did the agent do its job? (task resolution & correctness)

Metric	What it tells you
Binary LLM Judge	A yes/no answer to your exact success question
Agentic LLM Judge	An evidence-backed answer when success depends on what the agent did behind the scenes
Composite Evaluation	Several pass/fail criteria scored together
End Reason	How the conversation ended (resolved, transferred, dropped)
Match Expected Output	Whether the outcome matches a known-correct value

Was it fast and responsive? (latency & reliability)

Metric	What it tells you
Latency	How long the agent takes to respond
Time to First Audio	Delay before the agent starts speaking
Agent Fails to Respond	Turns where the agent went silent
Agent Needs Reprompting	Times the user had to repeat themselves
Agent Repeats Itself	Looping or repeated agent responses
LLM / STT / TTS Time to First Byte	Where latency comes from in your pipeline

Does it sound natural? (voice quality — voice agents)

Metric	What it tells you
Interruption Rate	How often the agent talks over the user
Phonemes per Second	Whether the agent speaks too fast or too slow
Pitch Variability	Monotone vs. natural intonation
Volume-Pitch Misalignment	Unnatural volume/pitch mismatches
Voice Quality	Acoustic naturalness of the voice; higher is better
Pause Analysis	Pause timing and frequency during speech

Does the audio have signal artifacts? (voice agents)

Metric	What it tells you
Clipping Artifact	Fraction of analyzed speech in runs of near-full-scale saturation; lower is better
Dropout Artifact	Brief, abrupt attenuation events per minute; lower is better
Codec Artifact	Mean spectral anomaly severity, from 0 to 1; lower is better
Speech Artifact Anomaly	Composite score across supported speech artifact checks; higher is better

These metrics flag different audio patterns; their scores do not identify a technical cause. A low clipping score can miss distortion after rescaling, and natural speech gaps can resemble dropouts. See the Statistical metrics page for each metric’s scope and interpretation.

How did the customer feel? (sentiment & experience)

Metric	What it tells you
Transcript Sentiment	Sentiment inferred from the text
Audio Sentiment	Sentiment inferred from the audio (tone, not just words)
Composite Evaluation	A blended experience score across criteria you define

Did it follow the rules? (compliance & scripting)

Metric	What it tells you
Transcript Regex Match	Whether required phrases appear — or use absent-match to enforce “never say X”
Binary / Categorical LLM Judge	Policy, tone, and disclosure checks

Did it do the right things behind the scenes? (tools & traces)

Metric	What it tells you
Tool Call Count	Whether the agent called the tools it should have
Custom Trace	Any value extracted from your OpenTelemetry spans
API State	State your backend reports for the call
LLM Judge with Trace context	Verify tool usage and order that isn’t visible in the transcript
Agentic LLM Judge	Investigate conversation and execution evidence when the answer depends on what the agent actually did

Was the audio transcribed accurately? (STT accuracy)

Metric	What it tells you
STT Word Error Rate	How accurately speech was transcribed
Transcription Error	Likely transcription mistakes in the conversation

What did it cost? (usage)

Metric	What it tells you
LLM Token Usage	Tokens consumed per conversation

Need a measurement specific to your agent?

Use a SQL metric when you can define an exact rule over recorded conversation data: count calls to a particular tool, measure speaking-segment durations, or count messages matching a text rule. SQL produces a numeric score without an LLM judge. For questions that need interpretation, such as whether a response was helpful, choose an LLM Judge.

Once you know which metrics you want, add them to a run. For anything custom, see Write judge prompts and Configure metrics.

Simulated Conversations

Evaluate

Analyze

Review

Cookbooks

Sofia

Reference

Did the agent do its job? (task resolution & correctness)

Was it fast and responsive? (latency & reliability)

Does it sound natural? (voice quality — voice agents)

Does the audio have signal artifacts? (voice agents)

How did the customer feel? (sentiment & experience)

Did it follow the rules? (compliance & scripting)

Did it do the right things behind the scenes? (tools & traces)

Was the audio transcribed accurately? (STT accuracy)

What did it cost? (usage)

Need a measurement specific to your agent?

​Did the agent do its job? (task resolution & correctness)

​Was it fast and responsive? (latency & reliability)

​Does it sound natural? (voice quality — voice agents)

​Does the audio have signal artifacts? (voice agents)

​How did the customer feel? (sentiment & experience)

​Did it follow the rules? (compliance & scripting)

​Did it do the right things behind the scenes? (tools & traces)

​Was the audio transcribed accurately? (STT accuracy)

​What did it cost? (usage)

​Need a measurement specific to your agent?

Did the agent do its job? (task resolution & correctness)

Was it fast and responsive? (latency & reliability)

Does it sound natural? (voice quality — voice agents)

Does the audio have signal artifacts? (voice agents)

How did the customer feel? (sentiment & experience)

Did it follow the rules? (compliance & scripting)

Did it do the right things behind the scenes? (tools & traces)

Was the audio transcribed accurately? (STT accuracy)

What did it cost? (usage)

Need a measurement specific to your agent?