Working through your assigned conversations
Open Human Review in the sidebar, choose your project, and open Assignments. Select a conversation from the queue to see its transcript, an audio player when a recording is available, and the metrics or triage labels to review. Open test-case details or conversation metadata when you need more context. To focus on a batch, sort by Date added or Conversation date, show completed conversations, or filter by how conversations entered the project: manually, through the API, or through an auto-add rule. Rule details help explain why a conversation was selected.Sharing work with other reviewers
In an Individual project, you provide your own answers independently. In a Collaborative project, the team shares the work. When conversation claiming is enabled, the banner confirms ownership before you can edit, and you hold one claim at a time. If you browse elsewhere, use Return to claim to resume your active review or End labeling session to release it. Choose another conversation if someone else owns the one you opened. Wait for ownership to be confirmed after reconnecting. If the workspace says a project manager is required, ask the person managing the project to assign one.Assessing the conversation evidence
Read the metric question and any review instructions before deciding on an answer. When project instructions are available, reopen them with Instructions while reviewing. Use the transcript to understand the full exchange, including what led up to an apparent failure and whether the agent later corrected it. Test-case details and conversation metadata can provide the expected outcome or context needed to judge the interaction. Listen to the recording when a judgment depends on timing, tone, interruptions, or what was actually audible. For example, a transcript alone may not tell you whether the agent interrupted the caller or paused too long before responding. If an audio review requires a recording that is unavailable, raise the missing evidence with your project manager rather than guessing.Making an independent judgment
If Blind Review is enabled, machine scores are hidden for the metrics your manager selected for blind review. Some scores may remain visible as context. Base your answer on the conversation and the evaluation criteria, whether or not an automated score is visible. If the criteria are ambiguous, record what makes the case difficult and ask your project manager to clarify the expected interpretation. Apply the same interpretation to similar cases; use N/A or UNKNOWN only where the metric supports that answer and the review criteria call for it.Label each metric
Work through the project metrics in their configured order and provide a human answer for each. Judge each criterion separately: a conversation can meet one requirement and fail another. How you record the answer depends on the metric type.Direct value metrics
You give a single value for the whole conversation.- Binary (Pass/Fail) — select Yes, No, or N/A. Applies to binary LLM judge, audio binary judge, and agent-repeats-itself.
- Numerical — enter a number within the configured min/max range. Applies to numerical and audio numerical judges.
- Categorical — choose from a configured dropdown. Applies to categorical and audio categorical judges.
- Transcript sentiment — pick a sentiment label (e.g. Rude, Polite, Encouraging, Professional).
- Composite evaluation — assess each criterion with MET / NOT_MET / UNKNOWN.
Audio region metrics
For audio region metrics, label the part of the recording where the event occurs. Mark or adjust the relevant regions on the audio waveform, checking their boundaries against playback. You can add regions and edit or delete existing ones; if the review is pre-filled, check those regions before accepting them. These reviews require an audio recording and include metrics such as interruptions, latency, pauses, and vocal characteristics. Save waveform edits with Save Review before completing the conversation. Revert All Changes discards unsaved waveform edits.Per-segment labeling
Assign a label to each speaking segment — for audio sentiment, mark each segment as Neutral, Angry, Happy, or Sad.Per-message review
Provide a value for each message — for words per message, the word count of each assistant message.Only metrics that support human labeling appear in the review interface. If an expected question is missing, check the project’s selected metrics with its manager.
Applying triage labels
If your project uses triage labels, choose categories that describe the conversation’s issue, theme, or required follow-up. These categories are separate from metric answers. A triage-only project shows the label panel without metric questions. Choose one category in a single-selection project or all applicable categories in a multiple-selection project. Enter custom categories only when the project allows them, and reuse an existing category when it describes the same issue. For example, a resolution metric might receive No while the conversation receives an “incorrect policy” triage label. The metric answer says whether the requirement was met; the category helps explain the kind of issue to investigate. Triage changes save automatically. Wait for saving to finish, and use the retry option if a save fails, before moving on.Explaining your decisions
A useful explanation connects your answer to evidence. Identify the requirement, point to the relevant exchange or audio moment, and explain why it meets or fails the criterion. For example: “The caller asked to cancel, but the agent only explained the cancellation policy and never confirmed cancellation.” Use a metric note to explain the answer itself and transcript comments to discuss a specific passage with collaborators. If the project requires disagreement notes, attach your explanation to the disagreeing metric answer; a general transcript comment does not replace that required note. An answer missing its required note does not count as complete. Disagreeing with the automated score is an expected part of review. Keep the answer supported by the evidence and explain the difference rather than changing your answer simply to match the metric.Completing and revisiting reviews
Before submitting, check that the required metric answers are filled in, audio edits and triage labels have saved, and any required disagreement notes explain your decisions. Then click Submit review. Saving an answer alone does not submit the entire conversation. If the completion check finds remaining requirements, Coval lists them. Resolve those items and submit again. After successful completion, the workspace advances to the next assignment when one is available. Use the previous/next controls or assignment number to move through the queue, and show completed conversations when you need to revisit one.Revisiting a decision
Show completed conversations to find a previous review and reread its evidence and notes. If you need to correct an answer, make the change where editing is available and follow any completion requirements shown by the workspace. In a shared project, ownership can limit which answers you can change; involve the project manager if the review is read-only.Understanding how your reviews are used
Your answers and explanations feed the project’s Insights and annotations. Human-to-human agreement helps the team identify unclear criteria, while human-to-machine agreement helps identify metric errors. Triage labels reveal recurring conversation themes. A review may be flagged as a suspect label because it differs from another reviewer or the automated result. That is a request for a second look, not a conclusion that your answer is wrong. Revisit the evidence and explain or correct the judgment as needed. Keep the note with the answer so another reviewer can understand the decision.Moving efficiently through the queue
Navigate rows withj / k, open an assignment with Enter, move between neighboring conversations with h / l (or a / d, or the arrow keys), and back out of a surface with b or Escape. See the Keyboard Navigation guide for the full model.