> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coval.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Human Review Projects
> Organize human labeling of conversations to validate your metrics and triage production calls.
Human review is how you get ground truth from people: reviewers label real conversations so you can trust your metrics and find exactly where they're wrong. A **review project** organizes that work — it bundles the metrics to validate, the reviewers, and the conversations to label, then tracks who has labeled what.
A project supports two kinds of labeling, and you can use either or both:
* **Metric review** — reviewers give the ground-truth answer for a metric, so Coval can measure how often the metric agrees with humans. This is what powers [improving your metrics](/guides/improving-metrics-with-human-review).
* **Triage** — reviewers apply your own labels to sort and categorize conversations (failure type, theme, severity, follow-up). Triage labels are yours to define and aren't tied to any metric.
## What's in a project
| Setting | What it does |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Metrics** | Which metrics reviewers validate. Optional — leave it empty for a triage-only project. |
| **Team members** | The reviewers assigned to label (at least one required). |
| **Collaborative mode** | When on, each conversation × metric is labeled once and locked so no two reviewers label the same row. When off (**Individual**, the default), every reviewer labels independently in their own queue. |
| **Require disagreement notes** | When on, a reviewer who disagrees with the metric's output must leave a note, or the row doesn't count as complete. |
| **Email notifications** | Notifies assignees when they're added or when new work arrives. On by default. |
| **Triage** | Optional. Lets reviewers tag conversations with your own categories (see below). |
Choose **Collaborative** when you want one authoritative label per conversation and want to divide the work; choose **Individual** when you want to measure how consistently reviewers agree.
### Triage categories
If you enable triage on a project, you configure:
* **Input mode** — whether reviewers pick from **predefined categories only**, enter **custom categories only**, or **either**.
* **Selection mode** — whether a conversation can take a **single** category or **multiple**.
* **Categories** — the predefined list reviewers choose from.
Reviewers apply these in the [review interface](/concepts/metrics/human-review/how-to-review#applying-triage-labels); afterward you can sort and group conversations by label.
## Create a project
On the **Projects** tab of the Human Review page, create a project and give it a name and (optionally) a description. Choose the metrics to validate, assign reviewers, and set the options above — collaborative mode, disagreement notes, notifications, and triage.
A new project starts empty. Add conversations to label from **Runs** (simulations) or **Monitoring** (live conversations) — or set up an auto-add rule so they flow in on their own.
### Keep the queue full with auto-add
Auto-add is what makes review continuous instead of a one-off.
Add an **auto-add rule** to a project and conversations are pulled in automatically whenever their metric results match the conditions you set — for example, every call where a resolution metric returns **No**. You can require all conditions to be met, scope the rule, and apply a **random sampling rate** so you label a representative slice rather than everything. Assignees are notified as new work lands, so ground truth keeps building without anyone adding rows by hand.
Any conversation can also be annotated ad hoc from its results page — no project required. Projects add assignment, triage labels, progress tracking, and agreement analytics on top.
## Track progress
The Projects tab shows how labeling is going, with three ways to slice it:
* **By conversation** — a `completed / total` count and progress bar for each conversation, so you can see what's finished, in progress, or untouched.
* **By metric** — completion per metric, plus an **agreement rate**: how often the metric matched human ground truth, color-coded so low-agreement metrics stand out. This is your cue to [improve that metric](/guides/improving-metrics-with-human-review).
* **By assignee** — how many annotations and conversations each reviewer has completed.
Totals account for the project mode — in Individual mode every assignee labels every row, so the target is conversations × reviewers; in Collaborative mode it's one label per row. Filter the view by **review status** (Review Complete, In Progress, Pending) or a **date range** to focus on a batch.
You can also track review progress from a [dashboard](/concepts/dashboard/overview): the **Human review** widget surfaces assignees and progress across projects alongside your other metrics.
## Next
For reviewers: open assignments, label each metric, apply triage labels, and move fast.
Turn the labels into agreement scores and tighten the metrics that disagree.