> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Create a Test Set

> Generate test cases with AI, build them manually, or upload a CSV/Excel file.

There are three ways to build a test set: build it by hand, use the Coval Test Set Generator, or upload existing test cases from a file.

## Build manually

Choose **Create manually** to start a blank test set and add test cases yourself.

## Use the Coval Test Set Generator

The Coval Test Set Generator drafts test cases from a description of what you want to test. Open **Test Sets** in the sidebar and click **New Test Set** to reach it.

**1. Enter your test scenario**

Type the scenario you want to test into the input box.

**2. (Optional) Add context**

Attach files (text, JSON, or markdown), choose an agent to evaluate, or pick a suggested category to sharpen the results.

**3. (Optional) Add attributes**

Define attributes to generate for each test case — e.g. a `ticket_number` of format `X-###` yields `X-001`, or a `destination` described as "a possible airport code" yields `SFO`.

**4. Submit and review**

Submit with the arrow button or Enter, then review and edit the generated cases in the test set editor.

> [!TIP]
>
> Be specific in your description, and attach agent prompts or documentation for more relevant tests. You can always edit, add, or remove cases after generation.

> [!TIP]
>
> Say how many cases you want — for example, "create 8 scenarios for password resets". The generator drafts five when you don't give a number.

## Upload a CSV or Excel file

Import test cases in bulk from a `.csv` or `.xlsx` file. The first row must be column headers (case-insensitive), and only `input` is required.

| Column | Required | Description |
| - | - | - |
| `input` | Yes | The test case input or prompt |
| `expected_behaviors` | No | Behaviors to grade against — a JSON array, a comma-separated string, or a single string |
| `type` | No | `SCENARIO` or `TRANSCRIPT` |
| `metadata` | No | A JSON object of test-case attributes |
| `agent_ids` | No | Agent IDs for the whole test set — a JSON array or comma-separated string (first non-empty value wins) |
| `knowledge_base_entries` | No | Knowledge base entries to attach — see formats below |
| *any other column* | No | Treated as an attribute |

**Knowledge base entries** accept any of these formats:

* JSON array of objects: `[{"id": "entry-1", "type": "web_url"}, {"id": "entry-2"}]` (`type` optional)
* JSON array of IDs: `["entry-1", "entry-2"]`
* Comma-separated `id:type` (or just `id`): `entry-1:web_url,entry-2,entry-3:pdf`

Accepted types: `web_url` (default), `plain_text`, `json`, `zendesk`, `shelf`, `file`.

> [!WARNING]
>
> Files must be under 10 MB and contain at least one row with a valid `input` value. Empty rows and rows with no `input` are skipped on import.

## Create programmatically

You can also create and manage test sets outside the app:

> [card] [API](https://docs.coval.ai/api-reference/test-sets/create-test-set.md)
>
> Create, list, update, and delete test sets via the REST API.

> [card] [CLI](https://docs.coval.ai/cli/test-sets.md)
>
> Manage test sets from your terminal with the `coval` CLI.

> [card] [MCP](https://docs.coval.ai/mcp/tools.md)
>
> Create and list test sets from an MCP client like Claude or Cursor.

> [card] [API SDKs](https://docs.coval.ai/sdks/api-sdks.md)
>
> Manage test sets with the typed TypeScript and Python clients.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.