> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Create a Test Set

> Generate test cases with AI, build them manually, or upload a CSV/Excel file.

There are three ways to build a test set: build it by hand, use the Coval Test Set Generator, or upload existing test cases from a file.

## Build manually

Choose **Use manual creation mode** to start a blank test set and add test cases yourself.

## Use the Coval Test Set Generator

The [Coval Test Set Generator](https://app.coval.dev/test-sets/create) drafts test cases from a description of what you want to test.

<Steps>
  <Step title="Enter your test scenario">
    Type the scenario you want to test into the input box.
  </Step>

  <Step title="(Optional) Add context">
    Attach files (text, JSON, or markdown), choose an agent to evaluate, or pick a suggested category to sharpen the results.
  </Step>

  <Step title="(Optional) Add attributes">
    Define attributes to generate for each test case — e.g. a `ticket_number` of format `X-###` yields `X-001`, or a `destination` described as "a possible airport code" yields `SFO`.
  </Step>

  <Step title="Submit and review">
    Submit with the arrow button or Enter, then review and edit the generated cases in the test set editor.
  </Step>
</Steps>

<Tip>
  Be specific in your description, and attach agent prompts or documentation for more relevant tests. You can always edit, add, or remove cases after generation.
</Tip>

## Upload a CSV or Excel file

Import test cases in bulk from a `.csv` or `.xlsx` file. The first row must be column headers (case-insensitive), and only `input` is required.

| Column                   | Required | Description                                                                                            |
| ------------------------ | -------- | ------------------------------------------------------------------------------------------------------ |
| `input`                  | Yes      | The test case input or prompt                                                                          |
| `expected_behaviors`     | No       | Behaviors to grade against — a JSON array, a comma-separated string, or a single string                |
| `type`                   | No       | `SCENARIO` or `TRANSCRIPT`                                                                             |
| `metadata`               | No       | A JSON object of test-case attributes                                                                  |
| `agent_ids`              | No       | Agent IDs for the whole test set — a JSON array or comma-separated string (first non-empty value wins) |
| `knowledge_base_entries` | No       | Knowledge base entries to attach — see formats below                                                   |
| *any other column*       | No       | Treated as an attribute                                                                                |

**Knowledge base entries** accept any of these formats:

* JSON array of objects: `[{"id": "entry-1", "type": "web_url"}, {"id": "entry-2"}]` (`type` optional)
* JSON array of IDs: `["entry-1", "entry-2"]`
* Comma-separated `id:type` (or just `id`): `entry-1:web_url,entry-2,entry-3:pdf`

Accepted types: `web_url` (default), `plain_text`, `json`, `zendesk`, `shelf`, `file`.

<Warning>
  Files must be under 10 MB and contain at least one row with a valid `input` value. Empty rows and rows with no `input` are skipped on import.
</Warning>

## Create programmatically

You can also create and manage test sets outside the app:

<CardGroup cols={2}>
  <Card title="API" icon="square-terminal" href="/api-reference/v1/test-sets/test-sets/create-test-set">
    Create, list, update, and delete test sets via the REST API.
  </Card>

  <Card title="CLI" icon="terminal" href="/cli/test-sets">
    Manage test sets from your terminal with the `coval` CLI.
  </Card>

  <Card title="MCP" icon="diagram-project" href="/mcp/tools">
    Create and list test sets from an MCP client like Claude or Cursor.
  </Card>

  <Card title="SDKs" icon="code" href="/sdks/overview">
    Manage test sets with the typed TypeScript and Python clients.
  </Card>
</CardGroup>