Tracing Skills

I want you to configure this agent to send traces to Coval using Coval's external tracing skills.

Use this skills repository:
https://github.com/coval-ai/coval-external-skills

If your environment supports Agent Skills, install or load the repo with:
npx skills add coval-ai/coval-external-skills

Then use the tracing skills in `skills/traces/`:
1. Start with `setup-tracing` to add Coval OpenTelemetry export and launch one real Coval validation run.
2. While that run is pending, use the waiting time to improve trace coverage and prepare trace metrics instead of sitting idle.
3. After spans appear, use `optimize-trace-observability` to confirm and refine span coverage and attributes.
4. Use `configure-trace-metrics` to recommend and create useful custom trace metrics from confirmed real span data.
5. Use `debug-traces` if traces are missing, sparse, duplicated, or attached to the wrong Coval result.

Do not ask me to fill out a setup template first. Infer as much as you can from the repository, project files, existing Coval configuration, deployment files, logs, README/runbooks, and authenticated Coval CLI/API access if available.

During read-only discovery, determine:
- Coval agent type: SIP inbound voice, inbound phone over PSTN, outbound voice, WebSocket voice/chat, or conversation monitoring
- Coval agent ID or name
- How the agent runs locally
- How the agent is deployed
- How to trigger one test in Coval
- Existing observability in this repo, such as OpenTelemetry, Langfuse, Arize, LangSmith, or none

Rules:
- Assume you do not have access to Coval internal backend, frontend, docs, wizard, research, or example source repositories. Do not ask me for Coval internal source code. Use this agent repository, public Coval docs, Coval CLI/API access, and fetched public OpenAPI specs only.
- Make code changes only in this customer-owned agent repository. Coval-side changes must be limited to documented configuration through the Coval CLI, public API, or dashboard, and you should explain them before mutation.
- Start with read-only analysis. Do not edit files until you summarize the current agent entry point, Coval connection path, correlation ID path, existing telemetry, and smallest additive implementation plan.
- Ask clarifying questions only for details you cannot discover and that block safe implementation or validation. Keep questions concise and grouped.
- Do not ask me to choose between SIP headers, pre-call webhooks, registration endpoints, trigger payloads, or WebSocket initialization when the repository and Coval agent configuration make one route clearly safer. Pick the route, explain why, and ask only for concrete missing permission such as exposing a webhook, updating Coval agent configuration, or provisioning SIP.
- Do not print, store, or hard-code secrets. If you need credentials, ask me to provide them as environment variables.
- If this is an inbound phone/PSTN agent, do not assume SIP headers are available. Use a pre-call or registration-webhook correlation path, or tell me if I need a SIP-capable route.
- Keep changes additive and preserve existing telemetry providers when present.
- Run relevant local checks and tell me exactly what to deploy.
- Validate with one real Coval simulation or monitoring conversation launched through the Coval CLI/API. This is asynchronous: start the run, capture the run/result/conversation IDs, poll through CLI/API, and while waiting continue with safe trace enrichment and metric preparation. Create custom trace metrics while waiting only if existing or in-flight Coval traces already prove the span/attribute exists; otherwise stage the metric definitions and create them once the validation trace appears.
- If traces are low-signal (for example most spans share one generic name such as `conversation`), treat setup as incomplete and use `optimize-trace-observability` to add role- and phase-specific spans and attributes before calling the workflow done.
- After creating or updating trace metrics, launch or reuse a recent run with traces and confirm each metric reaches a terminal computed state. Do not stop at `IN QUEUE` or `IN PROGRESS`.
- Report the files changed, commands run, correlation path used, Coval launch/polling commands or API endpoints, metrics created or staged, and the Coval simulation or conversation ID that proves tracing works.
- Include direct URLs in this exact format:
  - display of all runs: `https://app.coval.dev/<org>/runs?sort=createdAt%3Adescending`
  - display of specific run: `https://app.coval.dev/<org>/runs/<runId>`
  - display of simulation within run: `https://app.coval.dev/<org>/runs/<runId>/results/<resultId>`
  - display of traces for that simulation: `https://app.coval.dev/<org>/runs/<runId>/results/<resultId>/traces`

npx skills add coval-ai/coval-external-skills

git clone https://github.com/coval-ai/coval-external-skills.git

export COVAL_API_KEY="<your-coval-api-key>"

brew install coval-ai/tap/coval
coval login
coval whoami

Use the Coval `setup-tracing` skill from `coval-ai/coval-external-skills`.

Goal: configure this agent to send OpenTelemetry traces to Coval and prove one real trace reaches Coval.

Do not ask me to fill out a setup template first. Infer as much as you can from the repository, project files, existing Coval configuration, deployment files, logs, README/runbooks, and authenticated Coval CLI/API access if available.

Instructions:
1. Treat Coval internals as unavailable. Do not ask for or rely on Coval backend, frontend, docs, wizard, research, or example source repositories. Use this agent repository, public Coval docs, Coval CLI/API access, and fetched public OpenAPI specs only. Make code changes only in this customer-owned agent repository.
2. Start with read-only analysis. Do not edit files until you summarize:
   - the app entry point
   - the current Coval connection path
   - where the Coval simulation output ID or conversation ID can enter the process
   - whether existing telemetry should be reused
   - how the agent appears to run locally
   - how the agent appears to be deployed
   - how to trigger one Coval validation
   - the smallest additive implementation plan
3. Ask clarifying questions only for details you cannot discover and that block safe implementation or validation. Keep questions concise and grouped.
4. If this is an inbound phone/PSTN agent, do not assume SIP headers are available. Guide me through a pre-call or registration-webhook correlation path, or tell me if I need a SIP-capable route.
5. Do not ask me to choose between SIP headers, pre-call webhooks, registration endpoints, trigger payloads, or WebSocket initialization when the repository and Coval agent configuration make one route clearly safer. Pick the route, explain why, and ask only for concrete missing permission such as exposing a webhook, updating Coval agent configuration, or provisioning SIP.
6. Keep changes additive. Do not hard-code API keys or replace existing telemetry providers.
7. Add only the dependencies and helper files needed for tracing.
8. Run the relevant local checks for this repo.
9. Tell me exactly what to deploy, then launch one Coval validation through the Coval CLI/API. Capture the run ID, simulation output ID, conversation ID, dashboard URL, or polling endpoint returned by the launch.
10. Poll the validation through CLI/API with a bounded interval and timeout. While it is pending, continue with safe trace enrichment and metric preparation rather than waiting idle.
11. Create custom trace metrics during the wait only if historical traces or the in-flight validation already prove the span name and metric attribute exist. If this is the first trace for the agent, stage the metric definitions and create them immediately after the validation trace appears.
12. After validation, summarize the files changed, the correlation path used, commands run, Coval launch/polling commands or API endpoints, metrics created or staged, and the Coval simulation or conversation ID that proves the trace worked.
13. If spans are low-signal (for example mostly `conversation`), keep going with `optimize-trace-observability` and produce a second validation trace that demonstrates richer span coverage.
14. After creating or updating metrics, run or inspect a trace-backed simulation result and confirm terminal metric outputs. Report the metric output status and value, not only metric creation.
15. Output proof URLs for runs list, run, result, and result traces using this exact format:
   - `https://app.coval.dev/<org>/runs?sort=createdAt%3Adescending`
   - `https://app.coval.dev/<org>/runs/<runId>`
   - `https://app.coval.dev/<org>/runs/<runId>/results/<resultId>`
   - `https://app.coval.dev/<org>/runs/<runId>/results/<resultId>/traces`

Use the Coval `optimize-trace-observability` skill.

I have launched or already have a Coval validation trace for this agent. Improve the trace quality so it is useful for debugging and trace-based metrics.

Find a recent traced simulation or conversation through the repo, Coval CLI/API, dashboard context I provide, or Trace Search if available. If the current validation run is still pending, poll it through the Coval CLI/API and use the waiting time for safe code-visible enrichment. If you cannot find or launch a validation run, ask me for the smallest missing detail before editing.

Start by inspecting the current spans. Prefer enriching existing framework/provider spans over creating duplicates. Add or improve spans for conversation, turn, speech recognition, language model calls, speech synthesis, tool calls, transport, or pipeline steps where the code exposes them. Add safe attributes such as transcripts, time to first byte, finish reasons, token counts, tool names, bounded tool arguments, status, and errors. Do not record secrets, raw audio, or unbounded payloads.

Run local checks, then tell me what changed and what new debugging questions the trace can answer.

Use the Coval `configure-trace-metrics` skill.

I have traces for this agent, or an initial validation run is currently pending, and want a small set of high-signal metrics.

Infer the agent, recent traced results, and likely use case from the repository, Coval CLI/API, existing run history, dashboards, docs, or naming. If a validation run is pending, poll it through the Coval CLI/API while preparing metric definitions.

Inspect real span names and attributes first. Recommend 3-6 custom trace metrics using attributes that actually exist. Include the span name, metric attribute when needed, aggregation method, unit, and why each metric is useful. Create metrics while waiting only when historical traces or the in-flight validation already prove the span/attribute exists. If the initial validation is the first trace, stage the metric definitions and create them once the trace appears. Ensure at least two metrics are high-signal for agent success, health, or performance (for example interruption rate, tool failure rate, critical tool latency, or end-to-end conversation success proxy). If the business goal is ambiguous, ask me what outcomes I care about before creating metrics.

Use the Coval `debug-traces` skill.

Expected traces are missing, sparse, duplicated, or attached to the wrong Coval result.

Infer the agent type, likely Coval result, export path, and recent errors from the repository, Coval CLI/API, logs, dashboard context I provide, and existing tracing code. If you cannot identify the failing Coval run, simulation, conversation, or export response, ask me for the smallest missing detail needed to continue.

Separate the failure boundary first: agent export, Coval ingestion, target ID correlation, organization/API key, UI lookup, payload size, duplicate retries, or sparse span coverage. Ask me to run commands locally if needed, but do not ask for raw API keys.