Skip to main content
Beta. Organization admins can connect Slack and enable Conversations in the channel configuration. No separate Coval invitation is needed. Daily Brief is optional. Start with one dedicated channel; features and setup steps may change.
Sofia can post a daily summary of scheduled-investigation findings to a Slack channel that your organization chooses. The brief contains evaluation summaries and links back to Coval. It does not include transcript excerpts. Choose Daily brief, Conversations, or both when saving a channel binding. Connecting Slack alone does not turn on either feature.

Before you start

You need:
  • An organization admin account in Coval.
  • Permission to install or approve apps in your Slack workspace. Your Slack administrator may need to approve the Coval app.
  • A Coval workspace with scheduled-investigation coverage for Daily Brief, or data you can access for conversations.
  • A dedicated Slack channel for the beta. Start with a channel that does not include guests and is not shared through Slack Connect.
If Sofia channel bindings is missing, or you see Sofia in Slack is not available yet, confirm that you are using an organization admin account in the intended organization, then contact Coval for help.

Connect Slack

1

Open the Slack integration

In Coval, open Settings > Integrations > Slack.
2

Connect your workspace

Select Connect Slack. In Slack, choose the intended workspace, review the requested permissions, and approve the installation.Slack lists the app as Coval. This is the app that delivers Sofia’s daily briefs and answers questions when conversations are enabled.Connect Slack button in Coval settings
3

Return to Coval

After Slack redirects you back, confirm that Coval shows the correct connected workspace. If Coval asks you to reconnect, follow the on-screen reconnect flow before adding a channel.

Bind a channel for the daily brief

Under Sofia channel bindings, select Add channel, then configure the binding:
  1. Choose the Slack channel.
  2. Choose the Coval workspace whose investigation findings should appear.
  3. Leave Conversations off for brief-only delivery, or enable it to allow questions too.
  4. Enable Daily brief.
  5. Choose the delivery hour and enter an IANA timezone, such as America/Los_Angeles.
  6. Read and accept the channel-audience acknowledgement.
  7. Select Save binding.
Choose the channel before adjusting its settings. If you change the selected channel, review the settings again before saving; Coval loads that channel’s saved settings or defaults. Sofia Slack channel binding configured for a daily brief at 09:00 America/Los_Angeles Everyone who can read the channel, including future members, can read Sofia’s summaries and follow their links. Choose a channel whose audience is appropriate for the Coval workspace’s evaluation results.

Private channels

Slack does not reveal a private channel to an app until the app has joined it. In the private channel, run:
Select the Coval app in Slack’s invite picker. Then return to Coval and select Refresh Slack channels. Sofia can join a public channel when you save, but it must already be a member of a private channel. When the binding is saved, Coval shows the channel, workspace, delivery time, and timezone together. Confirm that all four match your intended setup. Saved private-channel binding for the Default workspace with Daily brief at 09:00 America/Los_Angeles

Rebind after reconnecting Slack

After reconnecting the same Slack workspace, an existing channel binding can show Rebind after reconnecting Slack. It still displays its previous settings, but you must review and save it before delivery can resume. Reloading the page does not reactivate it. Existing Slack binding with a reconnect warning and Rebind and Remove controls
  1. Select Rebind on the affected channel.
  2. Confirm the Slack channel, Coval workspace, delivery hour, and timezone.
  3. Restore only the features enabled for your beta. For a Daily Brief-only beta, keep Daily brief enabled and Conversations off.
  4. Read and accept the channel-audience acknowledgement again, then select Save binding.
  5. Confirm that the reconnect warning is gone and the saved settings are correct.
Select Remove instead if you no longer want that channel to receive briefs. You do not need to disconnect the shared Slack integration to remove one binding.

Read the brief

The active finding count includes both open and acknowledged findings. Findings can come from simulations, uploaded conversations, or both. Each finding’s Sources label identifies the sources represented by its retained investigation records:
  • Simulations: simulated conversations contributed to the finding.
  • Uploaded conversations: uploaded conversations contributed to the finding.
  • Both (simulations and uploaded conversations): both sources contributed to the same finding.
  • Source unavailable: source records are no longer available; Sofia does not guess from the finding’s wording.
Coverage is reported separately as Simulations: X of Y analyzed and Uploaded conversations: X of Y analyzed. Both use the selected workspace and the window between the previous and current scheduled delivery times. The totals count individual simulation outputs and uploaded conversations created in that window, respectively. They are not counts of findings or simulation runs. A zero upload count does not mean Sofia skipped simulations. Simulations: coverage unavailable means Sofia cannot report that coverage yet; it does not mean zero simulations were analyzed. Coverage describes the current delivery window, while a finding may include evidence from earlier windows. Source labels describe retained records and may omit older evidence that has expired.

Test the first daily brief

Use this small test before adding more channels or people:
  1. Choose the next full delivery hour, leaving enough time to finish setup before that hour begins.
  2. Confirm the saved binding shows the intended channel, Coval workspace, delivery hour, and timezone.
  3. Wait for the first scheduled brief. It normally arrives shortly after the selected hour.
  4. Confirm that exactly one message appears in the bound channel.
  5. Check that the workspace name, date range, finding summaries, and Coval links are correct.
  6. Open at least one link and confirm it stays in the intended Coval organization and workspace.
  7. Confirm that the message contains no transcript excerpts or information that should not be visible to everyone in the channel.
  8. Keep the binding unchanged for the next hour and confirm that no duplicate brief appears.

Start a conversation

An organization admin enables conversations in the Slack channel configuration. Connecting Slack or receiving a daily brief does not enable conversations by itself. Each person who asks questions links their own Coval account.

Enable the channel

An organization admin opens the channel binding, chooses the Coval workspace, enables Conversations, reviews the channel-audience acknowledgement, and selects Save binding. Daily Brief is optional for conversations; keep its existing settings if you already use it. If Conversations is missing, contact Coval rather than removing and recreating your binding. For the initial conversation beta, use one dedicated public channel within your Slack workspace, without guests or Slack Connect sharing. Unmentioned thread follow-ups in private channels are not enabled in this beta. Private-channel Daily Brief setup remains supported. Answers are visible to everyone who can read the channel, including future members. Personal account linking checks the person asking; it does not make the answer private.
  1. In the bound channel, mention the Coval app and ask a question. Select the app from Slack’s mention picker; plain text is not a mention.
  2. If Sofia asks you to link your account, open its account-link URL. Sign in to Coval as yourself and confirm the intended Coval organization.
  3. Select Continue with Slack, then choose the same Slack account and workspace you used to ask the question.
  4. Confirm that the page says your account is linked. Return to Slack and ask the question again.
Each person links their own account and must have access to the bound Coval workspace. Do not share your account-link URL. If it expires or you selected the wrong account, ask Sofia again for a fresh link.

Ask and follow up

Start with a small, verifiable question:
@Coval What is the latest completed run in this workspace? Give its exact name and run ID.
After Sofia replies, use Reply in thread to continue. You do not need another mention in that conversation thread:
Which workspace is that run in, and how many simulations completed or failed?
Give me the link to that run so I can inspect the results.
Check the workspace, run ID, and counts against Coval, then open the link to confirm it reaches the same run. Completed and failed counts describe execution status; they are not an evaluation quality score. A new question outside the thread needs a new mention. Sofia can read authorized workspace data, explain results, and suggest next steps. Conversations are read-only by default. An organization admin can enable confirmed actions for a channel as described below. Keep sensitive questions in channels with an appropriate audience, and review the underlying evidence before acting on an answer.

Enable confirmed actions

In Settings > Integrations > Slack, edit the channel binding and enable Conversations, then Allow actions with confirmation. Review the channel and Coval workspace, accept the audience acknowledgement, and save. Leaving this setting off keeps conversations read-only. Only an organization admin can change it. Actions are unavailable in Slack Connect channels or when guest access is enabled. For a binding that allows guests, explicitly turn off Allow guest conversations before saving confirmed actions. Review this audience change with your channel members. With actions enabled, you can ask Sofia to:
  • Launch a simulation run, rerun a run, or rerun its failed test cases.
  • Rerun selected metrics against existing simulated conversations.
  • Create test sets, add test cases, and edit test-set details or individual cases.
  • Update an agent’s name, phone number, prompt or language, or attach metrics.
  • Create supported metrics or update their configuration.
For example:
@Coval Rerun the failed cases from the latest run. Show me the proposed run first.
Update the greeting test case to require the agent to introduce itself by name.
Rerun the resolution metric against these simulations: [simulation IDs].
Sofia prepares a card showing the exact change and workload. Preparing a card does not run anything. Select Approve to execute it or Cancel to leave it undone. A message such as “yes” or “go ahead” does not approve a card. Each card takes one decision. After someone selects Approve or Cancel, the card shows who decided and its Approve and Cancel buttons are removed. Sofia shows a thinking indicator on the card while an approved action runs. An approved card keeps Check status so you can check the result later. Other cards in the same reply stay available. The person approving must link their own Coval account and have current write access to the bound workspace. A different eligible person may approve your proposal; it runs as that approver. Organization members need membership in a custom workspace, while organization administrators can approve across their organization’s workspaces. Restricted reviewer roles cannot approve writes. Each card expires after 15 minutes. A new conversation turn, changed resource, account unlink, binding change or disabling actions can invalidate a pending card. Ask for a fresh proposal when the original is stale. Sofia can propose up to three separate actions in one reply. Each run approval is limited to 100 simulations; each metric-rerun approval is limited to 100 simulation–metric pairs and 20 distinct simulations and metrics combined. Use Coval for larger metric batches. Slack run approvals require a test set with at most 100 cases, including when rerunning selected cases. Use Coval for larger test sets. Existing Coval quotas still apply.

Check the outcome

Sofia posts a receipt identifying the requester and approver. Run queued and Metric rerun accepted mean Coval accepted work; they do not mean evaluation has finished. Select Check status to read the current state, or Open Coval to open the affected resource. For a created test set or metric, Open Coval opens the new resource. Configuration receipts distinguish acceptance from any verified saved fields. If Sofia reports an unknown outcome, the request may already have reached Coval. The same card will not execute again. Inspect the resource in Coval before asking for another action, to avoid duplicating a run or edit. Deletion, credentials, provider connection settings, permissions, billing and Python metric publication are outside Slack actions. Use Coval for those changes. Turning off Allow actions with confirmation stops pending approvals while preserving read-only conversations and the binding’s Daily Brief setting.

Stop conversations

An organization admin can turn Conversations off on the binding while leaving Daily brief enabled. Remove the binding to stop both features in that channel. Disconnecting the Slack integration affects other bindings using it. These controls do not erase messages already posted in Slack.

Troubleshooting

Beta boundaries

  • Start with one Coval workspace and one dedicated internal Slack channel.
  • Do not use a Slack Connect channel for the initial test.
  • Enable Conversations only in channels where members should see the answers.
  • Guest conversations and Slack Connect channels are outside the initial beta.
  • Treat Slack as a summary surface. Review linked evidence in Coval before making a high-impact decision.
  • Remove the binding or disconnect Slack from Settings > Integrations > Slack if you need to stop future Sofia messages.