# Add a proposed case to a dataset

Source: <https://everruns.com/docs/api/operations/accept_eval_proposal/>

[Docs](https://everruns.com/docs/)/[API reference](https://everruns.com/docs/api/)/[Evaluation & reporting](https://everruns.com/docs/api/groups/evaluation-reporting/)/Add a proposed case to a dataset

# Add a proposed case to a dataset

Preview

Add an Eval Builder proposal to a dataset as a new case. The dataset must target the proposal’s agent.

Public APIRequires the evals feature flag

POST`/v1/eval-proposals/{proposal_id}/accept`

See [authentication](https://everruns.com/docs/sre/runbooks/authentication/) for credentials and permissions.

## Parameters

### Path Parameters

**proposal_id**

required

string

Prefixed public identifier

## Request Bodyrequired

Media typeapplication/json

Request to add a proposal to a dataset.

object

**eval_id**

required

Dataset (eval) to add the case to.

string

### Example

```plaintext
eval_01933b5a000070008000000000000001
```

## Responses

201Created

### 201

Created

Media typeapplication/json

A single test case within an eval.

object

**artifacts**

Session files to collect after scoring completes.

Array\<object> | null

Named session file to collect after an eval case completes.

object

**name**

required

Export key for this artifact (for example `patch` or `log`).

string

**path**

required

Absolute path in the session filesystem.

string

**conversation**

required

Input messages sent sequentially.

Array\<object>

A message to send to the agent during an eval case.

object

**content**

required

The text content to send.

string

**created_at**

required

When the case was created.

string format: date-time

**description**

Optional description of what the case checks.

string | null

**id**

required

External identifier (evalcase\_<32-hex>).

string

**max_turns**

Max agent turns (default: 10).

integer | null format: int32

**name**

required

Case name.

string

**position**

required

Display order.

integer format: int32

**post**

Verification messages sent after conversation completes and session idles. Scorers run after post messages complete (not after conversation).

Array\<object> | null

A message to send to the agent during an eval case.

object

**content**

required

The text content to send.

string

**scorers**

required

Scoring rules.

Array

One of:

* object

  Final assistant message contains substring.

  object

  **text**

  required

  string

  **type**

  required

  string

  Allowed values: contains

  **weight**

  number format: double

* object

  Final assistant message does NOT contain substring.

  object

  **text**

  required

  string

  **type**

  required

  string

  Allowed values: not_contains

  **weight**

  number format: double

* object

  Final assistant message matches regex pattern.

  object

  **pattern**

  required

  string

  **type**

  required

  string

  Allowed values: regex

  **weight**

  number format: double

* object

  Agent called named tool at least `min` times. With `arguments_contain`, only calls whose argument JSON contains that text count, so a case can check what the agent asked a tool to do.

  object

  **arguments_contain**

  string | null

  **min**

  integer format: int32

  **tool**

  required

  string

  **type**

  required

  string

  Allowed values: tool_called

  **weight**

  number format: double

* object

  Agent did NOT call named tool.

  object

  **tool**

  required

  string

  **type**

  required

  string

  Allowed values: tool_not_called

  **weight**

  number format: double

* object

  Total tool calls within range.

  object

  **max**

  integer | null format: int32

  **min**

  integer | null format: int32

  **type**

  required

  string

  Allowed values: tool_call_count

  **weight**

  number format: double

* object

  Completed within N turns.

  object

  **max**

  required

  integer format: int32

  **type**

  required

  string

  Allowed values: turns_within

  **weight**

  number format: double

* object

  The agent spent at most `max` tokens (input plus output).

  object

  **max**

  required

  integer format: int64

  **type**

  required

  string

  Allowed values: tokens_within

  **weight**

  number format: double

* object

  Session filesystem file contains substring.

  object

  **path**

  required

  string

  **text**

  required

  string

  **type**

  required

  string

  Allowed values: file_contains

  **weight**

  number format: double

* object

  Final assistant message parses as JSON matching schema.

  object

  **schema**

  required

  **type**

  required

  string

  Allowed values: json_schema

  **weight**

  number format: double

* object

  Citation faithfulness: the answer’s citations must cover the claim and be verified as supported. Scored from the `TextAnnotation`s on the final message — pair with the `citation_verification` capability so verdicts are present.

  object

  **min_citations**

  Minimum number of citations the answer must carry.

  integer format: int32

  **pass_threshold**

  Minimum fraction of citations verified `entailed` to pass.

  number format: double

  **type**

  required

  string

  Allowed values: citation_faithful

  **weight**

  Relative weight of this scorer in the case’s weighted average.

  number format: double

* object

  Citation faithfulness judged by an LLM: each cited claim/source pair is graded by a model, so the eval works even without the `citation_verification` capability.

  object

  **model_id**

  One of:

  * string

    Judge model; the org’s default is used when absent.

    string

    /^model\_\[0-9a-f]{32}$/

  * null

    null

  **pass_threshold**

  Minimum judged score `[0,1]` to pass.

  number format: double

  **rubric**

  Rubric override; a citation-faithfulness rubric is used when absent.

  string | null

  **type**

  required

  string

  Allowed values: citation_judged

  **weight**

  Relative weight of this scorer in the case’s weighted average.

  number format: double

* object

  Judge model; the org’s default is used when absent.

  string

  /^model\_\[0-9a-f]{32}$/

* object

  null

* string

  The org’s chat model grades the answer against a rubric and returns a value from 0 to 1 with its reasoning.

  object

  **model_id**

  One of:

  * string

    Judge model; the org’s default is used when absent.

    string

    /^model\_\[0-9a-f]{32}$/

  * null

    null

  **pass_threshold**

  Minimum judged value to pass.

  number format: double

  **rubric**

  required

  What a good answer does and what a bad one does.

  string

  **type**

  required

  string

  Allowed values: judge

  **weight**

  number format: double

* null

  Judge model; the org’s default is used when absent.

  string

  /^model\_\[0-9a-f]{32}$/

* string

  null

* null

  A decision model answers one yes/no question about the answer; the probability of yes is the value. Cheaper than `judge` and typed, so it suits any check with a yes/no shape.

  object

  **no**

  What a no means.

  string | null

  **pass_threshold**

  Minimum probability of yes to pass.

  number format: double

  **question**

  required

  The yes/no question, asked about the user message, the answer and the tools used.

  string

  **type**

  required

  string

  Allowed values: decision

  **weight**

  number format: double

  **yes**

  What a yes means, when the question alone is ambiguous.

  string | null

**simulated_user**

One of:

* object

  A model that plays the user after the scripted messages.

  object

  **done_when**

  When the user stops: the simulator ends the conversation once this holds. Defaults to “the goal is met or clearly cannot be”.

  string | null

  **goal**

  required

  What the user wants from the conversation.

  string

  **max_turns**

  Most messages the simulator sends (default 8, at most 30).

  integer | null format: int32

  \>= 1 <= 30

  **model_id**

  Model that plays the user. Omitted: the organization’s default model.

  string | null

  **persona**

  required

  Who the user is, what they know and how they write.

  string

  **virtual_user_id**

  Virtual user the simulator speaks as, so the agent sees that end user’s identity, connections and memory.

  string | null

* null

  null

**tags**

Free-form tags for filtering runs.

Array\<string>

**target**

One of:

* object

  One of:

  * object

    Session creation parameters (mirrors CreateSessionRequest).

    object

    **agent_id**

    Agent to work in this session.

    string | null

    **harness_id**

    Harness for the session. If omitted, org default harness is used.

    string | null

    **harness_name**

    Addressable harness name (alternative to harness_id).

    string | null

    **max_iterations**

    Max LLM iterations per turn.

    integer | null

    **model_id**

    LLM model override.

    string | null

    **system_prompt**

    System prompt override (prepended to agent prompt).

    string | null

    **type**

    required

    string

    Allowed values: session

  * object

    Reference to a deployed app.

    object

    **app_id**

    required

    string

    **type**

    required

    string

    Allowed values: app

  * object

    Label-only target for externally-executed runs (e.g. imported from Mira).

    Carries provider/model labels and opaque params instead of session setup: external runs are ingested already-complete, so everruns never builds a session from this. Mirrors a provider-agnostic `(provider, model)` pair.

    object

    **model**

    required

    string

    **params**

    **provider**

    required

    string

    **type**

    required

    string

    Allowed values: external

* null

  Session creation parameters (mirrors CreateSessionRequest).

  object

  **agent_id**

  Agent to work in this session.

  string | null

  **harness_id**

  Harness for the session. If omitted, org default harness is used.

  string | null

  **harness_name**

  Addressable harness name (alternative to harness_id).

  string | null

  **max_iterations**

  Max LLM iterations per turn.

  integer | null

  **model_id**

  LLM model override.

  string | null

  **system_prompt**

  System prompt override (prepended to agent prompt).

  string | null

  **type**

  required

  string

  Allowed values: session

* object

  Reference to a deployed app.

  object

  **app_id**

  required

  string

  **type**

  required

  string

  Allowed values: app

* object

  Label-only target for externally-executed runs (e.g. imported from Mira).

  Carries provider/model labels and opaque params instead of session setup: external runs are ingested already-complete, so everruns never builds a session from this. Mirrors a provider-agnostic `(provider, model)` pair.

  object

  **model**

  required

  string

  **params**

  **provider**

  required

  string

  **type**

  required

  string

  Allowed values: external

* object

  null

**timeout_seconds**

Per-case timeout in seconds (default: 120).

integer | null format: int32

**tool_fixtures**

Tools the case gives the agent, answered with canned responses.

Array\<object> | null

A tool the case gives the agent, answered with a canned response.

object

**description**

What the tool does, as the agent reads it.

string | null

**name**

required

Tool name the agent calls.

string

**parameters**

JSON Schema of the arguments. Omitted: an object with any fields.

**response**

required

What every call returns.

**updated_at**

required

When the case was last updated.

string format: date-time

### Example

```json
{
  "artifacts": [
    {
      "name": "patch",
      "path": "/workspace/fix.patch"
    }
  ],
  "conversation": [
    {
      "content": "Fix the failing test in src/lib.rs"
    }
  ],
  "created_at": "2026-01-15T10:30:00Z",
  "description": "Agent fixes a failing unit test",
  "id": "evalcase_01933b5a000070008000000000000001",
  "max_turns": 10,
  "name": "fix-failing-test",
  "position": 0,
  "post": [
    {
      "content": "Run the tests again and report the result"
    }
  ],
  "scorers": [
    {
      "text": "tests pass",
      "type": "contains"
    }
  ],
  "simulated_user": {
    "done_when": "The agent opened a refund ticket with the right order number.",
    "goal": "Get the duplicate charge refunded, or a clear date when it will be.",
    "max_turns": 8,
    "model_id": "model_01933b5a00007000800000000000022c",
    "persona": "Small shop owner charged twice for the Pro plan. Writes short messages. Gives the order number ORD-48213 only when asked.",
    "virtual_user_id": "identity_01933b5a000070008000000000000001"
  },
  "tags": [
    "regression",
    "nightly"
  ],
  "target": {
    "type": "session"
  },
  "timeout_seconds": 120,
  "tool_fixtures": [
    {
      "description": "Open a support ticket",
      "name": "create_ticket"
    }
  ],
  "updated_at": "2026-01-15T10:30:00Z"
}
```

409The proposal was already decided

### 409

The proposal was already decided

FetchcURL

Request

```js
const url = 'https://app.everruns.com/api/v1/eval-proposals/example/accept';
const options = {
  method: 'POST',
  headers: {'Content-Type': 'application/json'},
  body: '{"eval_id":"eval_01933b5a000070008000000000000001"}'
};


try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

Request

```sh
curl --request POST \
  --url https://app.everruns.com/api/v1/eval-proposals/example/accept \
  --header 'Content-Type: application/json' \
  --data '{ "eval_id": "eval_01933b5a000070008000000000000001" }'
```

Additional language examples require JavaScript. The default request example is available above.

201409

201 · Created

```json
{
  "artifacts": [
    {
      "name": "patch",
      "path": "/workspace/fix.patch"
    }
  ],
  "conversation": [
    {
      "content": "Fix the failing test in src/lib.rs"
    }
  ],
  "created_at": "2026-01-15T10:30:00Z",
  "description": "Agent fixes a failing unit test",
  "id": "evalcase_01933b5a000070008000000000000001",
  "max_turns": 10,
  "name": "fix-failing-test",
  "position": 0,
  "post": [
    {
      "content": "Run the tests again and report the result"
    }
  ],
  "scorers": [
    {
      "text": "tests pass",
      "type": "contains"
    }
  ],
  "tags": [
    "regression",
    "nightly"
  ],
  "timeout_seconds": 120,
  "updated_at": "2026-01-15T10:30:00Z"
}
```

409 · The proposal was already decided

No response body example is declared for this status.

## Matching endpoints