# List the Eval Builder's proposals for an agent

Source: <https://everruns.com/docs/api/operations/list_eval_proposals/>

[Docs](https://everruns.com/docs/)/[API reference](https://everruns.com/docs/api/)/[Evaluation & reporting](https://everruns.com/docs/api/groups/evaluation-reporting/)/List the Eval Builder's proposals for an agent

# List the Eval Builder's proposals for an agent

Preview

List an agent’s open eval case proposals, newest first, with the Eval Builder’s latest run.

Public APIRequires the evals feature flag

GET`/v1/agents/{agent_id}/eval-proposals`

See [authentication](https://everruns.com/docs/sre/runbooks/authentication/) for credentials and permissions.

## Parameters

### Path Parameters

**agent_id**

required

string

Prefixed public identifier

## Responses

200Success

### 200

Success

Media typeapplication/json

An agent’s open proposals and its latest Eval Builder run.

object

**data**

required

Open proposals, newest first.

Array\<object>

A case the Eval Builder suggests adding to one of the agent’s datasets.

object

**agent_id**

required

Agent the case is for.

string

**case**

required

The case as it would be created.

object

**artifacts**

Session files to capture after scoring completes.

Array\<object> | null

Named session file to collect after an eval case completes.

object

**name**

required

Export key for this artifact (for example `patch` or `log`).

string

**path**

required

Absolute path in the session filesystem.

string

**conversation**

Input messages sent to the agent sequentially. May be empty when a simulated user opens the conversation.

Array\<object>

A message to send to the agent during an eval case.

object

**content**

required

The text content to send.

string

**description**

Human-readable description. Safe to render in user-facing messages.

string | null

**max_turns**

Maximum agent turns before the case stops.

integer | null format: int32

**name**

required

Human-readable name. Safe to render in user-facing messages.

string

**position**

Display order within the eval.

integer | null format: int32

**post**

Verification messages sent after conversation completes and session idles.

Array\<object> | null

A message to send to the agent during an eval case.

object

**content**

required

The text content to send.

string

**scorers**

required

Scoring rules applied to the case output.

Array

One of:

* object

  Final assistant message contains substring.

  object

  **text**

  required

  string

  **type**

  required

  string

  Allowed values: contains

  **weight**

  number format: double

* object

  Final assistant message does NOT contain substring.

  object

  **text**

  required

  string

  **type**

  required

  string

  Allowed values: not_contains

  **weight**

  number format: double

* object

  Final assistant message matches regex pattern.

  object

  **pattern**

  required

  string

  **type**

  required

  string

  Allowed values: regex

  **weight**

  number format: double

* object

  Agent called named tool at least `min` times. With `arguments_contain`, only calls whose argument JSON contains that text count, so a case can check what the agent asked a tool to do.

  object

  **arguments_contain**

  string | null

  **min**

  integer format: int32

  **tool**

  required

  string

  **type**

  required

  string

  Allowed values: tool_called

  **weight**

  number format: double

* object

  Agent did NOT call named tool.

  object

  **tool**

  required

  string

  **type**

  required

  string

  Allowed values: tool_not_called

  **weight**

  number format: double

* object

  Total tool calls within range.

  object

  **max**

  integer | null format: int32

  **min**

  integer | null format: int32

  **type**

  required

  string

  Allowed values: tool_call_count

  **weight**

  number format: double

* object

  Completed within N turns.

  object

  **max**

  required

  integer format: int32

  **type**

  required

  string

  Allowed values: turns_within

  **weight**

  number format: double

* object

  The agent spent at most `max` tokens (input plus output).

  object

  **max**

  required

  integer format: int64

  **type**

  required

  string

  Allowed values: tokens_within

  **weight**

  number format: double

* object

  Session filesystem file contains substring.

  object

  **path**

  required

  string

  **text**

  required

  string

  **type**

  required

  string

  Allowed values: file_contains

  **weight**

  number format: double

* object

  Final assistant message parses as JSON matching schema.

  object

  **schema**

  required

  **type**

  required

  string

  Allowed values: json_schema

  **weight**

  number format: double

* object

  Citation faithfulness: the answer’s citations must cover the claim and be verified as supported. Scored from the `TextAnnotation`s on the final message — pair with the `citation_verification` capability so verdicts are present.

  object

  **min_citations**

  Minimum number of citations the answer must carry.

  integer format: int32

  **pass_threshold**

  Minimum fraction of citations verified `entailed` to pass.

  number format: double

  **type**

  required

  string

  Allowed values: citation_faithful

  **weight**

  Relative weight of this scorer in the case’s weighted average.

  number format: double

* object

  Citation faithfulness judged by an LLM: each cited claim/source pair is graded by a model, so the eval works even without the `citation_verification` capability.

  object

  **model_id**

  One of:

  * string

    Judge model; the org’s default is used when absent.

    string

    /^model\_\[0-9a-f]{32}$/

  * null

    null

  **pass_threshold**

  Minimum judged score `[0,1]` to pass.

  number format: double

  **rubric**

  Rubric override; a citation-faithfulness rubric is used when absent.

  string | null

  **type**

  required

  string

  Allowed values: citation_judged

  **weight**

  Relative weight of this scorer in the case’s weighted average.

  number format: double

* object

  Judge model; the org’s default is used when absent.

  string

  /^model\_\[0-9a-f]{32}$/

* object

  null

* string

  The org’s chat model grades the answer against a rubric and returns a value from 0 to 1 with its reasoning.

  object

  **model_id**

  One of:

  * string

    Judge model; the org’s default is used when absent.

    string

    /^model\_\[0-9a-f]{32}$/

  * null

    null

  **pass_threshold**

  Minimum judged value to pass.

  number format: double

  **rubric**

  required

  What a good answer does and what a bad one does.

  string

  **type**

  required

  string

  Allowed values: judge

  **weight**

  number format: double

* null

  Judge model; the org’s default is used when absent.

  string

  /^model\_\[0-9a-f]{32}$/

* string

  null

* null

  A decision model answers one yes/no question about the answer; the probability of yes is the value. Cheaper than `judge` and typed, so it suits any check with a yes/no shape.

  object

  **no**

  What a no means.

  string | null

  **pass_threshold**

  Minimum probability of yes to pass.

  number format: double

  **question**

  required

  The yes/no question, asked about the user message, the answer and the tools used.

  string

  **type**

  required

  string

  Allowed values: decision

  **weight**

  number format: double

  **yes**

  What a yes means, when the question alone is ambiguous.

  string | null

**simulated_user**

One of:

* object

  A model that plays the user after the scripted messages, until it is done or reaches its turn limit. Not combined with `post`.

  object

  **done_when**

  When the user stops: the simulator ends the conversation once this holds. Defaults to “the goal is met or clearly cannot be”.

  string | null

  **goal**

  required

  What the user wants from the conversation.

  string

  **max_turns**

  Most messages the simulator sends (default 8, at most 30).

  integer | null format: int32

  \>= 1 <= 30

  **model_id**

  Model that plays the user. Omitted: the organization’s default model.

  string | null

  **persona**

  required

  Who the user is, what they know and how they write.

  string

  **virtual_user_id**

  Virtual user the simulator speaks as, so the agent sees that end user’s identity, connections and memory.

  string | null

* null

  null

**tags**

Free-form tags attached to this resource.

Array\<string> | null

**target**

One of:

* object

  One of:

  * object

    Session creation parameters (mirrors CreateSessionRequest).

    object

    **agent_id**

    Agent to work in this session.

    string | null

    **harness_id**

    Harness for the session. If omitted, org default harness is used.

    string | null

    **harness_name**

    Addressable harness name (alternative to harness_id).

    string | null

    **max_iterations**

    Max LLM iterations per turn.

    integer | null

    **model_id**

    LLM model override.

    string | null

    **system_prompt**

    System prompt override (prepended to agent prompt).

    string | null

    **type**

    required

    string

    Allowed values: session

  * object

    Reference to a deployed app.

    object

    **app_id**

    required

    string

    **type**

    required

    string

    Allowed values: app

  * object

    Label-only target for externally-executed runs (e.g. imported from Mira).

    Carries provider/model labels and opaque params instead of session setup: external runs are ingested already-complete, so everruns never builds a session from this. Mirrors a provider-agnostic `(provider, model)` pair.

    object

    **model**

    required

    string

    **params**

    **provider**

    required

    string

    **type**

    required

    string

    Allowed values: external

* null

  Session creation parameters (mirrors CreateSessionRequest).

  object

  **agent_id**

  Agent to work in this session.

  string | null

  **harness_id**

  Harness for the session. If omitted, org default harness is used.

  string | null

  **harness_name**

  Addressable harness name (alternative to harness_id).

  string | null

  **max_iterations**

  Max LLM iterations per turn.

  integer | null

  **model_id**

  LLM model override.

  string | null

  **system_prompt**

  System prompt override (prepended to agent prompt).

  string | null

  **type**

  required

  string

  Allowed values: session

* object

  Reference to a deployed app.

  object

  **app_id**

  required

  string

  **type**

  required

  string

  Allowed values: app

* object

  Label-only target for externally-executed runs (e.g. imported from Mira).

  Carries provider/model labels and opaque params instead of session setup: external runs are ingested already-complete, so everruns never builds a session from this. Mirrors a provider-agnostic `(provider, model)` pair.

  object

  **model**

  required

  string

  **params**

  **provider**

  required

  string

  **type**

  required

  string

  Allowed values: external

* object

  null

**timeout_seconds**

Per-case timeout in seconds.

integer | null format: int32

**tool_fixtures**

Tools the case gives the agent, each answered with a canned response.

Array\<object> | null

A tool the case gives the agent, answered with a canned response.

object

**description**

What the tool does, as the agent reads it.

string | null

**name**

required

Tool name the agent calls.

string

**parameters**

JSON Schema of the arguments. Omitted: an object with any fields.

**response**

required

What every call returns.

**category**

required

What part of the job it covers.

string

Allowed values: happy_path edge_case failure adversarial

**created_at**

required

When it was proposed.

string format: date-time

**dismiss_note**

Note left when it was dismissed.

string | null

**id**

required

Proposal identifier.

string

**kind**

required

`simulated` when a model plays the user, otherwise `scripted`.

string

**name**

required

Short name of the case.

string

**reason**

required

Why the Eval Builder suggests it, in one or two sentences.

string

**source_session_ids**

required

Sessions that inspired it, by public identifier.

Array\<string>

**status**

required

Whether it is decided.

string

Allowed values: proposed accepted dismissed

**latest_run**

One of:

* object

  The most recent run, when there was one.

  object

  **completed_at**

  When it finished.

  string | null format: date-time

  **cost_usd**

  What the model call cost, when the provider reported it.

  number | null format: double

  **error**

  Why it failed, in words for a person.

  string | null

  **id**

  required

  Run identifier.

  string

  **model**

  Model that wrote the proposals.

  string | null

  **proposals**

  required

  Proposals it made.

  integer format: int32

  **sessions_read**

  required

  Production sessions it read.

  integer format: int32

  **started_at**

  required

  When it started.

  string format: date-time

  **status**

  required

  Where the run is.

  string

  Allowed values: running done failed unavailable

* null

  null

### Example

```json
{
  "data": [
    {
      "agent_id": "agent_01933b5a000070008000000000000001",
      "case": {
        "artifacts": [
          {
            "name": "patch",
            "path": "/workspace/fix.patch"
          }
        ],
        "conversation": [
          {
            "content": "Fix the failing test in src/lib.rs"
          }
        ],
        "description": "Agent fixes a failing unit test",
        "max_turns": 10,
        "name": "fix-failing-test",
        "position": 0,
        "post": [
          {
            "content": "Run the tests again and report the result"
          }
        ],
        "scorers": [
          {
            "text": "tests pass",
            "type": "contains"
          }
        ],
        "simulated_user": {
          "done_when": "The agent opened a refund ticket with the right order number.",
          "goal": "Get the duplicate charge refunded, or a clear date when it will be.",
          "max_turns": 8,
          "model_id": "model_01933b5a00007000800000000000022c",
          "persona": "Small shop owner charged twice for the Pro plan. Writes short messages. Gives the order number ORD-48213 only when asked.",
          "virtual_user_id": "identity_01933b5a000070008000000000000001"
        },
        "tags": [
          "regression",
          "nightly"
        ],
        "target": {
          "type": "session"
        },
        "timeout_seconds": 120,
        "tool_fixtures": [
          {
            "description": "Open a support ticket",
            "name": "create_ticket"
          }
        ]
      },
      "category": "happy_path",
      "id": "evalprop_01933b5a000070008000000000000001",
      "kind": "simulated",
      "name": "Refund over the approval limit",
      "source_session_ids": [
        "session_01933b5a000070008000000000000001"
      ],
      "status": "proposed"
    }
  ],
  "latest_run": {
    "cost_usd": 0.012,
    "id": "evalbuild_01933b5a000070008000000000000001",
    "model": "gpt-6.2-sol",
    "proposals": 5,
    "sessions_read": 6,
    "status": "running"
  }
}
```

FetchcURL

Request

```js
const url = 'https://app.everruns.com/api/v1/agents/example/eval-proposals';
const options = {method: 'GET'};


try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

Request

```sh
curl --request GET \
  --url https://app.everruns.com/api/v1/agents/example/eval-proposals
```

Additional language examples require JavaScript. The default request example is available above.

200

200 · Success

```json
{
  "data": [
    {
      "agent_id": "agent_01933b5a000070008000000000000001",
      "case": {},
      "category": "happy_path",
      "created_at": "2026-10-12T09:00:00Z",
      "id": "evalprop_01933b5a000070008000000000000001",
      "kind": "simulated",
      "name": "Refund over the approval limit",
      "reason": "string",
      "source_session_ids": [
        "session_01933b5a000070008000000000000001"
      ],
      "status": "proposed"
    }
  ]
}
```

## Matching endpoints