List observers
PreviewInspect agent quality, health, observers, semantic reports and notifications.
/v1/observersSee authentication for credentials and permissions.
Parameters
Section titled “Parameters”Query Parameters
Section titled “Query Parameters”Include archived observers.
Responses
200Success
Success
Response wrapper for list endpoints.
All list endpoints return responses wrapped in a data field.
object
Array of items returned by the list operation.
An observer: online scoring config over production sessions.
object
Pass-rate alert, when set.
object
Pass rate (0.0–1.0) under which the alert opens.
Scores the window needs before the alert can open. Defaults to 20.
Window the rate is measured over, in hours (1–168). Defaults to 24.
Optional description.
External identifier (observer_<32-hex>). Shown as “id” in API.
Which sessions to score.
object
Match sessions running any of these agents.
Match sessions on any of these harnesses.
Match sessions carrying any of these tags.
Display name.
Fraction of matching turns to score (0.0–1.0), applied after match.
Scoring rules.
One scorer inside an observer. key names the score series in listings
and future dashboards; scope selects the trace slice; method is how
it grades.
object
Deterministic rule. file_contains is rejected for observers (session
filesystems are not part of the observable trace contract).
object
Final assistant message contains substring.
object
Final assistant message does NOT contain substring.
object
Final assistant message matches regex pattern.
object
Agent called named tool at least min times. With
arguments_contain, only calls whose argument JSON contains that text
count, so a case can check what the agent asked a tool to do.
object
Agent did NOT call named tool.
object
Total tool calls within range.
object
Completed within N turns.
object
The agent spent at most max tokens (input plus output).
object
Session filesystem file contains substring.
object
Final assistant message parses as JSON matching schema.
object
Citation faithfulness: the answer’s citations
must cover the claim and be verified as supported. Scored from the
TextAnnotations on the final message — pair with the
citation_verification capability so verdicts are present.
object
Minimum number of citations the answer must carry.
Minimum fraction of citations verified entailed to pass.
Relative weight of this scorer in the case’s weighted average.
Citation faithfulness judged by an LLM: each cited claim/source pair is
graded by a model, so the eval works even without the
citation_verification capability.
object
Minimum judged score [0,1] to pass.
Rubric override; a citation-faithfulness rubric is used when absent.
Relative weight of this scorer in the case’s weighted average.
The org’s chat model grades the answer against a rubric and returns a value from 0 to 1 with its reasoning.
A decision model answers one yes/no question about the answer; the
probability of yes is the value. Cheaper than judge and typed, so it
suits any check with a yes/no shape.
object
What a no means.
Minimum probability of yes to pass.
The yes/no question, asked about the user message, the answer and the tools used.
What a yes means, when the question alone is ambiguous.
LLM-as-judge.
object
Org model to judge with. When None, the org’s default model is used.
Judge calls go through the org’s own providers and are billed to it.
Score value at/above which pass is true.
Grading rubric shown to the judge model. Should describe what a high vs. low score means.
Stable name within the observer (score series name).
Trace slice this scorer grades.
Tool scope only: grade calls to these tools. Empty or absent grades every tool call.
Lifecycle status.
Example
{ "data": [ { "alert": { "min_pass_rate": 0.8, "min_scored": 20, "window_hours": 24 }, "description": "Grades support replies for grounded answers.", "id": "observer_01933b5a000070008000000000000001", "name": "Support answer quality", "sampling_rate": 0.1, "scorers": [ { "method": "llm_judge", "scope": "turn" } ], "status": "active" } ]}