Framework: build and run agents inside a Rust application with the everruns crate
---
# Everruns Documentation
> Tutorials, how-to guides, reference, and explanation for building and operating Everruns agents.
Source:
# Documentation Everruns is a **durable agentic harness engine** built on Rust. It provides APIs for managing agents, sessions, and long-running tasks, with real-time SSE event streaming and PostgreSQL-backed durability. Popular[Framework quickstart](https://everruns.com/docs/framework/quickstart/)[Docker Compose](https://everruns.com/docs/getting-started/docker-compose/)[Tools](https://everruns.com/docs/how-to/equip-agents-with-tools/)[Sessions API](https://everruns.com/docs/api/operations/tags/sessions/) ## Choose how you run Everruns [Start here compares the three →](https://everruns.com/docs/getting-started/introduction/) [Framework](https://everruns.com/docs/framework/quickstart/) Build and run agents inside a Rust process with the everruns crate. Nothing else to deploy. * [Quickstart](https://everruns.com/docs/framework/quickstart/) * [Agents and tools](https://everruns.com/docs/framework/agents/) * [Sessions](https://everruns.com/docs/framework/sessions/) * [Models and providers](https://everruns.com/docs/framework/models-and-providers/) `$ cargo add everruns` [Self-hosted](https://everruns.com/docs/getting-started/docker-compose/) Run the Platform (API, workers, PostgreSQL, UI) on infrastructure you control, starting with Docker Compose. * [Docker Compose quickstart](https://everruns.com/docs/getting-started/docker-compose/) * [Configure a model provider](https://everruns.com/docs/getting-started/docker-compose/#configure-llm-provider) * [Environment variables](https://everruns.com/docs/sre/environment-variables/) * [Architecture](https://everruns.com/docs/explanation/architecture/) `$ docker compose up -d` [Everruns Demo](https://everruns.com/docs/getting-started/cloud/) Use the Platform at app.everruns.com without running it. Explore with available promotional credit for built-in models, or bring your own provider keys. * [Try the demo](https://everruns.com/docs/getting-started/cloud/) * [Personal access tokens](https://everruns.com/docs/getting-started/cloud/#3-create-a-personal-access-token) * [Credit and the spend cap](https://everruns.com/docs/getting-started/cloud/#credit-and-the-spend-cap) * [Bring your own keys](https://everruns.com/docs/getting-started/cloud/#bring-your-own-keys-optional) `https://app.everruns.com` ## What kind of help do you need? [LearnTutorialsStep-by-step lessons that end with a running agent. Start here if you're new.](https://everruns.com/docs/tutorials/run-an-agent/)[DoHow-to guidesTask-oriented recipes for common problems, when you know what you want to build.](https://everruns.com/docs/how-to/)[UnderstandExplanationBackground and design rationale: durable execution, sessions, and harnesses.](https://everruns.com/docs/explanation/)[Look upReferenceAPI endpoints, event types, the capability catalog, CLI flags, and environment variables.](https://everruns.com/docs/api/)
---
# Everruns Framework
> Build and run agents inside a Rust application with the application-facing everruns crate.
Source:
The **Everruns Framework** is the application-facing [`everruns`](https://docs.rs/everruns) crate. Use it to describe agents, attach models and tools, run multi-turn sessions, observe events, and embed agent execution directly in a Rust process.
```rust
use everruns::{Agent, Engine, OpenAI};
let agent = Agent::builder()
.instructions("Answer in one short sentence.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.build()?;
let engine = Engine::new();
let turn = engine.create(agent).send_and_wait("Say hello.").await?;
println!("{}", turn.response);
```
No database, server or worker is required: an agent runs inside your process. A model provider is: pick one from [Supported providers](https://everruns.com/docs/framework/models-and-providers/#supported-providers), or use the [test simulator](https://everruns.com/docs/framework/testing-and-simulation/) when writing tests. ## Choose the right surface | Surface | Use it for | | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Framework** | Rust applications that build and run agents in process through `everruns` | | **Advanced host crates** | Low-level execution-host composition through `everruns-core` (`host` feature) and focused siblings | | **SDKs** | Remote clients that call a running Everruns server | | **Platform (self-hosted)** | The control plane, server, workers, UI, and durable deployment you run yourself, for example with [Docker Compose](https://everruns.com/docs/getting-started/docker-compose/) | | **Everruns Demo** | The Platform operated for you at [app.everruns.com](https://app.everruns.com), with a built-in model provider and a starter credit; bring-your-own keys are optional. See the [Everruns demo quickstart](https://everruns.com/docs/getting-started/cloud/) | Normal library users should start with the Framework. Hosts that must replace storage or orchestration cross into [custom backends](https://everruns.com/docs/framework/custom-backends/). ## Start here * [Quickstart](https://everruns.com/docs/framework/quickstart/), install the crate and run one turn against a live model provider. * [Architecture](https://everruns.com/docs/framework/architecture/), understand Agent, Engine, Session, and the shared immediate/durable execution kernel. * [Upgrade notes](https://everruns.com/docs/framework/upgrade-notes/), the code changes each breaking release needs. ## Core APIs * [Agents and tools](https://everruns.com/docs/framework/agents/), instructions, files, workspaces, MCP, plugins, context inspection, and typed function tools through `everruns::tool`. * [Models and providers](https://everruns.com/docs/framework/models-and-providers/), the model/provider split, every driver that ships today, each driver’s environment variables, provider model catalogs, and custom providers. * [Direct calls and decisions](https://everruns.com/docs/framework/direct-model-calls/), one prompt and one answer without an agent, or a calibrated number rather than prose. * [Sessions](https://everruns.com/docs/framework/sessions/), independent multi-turn conversations, bounded history, typed resume, and engine-lifetime or crash-durable persistence. * [Events and cancellation](https://everruns.com/docs/framework/events-and-cancellation/), observe a live turn, stop work cooperatively, and record bounded canonical event envelopes. * [Observability](https://everruns.com/docs/framework/observability/), export every session to OpenTelemetry or Braintrust, or register your own event listeners. * [Workspaces and environments](https://everruns.com/docs/framework/workspaces-and-environments/), bind sessions to isolated or shared heads and configure read and write scopes with secure defaults. * [Lifecycle hooks](https://everruns.com/docs/framework/lifecycle-hooks/), run awaited application behavior at execution boundaries. * [Answer agent questions](https://everruns.com/docs/framework/ask-user/), implement `AskUser` so your application answers the agent’s structured questions. * [Session work and wakes](https://everruns.com/docs/framework/background-work/), immediate and scheduled work with explicit delivery and restart semantics. * [Coordinator agents](https://everruns.com/docs/framework/coordination/), one session that hands work to threads and hears back from them. ## Extend * [Capabilities](https://everruns.com/docs/framework/advanced-capabilities/), configure the optional standard policy bundle and open references, or package typed tools with stable metadata and lifecycle context. * [GitHub tools](https://everruns.com/docs/framework/github/), read PRs, explore repositories, and explicitly enable writes with application-owned credentials. * [Capability integrations](https://everruns.com/docs/framework/capability-integrations/), opt into filesystem, shell, web, Lua, and MCP implementation boundaries. * [Portable and hosted capabilities](https://everruns.com/docs/framework/capability-boundaries/), understand the Framework/Platform implementation boundary. * [Agent blueprints](https://everruns.com/docs/framework/agent-blueprints/), contribute a code-defined specialist agent from a capability. * [Host services](https://everruns.com/docs/framework/host-services/), compose a host’s own capabilities, provider resolution, retry budgets, MCP credentials and observers into Engine sessions. * [Session controls](https://everruns.com/docs/framework/session-controls/), run slash commands, hot-install capabilities, switch model, effort and MCP servers, and query context on a live session. * [Attach to a running session](https://everruns.com/docs/framework/attach-control/), let a second local process list, watch, drive and cancel a host’s sessions. * [Custom backends](https://everruns.com/docs/framework/custom-backends/), cross into low-level host composition deliberately. * [Testing and simulation](https://everruns.com/docs/framework/testing-and-simulation/), deterministic tests without credentials. * [Runnable examples](https://everruns.com/docs/framework/examples/), complete programs maintained with the crate. ## Expose and deploy * [Deploy a Framework app](https://everruns.com/docs/framework/deployment/), ship the binary or a container with a persistent data directory and environment-provided keys. * [Move to Platform or Cloud](https://everruns.com/docs/framework/moving-to-platform/), what carries over to a hosted agent and what has to be rebuilt. * [Serve](https://everruns.com/docs/framework/serve/) (experimental), attribute macros, file-layout discovery and a manifest, served over the Everruns server `/v1` API. * [Tutorial: Serve an Agent and Call It from Code](https://everruns.com/docs/framework/serve-tutorial/) (experimental), build a serve app, protect it with an API key, and call it with the SDK’s `AgentClient`. * [Serve AG-UI](https://everruns.com/docs/framework/ag-ui/), stream a session to CopilotKit or any AG-UI 1.0 client from your own HTTP server. * [A2A](https://everruns.com/docs/framework/a2a/), serve Framework agents to other agents over A2A 1.0, and delegate work to remote A2A agents. * [Serve on AgentCore](https://everruns.com/docs/framework/serve-agentcore/) (experimental), deploy a serve app to Amazon Bedrock AgentCore Runtime. * [Serve on celld](https://everruns.com/docs/framework/serve-celld/) (experimental), run a serve app durably on celld, self-hosted Durable Objects. [Application services](https://everruns.com/docs/framework/service-integrations/) covers authenticated integrations, data services, durable event triggers, and portable evaluation/trace APIs.
---
# A2A
> Serve Framework agents to other agents over A2A 1.0, and let a Framework agent delegate work to remote A2A agents.
Source:
[A2A](https://a2a-protocol.org) is an open protocol for agents to call each other over HTTP: a JSON-RPC endpoint plus a public Agent Card that says what the agent is and how to reach it. A Framework app can use it in both directions: * **Serve**: other agents, the official `a2a` CLI, or the A2A Inspector call your agents. This comes from the `a2a` feature of [`everruns-serve`](https://everruns.com/docs/framework/serve/). * **Call**: your agent hands work to a remote A2A agent with `spawn_agent`. This comes from the `a2a` feature of `everruns` and the `a2a_agent_delegation` capability. Both speak A2A 1.0. The routes match an Everruns [A2A endpoint](https://everruns.com/docs/features/a2a/), so a caller can move between a serve app and Everruns by changing the base URL. ## Serve your agents over A2A Turn on the feature:
```toml
[dependencies]
everruns-serve = { version = "*", features = ["a2a"] }
```
Every top-level agent in the app is then served at: | Route | What it is | | ---------------------------------------------------------- | -------------------------------------------------------------- | | `POST /v1/channels/{agent}/a2a` | A2A 1.0 JSON-RPC. Requests need the `A2A-Version: 1.0` header. | | `GET /v1/channels/{agent}/a2a/.well-known/agent-card.json` | The Agent Card, built from the agent’s name and `description`. | Nothing else changes in the agent. A `description` is worth setting, because it is what other agents read on the card:
```rust
use serve::prelude::*;
/// Researches a topic and answers with short factual notes.
#[agent]
fn researcher() -> Agent {
Agent::builder()
.model("openai/gpt-6.2-sol")
.description("Researches a topic and answers with short factual notes.")
.instructions(md!("instructions.md"))
.build()
}
```
Try it with the `a2a` CLI against a dev server:
```bash
CARD=http://localhost:3000/v1/channels/researcher/a2a/.well-known/agent-card.json
a2a card get -a "$CARD"
a2a send -a "$CARD" "Tide pools"
a2a send -a "$CARD" --stream "Coral reefs"
a2a task list -a "$CARD"
```
Pass the full card URL: the card lives under the agent’s route, not at the server root. ### How A2A maps onto serve * Each A2A `contextId` is one serve session. A follow-up message with the same `contextId` continues the conversation, and the session survives a restart. * Each A2A task is one turn. A task goes `working`, then delivers the final reply as a `response` artifact, then `completed` (or `failed`). * `SendMessage`, `SendStreamingMessage`, `GetTask`, `ListTasks`, `CancelTask`, and `SubscribeToTask` are supported. * A pending tool approval or `ask_user` question keeps the task `working` until it is answered through serve’s `/v1` routes. ### Limits * Only A2A 1.0 is served. Requests without `A2A-Version: 1.0` are rejected. * Tasks are kept in memory: after a restart, old task ids are unknown, while their contexts continue. * The endpoint takes the same credentials as the other serve routes ([Authentication](https://everruns.com/docs/framework/serve/#authentication)). With auth configured, the card’s `securitySchemes` and `securityRequirements` list one scheme per method (`apiKey` for static keys, `oidc` for an OpenID Connect issuer, `oauth2Bearer` for introspection), any one of which is enough; with none, the card lists no scheme and the endpoint is open. * Push notifications are not offered. Use streaming or `GetTask`. For a hosted endpoint with API keys, push notifications, and A2A 0.3 support, use an Everruns [A2A endpoint](https://everruns.com/docs/features/a2a/). ## Call remote A2A agents Turn on the feature and add the capability, listing the agents your agent may call:
```toml
[dependencies]
everruns = { version = "*", features = ["a2a", "local", "openai"] }
```
```rust
use everruns::{Agent, CapabilityRef, Engine, LocalConfig, OpenAI};
use serde_json::json;
let agent = Agent::builder()
.name("writer")
.instructions(
"Before writing, ask the `researcher` agent for notes with `spawn_agent` \
(target type `external_a2a`, id `researcher`, mode `foreground`). \
Then write one paragraph from its notes.",
)
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.capability(CapabilityRef::new("a2a_agent_delegation").config(json!({
"agents": [{
"id": "researcher",
"name": "Researcher",
"description": "Researches a topic and answers with short factual notes.",
"base_url": "https://agents.example.com/v1/channels/researcher/a2a"
}]
})))
// Delegated runs are recorded in session storage.
.local(LocalConfig::new("./writer-state"))
.build()?;
let session = Engine::new().create(agent);
```
The model sees one `spawn_agent` tool with an `external_a2a` target. It can only reach the agents in `agents`; it never supplies a URL. * `base_url` is where the Agent Card is resolved (`{base_url}/.well-known/agent-card.json`). * `mode: "foreground"` waits for the remote task and returns its reply as the tool result. `mode: "background"` returns a `task_id` at once and wakes the session when the remote task finishes. * `result_schema` on `spawn_agent` requires a structured result and validates the remote agent’s first data part against it. * URLs are checked before each call: localhost, private ranges, and metadata addresses are refused, resolved addresses are pinned, and redirects are not followed. Set `"allow_local_urls": true` on an agent entry only while developing against a server on your machine under `DEPLOYMENT_GRADE=dev`. The remote agent can be anything that speaks A2A 1.0: a serve app, an Everruns A2A endpoint, or another vendor’s agent. ## Run the example [A2A example](https://github.com/everruns/everruns/tree/main/examples/serve/a2a) puts both halves together. `researcher` is a serve app served over A2A, and `writer` is an `everruns` agent that delegates research to it and then drafts. It runs offline with scripted models, and on OpenAI when `OPENAI_API_KEY` is set:
```bash
cargo run -p serve-example-a2a --bin researcher # serves on :3000
cargo run -p serve-example-a2a --bin writer -- "tide pools" # in another shell
```
The writer’s `spawn_agent` call goes over real A2A even offline, so the example is also a quick way to check an A2A setup end to end. ## See also * [Serve](https://everruns.com/docs/framework/serve/#a2a), the serve routes and configuration. * [A2A](https://everruns.com/docs/features/a2a/), the Everruns A2A endpoint and the hosted delegation capability. * [Serve AG-UI](https://everruns.com/docs/framework/ag-ui/), the protocol for agent-to-UI rather than agent-to-agent.
---
# Configure and author capabilities
> Use one open AgentBuilder capability entrypoint for typed built-ins, dynamic references, and code-defined packages.
Source:
Every agent capability enters through `AgentBuilder::capability`. The method accepts the public, non-sealed `IntoCapability` contract, so Framework built-ins and third-party packages compose without adding a method or enum variant to `AgentBuilder`. Cargo features determine which environment-backed implementations a Framework binary contains. See [Capability integrations](https://everruns.com/docs/framework/capability-integrations/) for filesystem, Bashkit, web-fetch, Lua, and MCP boundaries; this page covers agent-level configuration and authoring after an implementation is available. ## Configure capabilities Use typed values when the Framework exposes a stable configuration, a `capability::Definition` for application code, and `CapabilityRef` when the ID and JSON arrive dynamically:
```rust
use everruns::{
Agent, CapabilityRef, CompactionConfig, OpenAI, ToolSearch,
};
use serde_json::json;
let weather_definition = build_weather_capability();
let agent = Agent::builder()
.instructions("Use configured capabilities when relevant.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.capability(CompactionConfig::new().budget_percent(0.85))
.capability(ToolSearch::automatic())
.capability(weather_definition)
.capability(
CapabilityRef::new("vendor.custom")
.config(json!({ "mode": "database-driven" })),
)
.build()?;
```
`ToolSearch::automatic` uses hosted deferred loading on supported models and the existing provider-neutral client-side implementation everywhere else. Its optional threshold and never-defer allowlist map to the built-in’s real configuration; there are no provider fields on the Framework value. `CapabilityRef` is the explicit escape hatch for database, plugin, or catalog configuration. Its ID stays open. An unknown ID is retained as a reference but contributes nothing until the selected host or plugin provides that implementation. This is not a function tool: ordinary functions remain on `AgentBuilder::tool` and `#[everruns::tool]`. The same rule applies to Everruns Platform capability IDs. The default Framework registry does not advertise or execute hosted knowledge, delegation, task, hook, or management capabilities. See [portable and hosted capabilities](https://everruns.com/docs/framework/capability-boundaries/). JSON capability config is not a credential store. Framework debug output redacts it, but a host may persist or inspect it; pass a provider-owned secret handle rather than API keys or tokens. Conversion is infallible. `AgentBuilder::build` validates ID syntax and the JSON object boundary, runs known built-in and declarative/plugin validators, and rejects duplicate IDs after built-in alias resolution. A code implementation cannot shadow a built-in or be paired with a second reference of the same ID; registrations never use last-write-wins behavior. Third-party typed values implement `IntoCapability` using only `everruns`:
```rust
use everruns::{CapabilityRef, CapabilitySpec, IntoCapability};
struct VendorSearch {
index: String,
}
impl IntoCapability for VendorSearch {
fn into_capability(self) -> CapabilitySpec {
CapabilityRef::new("vendor.search")
.config(serde_json::json!({ "index": self.index }))
.into()
}
}
```
No `everruns-core`, registry, store, or host dependency is needed. ## Choose the standard policy bundle The Framework’s default `builtins` feature links `everruns-core` (`builtins` feature), the backend-neutral implementation bundle for compaction, tool search, budgeting, loop/progress safeguards, prompt caching, tool-call repair, output handling, and guardrails. Linking the package has no registration side effect: each host constructs its registry explicitly, so a custom registry cannot be changed by dependency order. Applications that want only the open Framework contracts can disable default features and add the integrations they need. The policy bundle owns no network client, process runner, interpreter, database, or hosted service. Output persistence and distillation declare `session_file_system` as a host-provided dependency; enable them only in a composition that supplies that capability. The optional `ui-capabilities` feature also owns the namespaced `everruns_core::builtins::{openui,a2ui}` component catalogs and prompt generators; applications do not need separate UI-protocol crates. ## Choose an authoring level Use the smallest extension contract that fits the behavior you own. | Contract | `#[everruns::tool]` | `everruns::capability` | | ------------------------------------- | ----------------------------- | ----------------------------- | | Best for | One application function | A reusable capability package | | Typed input and result | Yes | Yes | | Generated input schema | Yes | Yes | | Inspectable output schema | No | Yes | | Multiple tools | Register functions separately | One stable capability id | | Capability instructions and metadata | No | Yes | | Session/workspace identity and locale | No | Curated `Context` accessors | | Progress events | No | `Context::progress` | | [Narration](#tool-narration) override | `narrate = …` | `Handler::narrate` | | Child-work cancellation | Turn future only | `Context::cancellation` | | Backend/store/tenancy access | No | No | ## Ordinary tools Annotate a typed async function. Its doc comment becomes the description and its arguments become JSON Schema.
```rust
/// Convert Celsius to Fahrenheit.
#[everruns::tool]
async fn fahrenheit(celsius: f64) -> f64 {
celsius * 1.8 + 32.0
}
let agent = everruns::Agent::builder()
.instructions("Use the conversion tool.")
.provider(everruns::OpenAI::from_env()?)
.model("gpt-6.2-sol")
.tool(fahrenheit())
.build()?;
```
Prefer this until you need a capability-level contract. ## Advanced capabilities An advanced capability is an immutable `capability::Definition`. It owns a stable id, catalog text, optional instructions and JSON metadata, and one or more typed handlers. `AgentBuilder::capability` installs its implementation on the private in-process runtime and activates that stable id once.
```rust
use everruns::{Agent, OpenAI, capability};
#[derive(capability::Deserialize, capability::JsonSchema)]
#[serde(crate = "everruns::capability::serde")]
#[schemars(crate = "everruns::capability::schemars")]
struct LookupInput {
id: String,
}
#[derive(capability::Serialize, capability::JsonSchema)]
#[serde(crate = "everruns::capability::serde")]
#[schemars(crate = "everruns::capability::schemars")]
struct Record {
id: String,
score: f64,
labels: Vec,
}
struct Lookup;
#[capability::async_trait]
impl capability::Handler for Lookup {
type Input = LookupInput;
type Output = Record;
type Error = capability::Error;
fn name(&self) -> &str { "lookup_record" }
fn description(&self) -> &str { "Look up one record by exact id." }
fn hints(&self) -> capability::Hints {
capability::Hints::default()
.readonly(true)
.idempotent(true)
}
async fn execute(
&self,
input: Self::Input,
context: capability::Context,
) -> Result {
context.progress("Looking up the record").await;
if input.id != "rec_42" {
return Err(capability::Error::user(
"record_not_found",
"No record has that id",
).details(capability::serde_json::json!({ "id": input.id })));
}
Ok(Record {
id: input.id,
score: 0.98,
labels: vec!["verified".into()],
})
}
}
let records = capability::Definition::new(
"records",
"Records",
"Application-owned record lookup.",
)
.instructions("Use exact record ids and do not infer missing records.")
.metadata(capability::serde_json::json!({ "owner": "risk" }))
.tool(Lookup);
let agent = Agent::builder()
.instructions("Answer with verified record data.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.capability(records)
.build()?;
```
Both input and output types must satisfy the compile-time protocol bounds. The generated schemas are available through `Definition::tools()[..].spec()` for tests, documentation, or a host catalog. Results are serialized directly to JSON, so structs, arrays, numbers, booleans, and null do not pass through a string conversion. ## Errors Return `capability::Error::user(code, message)` for an expected domain failure. Add bounded JSON details when they help the model recover. The code, message, and details travel through the model-visible tool-error channel. Return `capability::Error::internal(code, message)` for diagnostic details that are unsafe to show to the model, such as network internals or implementation bugs. The engine logs the diagnostic and gives the model a generic error; internal details do not cross the model boundary. Never include credential values or other secrets in any error because host logs may retain internal diagnostics. Custom application error enums can implement `Into` and be used as `Handler::Error`. ## Context, progress, and cancellation `capability::Context` exposes only stable lifecycle data: * opaque session and workspace ids; * the resolved locale, when present; * best-effort correlated `tool.progress` events; * a call-scoped cancellation signal. Observe progress through `Session::events()` and `SessionEventKind::ToolProgress`. Everruns does not currently expose a custom capability result-streaming protocol; return one typed result when execution finishes. Normal awaited work needs no cancellation branch. Cancelling a turn drops the handler future. Clone `context.cancellation()` only into child tasks, processes, or watchers that might otherwise survive after `execute` is dropped. The signal fires on cancellation and on every other call completion path. ## Tool narration Clients show one human-readable line per tool call phase (started, waiting, completed, failed), carried as `narration` on tool events and read back with `SessionEvent::narration()`. Every tool gets one without extra code: the engine renders a generic, localized line from the tool’s display name and, when it can pick a safe argument, a short detail, such as `Running Lookup Record`. Override it when a domain phrase reads better. Every authoring level has the same optional hook, and returning `None` keeps the generic line: * `capability::Handler::narrate(&self, &ToolNarration) -> Option` * `FunctionTool::narrate(|call: &ToolNarration| -> Option)` * `#[everruns::tool(narrate = |args: &Args, call: &ToolNarration| …)]`, which receives typed arguments; arguments that do not parse keep the generic line. `ToolNarration` gives the raw call `arguments()`, the `phase()`, the session `locale()`, and, on completion, the model-visible `result()`. The `everruns::narration` helpers phrase it the way built-in tools do: `safe_arg_str` reads a string argument but never one whose key looks like a secret, `truncate` and `narration_detail` bound a value to one short line (the latter also strips URL credentials and query strings), and `narrate_labeled_action` renders started, completed, and failed verbs with an optional detail and a Ukrainian locale variant.
```rust
use everruns::{capability, narration};
struct Lookup;
#[capability::async_trait]
impl capability::Handler for Lookup {
// Input, Output, Error, name, description, and execute as above.
fn narrate(&self, call: &capability::ToolNarration<'_>) -> Option {
// No id yet: keep the generic line.
narration::safe_arg_str(call.arguments(), &["id"])?;
Some(narration::narrate_labeled_action(
call.arguments(),
call.phase(),
call.locale(),
("Looking up record", "Looked up record", "Could not look up record"),
("Шукаю запис", "Знайшов запис", "Не вдалося знайти запис"),
&["id"],
))
}
}
/// Convert Celsius to Fahrenheit.
#[everruns::tool(narrate = |args: &FahrenheitArgs, _call: &everruns::ToolNarration<'_>| {
Some(format!("Converting {} °C", narration::truncate(&args.celsius.to_string(), 16)))
})]
async fn fahrenheit(celsius: f64) -> f64 {
celsius * 1.8 + 32.0
}
```
Narration arguments are untrusted model output and narration is shown to people and stored with session events. Render only fields you chose, keep them bounded, and never include credentials or other secrets. Hosts that implement the low-level `everruns_core::tools::Tool` trait directly override its `narrate` method with the same contract. ## Security boundary The advanced SPI intentionally does not export provider credentials, stores, tenant or organization objects, registries, payment authority, filesystem backends, or other host services. Pass application-owned clients or state into your handler struct when constructing the definition. A handler is trusted application code and retains whatever process authority those values provide; `Hints` describe behavior but do not enforce authorization, egress, or approval policy. Apply authorization, egress policy, timeouts, and input bounds at those application boundaries, and never place secrets in capability or tool metadata. For a complete provider-backed program, run the [`advanced_capability` example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/advanced_capability.rs). Some built-ins expect the application to supply behavior rather than configuration. `ask_user` is one: it needs someone to answer, so `AgentBuilder::ask_user` takes a responder instead of a config value. See [Answer agent questions](https://everruns.com/docs/framework/ask-user/).
---
# Serve AG-UI
> Stream a Framework session to CopilotKit or any AG-UI 1.0 client from your own HTTP server, with ask_user questions and tool approvals as interrupts.
Source:
[AG-UI](https://docs.ag-ui.com) is the event protocol between an agent and the application that renders it. CopilotKit and `@ag-ui/client` post a `RunAgentInput` and read back a stream of events, usually as server-sent events. The `ag-ui` feature lets any Rust host answer those requests from a Framework session, without the Everruns server:
```bash
cargo add everruns --features ag-ui
```
`Session::ag_ui(input)` sends the input’s last user message and returns the run as a stream of AG-UI 1.0 events: `RUN_STARTED`, the projected turn, then one `RUN_FINISHED` or `RUN_ERROR`. The projection is the one the Everruns server uses, with the trusted policy: assistant text, reasoning, and token usage are visible, and a failure carries the runtime’s message. ## Mount it on axum The `ag-ui-axum` feature adds `AgUiHandler`, a ready-made route:
```bash
cargo add everruns --features ag-ui-axum
```
```rust
use axum::Router;
use everruns::ag_ui::{AgUiHandler, AgUiOptions, AgUiThreads, InterruptGate, StaticToken};
use everruns::{Agent, Engine, OpenAI};
#[tokio::main]
async fn main() -> Result<(), Box> {
let gate = InterruptGate::new();
let agent = Agent::builder()
.instructions("You are a helpful assistant.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.ask_user(gate.clone())
.approver(gate.clone())
.build()?;
let handler = AgUiHandler::new(
AgUiThreads::new(Engine::new(), agent),
StaticToken::bearer(std::env::var("AG_UI_TOKEN")?),
)
.options(AgUiOptions::new().gate(gate));
let router = Router::new().route("/ag-ui", handler.route());
let listener = tokio::net::TcpListener::bind("127.0.0.1:3000").await?;
axum::serve(listener, router).await?;
Ok(())
}
```
Each request is authorized, its `threadId` resolved to a session with `AgUiThreads` (the first run of a thread creates one from your agent, later runs reopen it), and the run streamed as server-sent events: one unnamed `data: ` event per AG-UI event and a `: keepalive` comment every 15 seconds. Everything that can fail does so before the stream opens: | Status | When | | ------ | ---------------------------------------------------------------------------------------------- | | `401` | The authorizer refuses the request. The body is not read. | | `400` | The body is not a `RunAgentInput`, the `threadId` is invalid, or the run cannot use the input. | | `500` | The thread store or the session failed. The detail is logged, not returned. | Errors are `application/problem+json`. Once the stream is open, a failure is the run’s own `RUN_ERROR`. Request bodies are capped by axum’s default 2 MB limit. ### Authorization `AgUiHandler::new` takes an authorizer, so an open endpoint is a choice you write down: * `StaticToken::bearer(token)` accepts `Authorization: Bearer `, and `StaticToken::header(name, token)` a custom header. Tokens are compared in constant time; a refusal carries `WWW-Authenticate: Bearer` for the bearer form. * `Unauthenticated` accepts everything: for local development, or behind a proxy that already authenticates. * Any `Fn(&HeaderMap) -> Result`, or your own `AgUiAuthorizer` implementation when the check needs to await. The authorizer returns an `AgUiCaller`. `AgUiCaller::scoped(user_id)` gives each caller its own threads, so one user cannot continue another’s conversation by sending its `threadId`:
```rust
use axum::http::HeaderMap;
use everruns::ag_ui::{AgUiCaller, Unauthorized};
// Your proxy has verified the user and set this header.
let by_user = |headers: &HeaderMap| {
headers
.get("x-user-id")
.and_then(|value| value.to_str().ok())
.map(AgUiCaller::scoped)
.ok_or_else(Unauthorized::new)
};
let handler = AgUiHandler::new(threads, by_user);
```
### Your own route On another HTTP server, or to resolve sessions yourself, call `Session::ag_ui_with` (or `AgUiThreads::run`) and frame the stream. With `ag-ui-axum`, `sse_response(run)` does the framing; without it, each event is `serde_json::to_string(&event)` in a `data:` line. The crate’s `ag_ui_axum` example is the handler on the offline simulated model:
```bash
cargo run -p everruns --features ag-ui-axum --example ag_ui_axum
```
Point any AG-UI client at the route:
```ts
import { HttpAgent } from "@ag-ui/client";
const agent = new HttpAgent({
url: "http://127.0.0.1:3000/ag-ui",
headers: { Authorization: `Bearer ${token}` },
});
```
## Threads across restarts `AgUiThreads::new` keeps its thread map in memory. To keep threads when the process restarts, keep both halves on disk: the sessions, with an agent on the `local` backend, and the map, with `SqliteThreadStore` (both need the `local` feature):
```rust
use everruns::ag_ui::{AgUiThreads, SqliteThreadStore};
use everruns::{Agent, Engine, LocalConfig, OpenAI};
let config = LocalConfig::new("/var/lib/my-app");
let agent = Agent::builder()
.instructions("You are a helpful assistant.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.local(config.clone())
.build()?;
let threads = AgUiThreads::with_store(Engine::new(), agent, SqliteThreadStore::local(&config)?);
```
After a restart the next run of a thread reopens its session, history included. Implement `ThreadStore` to keep the map in your own database instead. With durable session hosting (`SqliteHosting`), use `AgUiThreads::hosted` and a registered build instead of an Agent value. Each new thread is created through the authority, at an identity derived from the thread, so a retried first run reuses the same session. Threads whose sessions were moved from the `local` backend with `Engine::cut_over_legacy_session` keep answering from those same sessions:
```rust
let threads = AgUiThreads::hosted(engine, build, SqliteThreadStore::local(&config)?);
```
A thread id is chosen by the client, and anyone who knows one continues that conversation. When one server serves several users, scope threads to the authenticated caller: `AgUiCaller::scoped` does it in `AgUiHandler`, and `run_in` on your own route:
```rust
let run = app.threads.run_in(&user_id, input, options).await?;
```
Thread ids must be 1 to 128 characters of `[A-Za-z0-9-_.]`; anything else is `AgUiError::InvalidInput`. ## History the client already has A client such as CopilotKit keeps the conversation and sends all of it with every run. When a thread is new to the server, for example after a restart without a durable backend, `AgUiThreads` records the input’s earlier user and assistant messages as the new session’s history before the run’s user message, so the agent picks up where the conversation was. Once the session has history, the earlier messages are not read again: the session’s own conversation wins. Only user and assistant text is seeded (an assistant’s tool calls as text); `tool`, `system`, `developer`, activity and reasoning messages are not. The most recent 256 messages and 512 KiB of text are kept. On a session you resolve yourself, turn it on with `AgUiOptions::seed_history(true)` for a thread’s first run. The client writes these messages, assistant turns included, so treat them as no more trusted than the user message. ## Questions and approvals become interrupts In the Framework, `ask_user` questions and tool approvals are answered by responders on the agent while the turn waits. `InterruptGate` is a responder for both that answers through AG-UI instead: register it with `.ask_user(...)` and `.approver(...)`, and pass it with `AgUiOptions::gate`. A question or approval then ends the run with the 1.0 interrupt outcome while the turn stays parked in your process: | Parked on | `reason` | Answer (`resume[].payload`) | | ----------------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------- | | `ask_user` | `everruns.ask_user` | `{ "answers": [{ "id", "selected", "other_text" }], "status"? }`, described by the interrupt’s `responseSchema` | | `ask_user` with a `secret` question | `everruns.secret_required` | none: only abandoning is accepted | | a tool that needs approval | `tool_approval` | `{ "decision": "allow" \| "allow_always" \| "reject" \| "reject_always" }` | The interrupt id is the tool call id. The next run sends `resume` entries and no new message; the entries answer the parked requests and the run streams the rest of the same turn. Abandoning an entry (`status: "cancelled"`) declines the question or rejects the call. An interrupt left without an entry is not abandoned: nothing is resolved and the run ends with the interrupts again, as it does when a new message arrives while one is open. An entry that cannot be applied returns `AgUiError::InvalidInput` before the stream opens, and nothing is resolved. Parked requests live in memory. If the process exits, the waiting turn is left unfinished in the session’s log; after a restart, `Session::resume_interrupted_turn` runs its waiting calls again, so they park on the gate anew and a client’s resume run can answer them (see [session history](https://everruns.com/docs/framework/sessions/#history-and-resume)). ## Policy `AgUiOptions::policy` takes a `ProjectionPolicy` to hide reasoning or usage, show fixed text while tools run, or map failures to your own error codes:
```rust
use everruns::ag_ui::{AgUiOptions, ProjectionPolicy};
let options = AgUiOptions::new().policy(ProjectionPolicy {
reasoning_visible: false,
usage_visible: false,
tool_activity_text: Some("Working on it...".into()),
..ProjectionPolicy::default()
});
```
## What a run reads from the input The session owns the conversation, so only the last message is sent, and it must be a user message (its text parts), unless the run carries frontend tool results (below). Earlier messages are read only to seed a new thread (above); `state` and `forwardedProps` are not read. `RUN_STARTED` carries `protocolVersion: "1.0"` only when the request declared a version, so pre-1.0 clients see the stream they expect. ## Frontend tools The input’s `tools` are tools your page runs, such as a confirmation dialog or a navigation action. Each run makes them the session’s client-side tools, so send them on every run; the latest set wins. When the model calls one, the turn parks and the run streams the call as `TOOL_CALL_START`, `TOOL_CALL_ARGS` and `TOOL_CALL_END` with its name and arguments, then finishes in success with `outcome.pendingToolCallIds`. Run the calls in the page, then post the next run with their results as trailing `tool` messages (`toolCallId` set, `content` the result, `error` for a failure). The same turn continues and the run streams the rest of it. A result for a call that is not parked is ignored. Until every parked call has a result, nothing is recorded and the run reports the calls again, as it does when a new user message arrives instead. Definitions are bounded: at most 64 tools, names matching `^[A-Za-z0-9_-]{1,64}$`, descriptions up to 4096 characters, parameter schemas up to 16 KiB, results up to 256 KiB. Names starting with `mcp_` are refused. Parked calls live in memory: if the process exits, they stay unanswered. `system` and `developer` messages and `context` entries are ignored by default, and a system message after the user message is refused. If your server authenticates whoever posts the input, you can let each run carry instructions:
```rust
use everruns::ag_ui::AgUiOptions;
let options = AgUiOptions::new().input_instructions(true);
```
Each run’s system and developer messages, in order, then its context entries, are then appended to the agent’s instructions for that run. They may appear anywhere in `messages`, including after the user message, which stays the run’s input. The next run replaces them with its own, and a run with none clears them; they never enter the conversation history. Leave this off for callers you do not trust: these messages carry the authority of the system prompt. A [serve](https://everruns.com/docs/framework/serve/#ag-ui-and-copilotkit) app gets this route built in with its `ag-ui` feature, at `/v1/channels/{agent}/ag-ui`. For a hosted agent with no server code, Everruns serves the same protocol at `/v1/channels/{channel_id}/ag-ui`.
---
# Author an agent blueprint
> Contribute a code-defined specialist agent from a capability, with a typed configuration contract the spawn path enforces
Source:
An **agent blueprint** is a code-defined specialist agent: a baked-in prompt, a set of private tools, a model-selection strategy, an iteration bound, and a narrow configuration surface. A host agent delegates to it through the ordinary [sub-agents](https://everruns.com/docs/capabilities/sub-agents/) tool without gaining access to its internals. Reach for a blueprint when work needs different tools, instructions, or model economics than the parent agent — repository scouting, catalog benchmarking, any job where the parent should get the answer without carrying the tools that produced it. Blueprints are not persisted user-created agents; they ship with a capability and are available wherever that capability is enabled. ## Contribute the blueprint A capability contributes blueprints by implementing `agent_blueprints()`. The returned `AgentBlueprint` carries everything the child runtime needs:
```rust
fn agent_blueprints(&self) -> Vec {
vec![AgentBlueprint {
id: "repo_scout",
name: "Repo Scout",
description: "Search repositories for code, files, and issues. \
Read-only agent for codebase exploration.",
model: BlueprintModel::Fixed("claude-haiku-5-5".to_string()),
system_prompt: REPO_SCOUT_PROMPT,
tools: vec![Box::new(SearchCodeTool), Box::new(ReadFileTool)],
max_turns: Some(15),
config_schema: Some(json_schema_for::()),
}]
}
```
The `description` is what a parent agent reads when deciding whether to delegate, so write it as a routing decision: what the blueprint is for, and when to pick it. `model` chooses one of three strategies. `Fixed` pins a model the host cannot override, which suits specialist work with a known cost/quality target. `Default` names a model the validated config may override. `Inherit` takes the parent’s model, for work that genuinely needs the parent’s characteristics. Tools listed here are **private**. They are instantiated only for the blueprint’s own child session and never appear in the host agent’s tool list. ## Define configuration as a type Derive the config schema from a Rust struct rather than writing JSON by hand. The struct is the single source of truth: field set, bounds, defaults, and descriptions all reach the spawning agent from one place.
```rust
use everruns_contracts::capability::json_schema_for;
use everruns_contracts::capability::schemars::JsonSchema;
use serde::{Deserialize, Serialize};
/// The same ceiling the search tools apply to their own arguments.
const MAX_REPOS: u32 = 50;
/// Configuration for the repository scout.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields, default)]
#[schemars(crate = "everruns_contracts::capability::schemars")]
pub struct RepoScoutConfig {
/// Maximum number of repositories to scan.
#[schemars(range(min = 1, max = MAX_REPOS))]
pub max_repos: u32,
}
impl Default for RepoScoutConfig {
fn default() -> Self {
Self { max_repos: 10 }
}
}
```
Three habits keep the contract honest: * **Express each bound once.** Write it as a constant shared by the schema attribute and the code that enforces it at runtime, so a schema edit cannot drift from the clamp it describes. * **Write doc comments for the caller.** They become the schema descriptions the spawning agent reads. Implementation notes belong in ordinary `//` comments. * **Close the object** with `deny_unknown_fields` unless the blueprint genuinely accepts open configuration. Requiring configuration is a matter of leaving a field without a default: fields serde cannot default become `required` in the derived schema, and the spawn path rejects a call that omits them. ## What the spawn path enforces Configuration is validated against the derived schema **before** a child session exists, so a declared bound constrains the host rather than merely advising the model. A spawn is rejected when: * a value violates the schema — out of range, wrong type, missing a required property; * the config carries a key the schema does not define (with `deny_unknown_fields`); * the blueprint declares no schema at all but config was supplied. The error returned to the calling agent names the violations and includes the schema, so a model can usually correct its own call and retry. Validation governs what a **host** may configure. It is not a substitute for a tool checking its own arguments: keep the runtime clamps in the blueprint’s tools, so a misbehaving child agent stays inside the same envelope. Configuration can only select behavior the blueprint intentionally exposes. It cannot replace the system prompt, inject tools, bypass model policy, or expand capability permissions. ## Delegation and lifetime A spawned blueprint session is a real durable session. It takes part in the same message, event, task, workspace, cancellation, and recovery infrastructure as any other sub-agent; the difference is only how its runtime is assembled. The session persists the blueprint identity and its validated config, so a worker can reconstruct the same runtime after a retry or handoff. The blueprint itself stays a stateless template. Follow-ups address the durable session, not the blueprint. ## Worked examples Two blueprints ship in-tree and are worth reading as references: * [`crates/integrations/src/github/lib.rs`](https://github.com/everruns/everruns/blob/main/crates/integrations/src/github/lib.rs) — the minimal case: one config field, a pattern constant shared with the runtime check that enforces it. * [`crates/integrations/src/openrouter/model_scout.rs`](https://github.com/everruns/everruns/blob/main/crates/integrations/src/openrouter/model_scout.rs) — the fuller case: numeric bounds tied to runtime constants, a nested config type, and a spend budget. ## Related * [Sub-agents](https://everruns.com/docs/capabilities/sub-agents/) — the delegation tool blueprints are invoked through * [GitHub Scout](https://everruns.com/docs/capabilities/github-scout/) — a shipped blueprint from the caller’s side
---
# Agents and Tools
> Describe an agent with instructions, a model, tools, files, integrations, and an optional workspace, and add typed Rust functions as tools.
Source:
An `Agent` is an immutable, validated application description. Pass it to an application-owned engine to create independent sessions.
```rust
use everruns::{Agent, McpServer, OpenAI};
let agent = Agent::builder()
.name("researcher")
.instructions("Research carefully and cite the evidence you used.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.file("brief.md", "Investigate the supplied question.")
.readonly_file("policy.md", "Never expose secrets.")
.mcp_server(McpServer::http("catalog", "https://example.com/mcp"))
.build()?;
```
Builder validation catches blank instructions, a missing model, duplicate providers, tools, or capabilities, invalid tool schemas, invalid capability IDs/configuration, implementation collisions, and invalid MCP configuration before a session starts. Configure typed built-ins, code-defined packages, and dynamic references through the single `capability(...)` entrypoint; see [Configure and author capabilities](https://everruns.com/docs/framework/advanced-capabilities/). ## Harness foundations Choose `Harness::base()` for the system essentials only, `conversation()` for dialogue, `worker()` for files, skills, long context, budgeting and task coordination without a shell, or `bashkit_worker()` to add the Bashkit shell. These constructors share the hosted platform’s [preset definitions](https://everruns.com/docs/features/harnesses/).
```rust
let engine = Engine::new();
let session = engine
.create(agent)
.harness(Harness::conversation())
.start()
.await?;
let response = session.send_and_wait("Tell me a joke.").await?;
```
Enable optional host integrations for the tools your application needs. Worker [delegation requires host backends](https://everruns.com/docs/built-ins/harnesses/worker/); the default in-memory host does not supply them. `Harness::generic()` is deprecated and preserves its legacy capabilities. With `builtins` enabled, sessions without a bound harness include managed compaction and history retrieval. Explicit custom harnesses keep their own capability selection. Compaction observation is available through the [Framework event API](https://everruns.com/docs/framework/evaluation-and-traces/#compaction-observation); organization monitoring consent and reporting remain host responsibilities. Presets preserve the framework’s read-only workspace default. For an agent that writes files, explicitly select `Agent::builder().workspace_policy(WorkspacePolicy::read_write())`. ## Files and workspaces * `file(path, content)` seeds an editable file. * `readonly_file(path, content)` seeds a file the agent may read but not change. * `workspace(root)` exposes one trusted real-disk root as `/workspace`. Choose workspace roots from trusted application configuration. Model output and untrusted request fields must not select executable paths or host directories. The underlying filesystem boundary rejects traversal and symlink escape. Use a [`WorkspacePolicy`](https://everruns.com/docs/framework/workspaces-and-environments/#workspace-security) to configure portable read, write, hidden-path, and recursive-delete restrictions. ## MCP and plugins `McpServer::http` adds a remote Streamable HTTP server. Headers may be supplied by the host and are redacted from `Debug`. Local-process MCP is separately feature-gated with `mcp-stdio`; its command, arguments, and environment are trusted host configuration. `AgentBuilder::plugin(path)` loads a local plugin directory and returns a typed error if it cannot be compiled. Non-fatal compiler warnings remain visible in the application-facing session context. ## Inspect effective context Inspect the next model call before or after a turn:
```rust
let engine = Engine::new();
let session = engine.create(agent);
let context = session.inspect().await?;
println!("messages: {}", context.messages.len());
println!("tools: {}", context.tools.len());
```
Inspection uses the same assembly path as execution, including MCP discovery, plugin prompt contributions, message filters, and model selection. ## Tools The default-enabled `everruns::tool` macro turns an async Rust function into a typed agent tool. Parameter types produce JSON Schema and call arguments are deserialized before the function runs.
```rust
use everruns::{Agent, OpenAI};
#[everruns::tool]
/// Add two integers.
async fn add(left: i64, right: i64) -> Result {
Ok(left + right)
}
let agent = Agent::builder()
.instructions("Use the add tool for arithmetic.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.tool(add())
.build()?;
```
Use `#[everruns::tool(name = "…", description = "…")]` to override metadata, or `#[tool(rename = "…")]` on a parameter to change its model-facing name. Functions must be async, non-generic, and have plain named parameters. The published `everruns-macros` package is an implementation crate. Applications should use the re-exported `everruns::tool` macro and should not depend on `everruns-macros` directly. ### Dynamic handlers `FunctionTool::new` is available when a tool schema is determined at runtime:
```rust
use everruns::FunctionTool;
use serde_json::json;
let echo = FunctionTool::new(
"echo",
"Return the supplied text.",
json!({
"type": "object",
"properties": { "text": { "type": "string" } },
"required": ["text"]
}),
|args: serde_json::Value| async move {
Ok::<_, String>(args["text"].clone())
},
);
```
Prefer the macro for normal typed application tools. Use the dynamic form for schemas obtained from configuration or another protocol. ### Call context and progress Make the first parameter a `ToolCallContext` (by value or reference) to learn which session, turn, and tool call a handler is serving, and to report progress. Progress arrives on `Session::events()` as `SessionEventKind::ToolProgress`. The context is not a model argument, so it never appears in the schema. The dynamic equivalent is `FunctionTool::with_context`.
```rust
use everruns::ToolCallContext;
/// Export the report.
#[everruns::tool]
async fn export(ctx: &ToolCallContext, pages: u32) -> String {
ctx.progress(format!("rendering {pages} pages")).await;
format!("exported for session {}", ctx.session_id())
}
```
### Approval Mark a tool with `needs_approval` to ask a host approver before it runs. The bare flag asks on every call. `needs_approval = ` asks only when the rule returns `true`. The rule receives the generated arguments struct, named after the function in PascalCase with an `Args` suffix (`run_sql` becomes `RunSqlArgs`). The approver gets the session id and the tool call, including its id and arguments, so a host serving many sessions can route the request. A rejected call never runs, and the model receives a tool error saying it was rejected. An agent with a gated tool but no `.approver(..)` fails to build.
```rust
use everruns::approval::{ApprovalDecision, ToolApprover, async_trait};
use everruns::{Agent, Model, SessionId, ToolCall, ToolDefinition};
/// Run one SQL statement.
#[everruns::tool(needs_approval = |args: &RunSqlArgs| args.sql.contains("DROP"))]
async fn run_sql(sql: String) -> String {
sql
}
struct Console;
#[async_trait]
impl ToolApprover for Console {
async fn approve(&self, _: SessionId, call: &ToolCall, _: &ToolDefinition) -> ApprovalDecision {
println!("approve {}? {}", call.name, call.arguments);
ApprovalDecision::Reject
}
}
let agent = Agent::builder()
.instructions("Answer with SQL.")
.model(Model::simulated("Done."))
.tool(run_sql())
.approver(Console)
.build()?;
```
`FunctionTool::needs_approval(|args: &Value| ..)` and `FunctionTool::always_needs_approval()` are the dynamic forms. ## Agents that hand work to other agents Add the agents this one can hand work to with `.member(agent)`. Each member keeps its own instructions, model and capabilities. The lead gets the `spawn_agent` tool and sees each member by its name and description; work it hands over runs as the member’s own session in the same Engine, in the foreground or as a background task.
```rust
use everruns::hosting::{AgentBuild, SqliteHosting};
use everruns::{Agent, Engine, Model, SessionId};
let reviewer = Agent::builder()
.name("reviewer")
.description("Reviews a change and lists what to fix.")
.instructions("Review the change you are given. Be specific.")
.model(Model::simulated("Looks good."))
.build()?;
let lead = Agent::builder()
.name("lead")
.instructions("Hand each change to reviewer before you answer.")
.model(Model::simulated("Done."))
.member(reviewer)
.build()?;
let build = AgentBuild::new("lead", "v1");
let engine = Engine::builder()
.hosting(SqliteHosting::open("./.everruns/authority.db")?)
.register(build.clone(), lead)
.build();
let session = engine.create_registered(SessionId::new(), &build).await?;
session.send_and_wait("Review the login change.").await?;
```
Members need a host that keeps child sessions: an Engine with `SqliteHosting`, or a local profile. Registering the lead is enough; a member’s sessions resume after a restart under a build derived from the lead’s. Members nest one level. Inviting a member into the lead’s own session is Platform-only. ## File-based agents Load and export the shared portable package format, including initial files and complete skill directories. See [Define agents as files](https://everruns.com/docs/how-to/define-agents-as-files/) for validation, diffs, model bindings and runnable examples.
---
# Framework Architecture
> Understand Agent, Engine, Session, pluggable session hosting, and the shared execution kernel.
Source:
Everruns uses one agent runtime across native processes, platform workers and Wasm isolates. Applications define ordinary `Agent` values and use `everruns::Engine` and `Session`. A session hosting profile owns durable admission, canonical state, ownership epochs and activation. It does not reimplement agent planning, tool policy or recovery.
```
flowchart TD
App[Framework app or serve] --> Engine[Engine and Session]
Engine --> Memory[Memory]
Engine --> SQLite[Native SQLite]
Engine --> External[External host]
Memory --> Driver[Shared SessionActorDriver]
SQLite --> Driver
External --> Driver
Platform[Platform workers] --> Driver
Driver --> Runtime[Input / Reason / Act runtime]
```
## Public Framework objects | Object | Responsibility | | ------------- | --------------------------------------------------------------------------------------------------------------------- | | `Agent` | Immutable behavior: instructions, model and provider, tools, capabilities, files, and lifecycle hooks | | `Engine` | Retained behavior registry, session admission, lifecycle, queries and activation through the selected hosting profile | | `Session` | First-class, engine-bound conversation used for turns, steering, events, cancellation, inspection, and history | | `Environment` | Session resources, including one exact backend-owned workspace head and typed extensions | New Framework code creates and resumes sessions through an Engine:
```rust
use everruns::{Agent, Engine, OpenAI};
let agent = Agent::builder()
.instructions("Answer concisely.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.build()?;
let engine = Engine::new();
let session = engine.create(agent);
let session_id = session.session_id();
let turn = session.send_and_wait("Begin.").await?;
assert!(turn.success);
drop(session);
let resumed = engine.resume(session_id).await?;
assert_eq!(resumed.session_id(), session_id);
```
## Shared execution, different authorities Framework profiles use the same core `SessionActorDriver`, turn loop and event-log recovery. Memory coordinates one process. SQLite commits creation, commands, receipts and wakeups in one database and can run an independent native supervisor. External hosts implement the [public hosting SPI](https://everruns.com/docs/framework/hosting/). The source-only Durable Object reference demonstrates object-local Wasm and a separately authenticated native Engine executor without a Framework vendor dependency. Admission commits before a send is acknowledged; a turn identity is assigned later. Retain command identities across uncertain replies. Dropping a Session, wait handle or SSE connection does not cancel durable accepted work. Reopen its admission handle to observe assignment, pending client tools, suspension or a finalized outcome. Awaited Framework completion hooks settle before that outcome becomes final. The Platform server uses its existing transactional PostgreSQL actor authority. Workers lease runnable sessions and drive the same runtime. The Platform API’s internal `TurnBackend` remains a control-plane integration detail. The generic `everruns-durable` task engine supplies Platform background jobs, worker management and schedules. Agent turns use the shared actor driver. ## Choose a recovery boundary * **Volatile Framework:** `Engine::new()` is offline and database-free. The creating Engine can resume a dropped Session, but process exit loses it. * **Native durable Engine:** select `hosting::SqliteHosting`, register the exact `AgentBuild` and Environment factory, and start its supervisor for independent recovery. SQLite durability covers the configured disk’s failure domain. * **External durable Engine:** implement `SessionHosting`, retain exact builds, and configure independent activation plus explicit host services. The [Durable Object reference](https://everruns.com/docs/framework/durable-objects/) is one source-only implementation, outside the product publish set. * **Legacy local storage:** explicit `LocalConfig` and custom host backends retain the older facade actor lane and its weaker recovery boundary. Persistent stores alone do not provide coherent session hosting or independent activation. * **Distributed durable Platform:** server and workers keep each session’s event log in PostgreSQL and lease sessions to workers, so a turn recovers across process or worker loss. Applications call it through the remote API or SDKs rather than configuring the facade Engine. See [Persistence](https://everruns.com/docs/framework/sessions/#persistence) and [Session History and Resume](https://everruns.com/docs/framework/sessions/#history-and-resume) for the exact application lifecycle. ## Extension boundaries Normal applications depend on `everruns`. Engine is concrete. Hosting plugins implement `SessionHosting` using `hosting::spi` and supply a coherent authority plus epoch-scoped storage ports. The Framework owns lifecycle execution and outcome projection; plugins never implement another agent loop. Providers implement `ChatDriver`. Advanced core hosts can still compose the experimental `Execution` and internal turn-backend contracts. Framework applications should use the session hosting boundary, which also covers creation, admission, callbacks, reads and wakeups. Workspaces, compute, secrets, channels and schedules require explicit services with their own lifetime and isolation guarantees. A durable journal does not preserve those resources automatically.
---
# Answer an agent's questions
> Implement the AskUser trait so an embedding application can answer an agent's structured questions from its own interface.
Source:
An agent with the [Ask User](https://everruns.com/docs/capabilities/ask-user/) capability can ask the person it is working with a small batch of structured questions and wait for the answer. In a hosted product the browser renders that card. In an embedding application there is no browser, so the application answers — which is what the `AskUser` trait is for.
```rust
use everruns::ask_user::{Answer, AnsweredBy, AskUser, Outcome, Question, Status, async_trait};
use everruns::{Agent, Model};
struct HouseRules;
#[async_trait]
impl AskUser for HouseRules {
async fn ask(&self, questions: &[Question]) -> Outcome {
let answers = questions
.iter()
.map(|question| Answer {
id: question.id.clone().unwrap_or_default(),
selected: question
.options
.iter()
.find(|option| option.is_default)
.or_else(|| question.options.first())
.map(|option| option.label.clone())
.into_iter()
.collect(),
other_text: None,
secret_ref: None,
})
.collect();
Outcome {
status: Status::Answered,
answered_by: AnsweredBy::Unattended,
answers,
}
}
}
let agent = Agent::builder()
.instructions("Confirm deployment choices before acting.")
.model(Model::simulated("Done."))
.ask_user(HouseRules)
.build()?;
```
`AgentBuilder::ask_user` registers the responder and enables the capability in one call. The responder runs **inside** the tool call, so the turn never parks waiting for an external result — the agent asks, your code answers, and the turn continues. ## Route by session A host that serves several sessions overrides `ask_in` instead of relying on the default, which ignores the context and calls `ask`. `AskContext` carries the session id, the `ask_user` tool call id, and the turn id when known.
```rust
use everruns::ask_user::{AskContext, AskUser, DefaultsResponder, Outcome, Question, async_trait};
struct Router;
#[async_trait]
impl AskUser for Router {
// Only reached when a caller has no context to give.
async fn ask(&self, questions: &[Question]) -> Outcome {
DefaultsResponder.ask(questions).await
}
async fn ask_in(&self, context: &AskContext, questions: &[Question]) -> Outcome {
// Send `questions` to the client that owns `context.session_id()`.
}
}
```
## Without a responder `.capability("ask_user")` on its own uses `DefaultsResponder`: it applies the options the model marked as recommended, falls back to the first option, and reports `AnsweredBy::Unattended`. Headless runs resolve immediately rather than waiting out the timeout for somebody who is not there. A text or secret question has no unattended value. `DefaultsResponder` declines a batch containing either kind rather than returning an empty answer. ## Report who answered `answered_by` is part of the contract, not decoration: | Value | Meaning | | ------------ | --------------------------------------------- | | `User` | A person actually chose this | | `Timeout` | The deadline passed and a default was applied | | `Unattended` | Nobody could be asked; a default was applied | Report `User` only when a person really answered. An agent that reads a fallback as a considered choice acts with more confidence than the answer earns, and that is the failure this field exists to prevent. ## A worked responder [Weekend Concierge host](https://github.com/everruns/everruns/tree/main/examples/weekend-concierge-host) implements `TerminalResponder` over stdin: numbered options, comma-separated toggles for a multi-select, a free-text path when the question allows one, and terminal echo turned off for a credential.
```plaintext
[Energy] How much energy does the group have on Friday?
*1. Up for anything — Games, noise, moving around.
2. Low-key — Sitting, talking, snacks.
3. Something else
>
```
Two details in it are worth copying into any responder: **An empty answer takes the declared default** rather than returning nothing. A question the model asked and nobody addressed is something it cannot distinguish from a deliberate skip. **A secret answer carries a reference, never a value.** `Answer` has no `value` field at all, so there is no path from a collected credential into the transcript. The host keeps the value; the agent gets `session:MY_TOKEN` and tools resolve it by name. **A text answer carries its string in `other_text`.** It has no selected option:
```rust
Answer {
id,
selected: Vec::new(),
other_text: Some("feature/open-question".to_string()),
secret_ref: None,
}
```
```rust
Answer {
id,
selected: Vec::new(),
other_text: None,
secret_ref: Some(everruns::ask_user::session_secret_ref("MY_TOKEN")),
}
```
For the smallest possible version, [Ask User example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/ask_user.rs) runs a responder and the unattended path side by side and prints what each decided. ## Questions are not permission `ask_user` auto-resolves, so it is for decisions and preferences only. A destructive, irreversible, or outward-facing action needs `request_approval`, whose wait does not auto-resolve. See [the boundary](https://everruns.com/docs/capabilities/ask-user/#not-a-consent-gate). ## See Also * [Ask User capability](https://everruns.com/docs/capabilities/ask-user/), the contract and its limits * [Configure and author capabilities](https://everruns.com/docs/framework/advanced-capabilities/) * [Lifecycle hooks](https://everruns.com/docs/framework/lifecycle-hooks/), for intercepting tool calls rather than answering them
---
# Attach to a Running Session
> Let a second local process list a host's sessions, watch their output, send input and cancel turns, with same-user access, a rotating token and revocation.
Source:
A terminal host often wants `myagent attach` in a second terminal: see which sessions are live, watch one’s output, type into it, stop a turn that went the wrong way. The Engine’s sessions live in the host process, so the host serves a small local control plane and the second process talks to it. The `everruns::control` module (feature `control`, Unix only) provides both halves. Stability: experimental.
```toml
everruns = { version = "*", features = ["control", "openai"] }
```
## Serve sessions from the host Put the sessions a second process may reach in a `ControlRegistry`, then start a `ControlServer` on a private directory. Nothing outside the registry is reachable.
```rust
use everruns::control::{ControlRegistry, ControlServer, default_dir};
use everruns::{Agent, Engine, OpenAI};
let agent = Agent::builder()
.instructions("Keep answers short.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.build()?;
let session = Engine::new().create(agent);
let registry = ControlRegistry::new();
registry.register("main", &session);
let server = ControlServer::start(default_dir("myagent"), registry.clone())?;
println!("attach with: myagent attach --dir {}", server.attach_info().dir.display());
// The host's own prompts go through the registry, so an attached
// `cancel` can reach the turn they start.
let reply = registry.send("main", "Hello").await?.wait().await?;
println!("{}", reply.response);
```
`default_dir(app)` is `$XDG_RUNTIME_DIR/` when set, otherwise `/-`. Keep the server value alive for as long as the host serves; dropping it stops serving and removes the socket and token file. A turn is cancellable from outside only if the registry knows its `TurnHandle`. Send through `registry.send`, or report a send made elsewhere with `registry.track(&session.id(), &sent)`. ## Drive it from a second process `ControlClient` is the attached side. Each call opens one connection.
```rust
use everruns::control::{ControlClient, default_dir};
let client = ControlClient::new(default_dir("myagent"));
for session in client.list().await? {
println!("{} {} {:?}", session.label, session.id, session.active_turns);
}
// Start a turn (or steer the running one) and wait for its reply.
let (_, turn) = client.send_and_wait("main", "Summarize the plan").await?;
println!("{}", turn.response);
// Stop whatever is running.
client.cancel("main").await?;
// Watch output; `Some(0)` replays the conversation first, `None` starts live.
let mut stream = client.attach("main", Some(0)).await?;
while let Some(event) = stream.next().await? {
if let Some(text) = event.render() {
print!("{text}");
}
}
```
Sessions are addressed by id or by a unique label. `attach` streams user messages from every sender, assistant text, tool names and turn boundaries. Tool arguments and results are not forwarded. Each event carries the durable `sequence` when it has one, so a client that disconnected or saw a `lagged` event re-attaches with `Some(last_sequence)` and misses nothing durable. The [`attached-cli` example](https://github.com/everruns/everruns/tree/main/examples/attached-cli) is a complete host and attached CLI in about 200 lines. Copy it to start. ## Who may attach Access is local and has two independent gates: * **Same OS user.** The control directory is created with mode 0700 and the socket with 0600. The server reads each connection’s kernel peer credentials and refuses a different uid before reading a byte. The client checks the directory and token file belong to it and are private, and that the process behind the socket runs as the same user, before it sends the token or any input. A directory that already exists with group or other permissions, or another owner, is refused by both ends. * **Bearer token.** 32 random bytes, hex encoded, in the 0600 `token` file next to the socket. Every connection’s first line carries it, and the server compares it in constant time. There is no way to grant another user access. A host that needs remote or multi-user access should serve sessions over an authenticated protocol such as [AG-UI](https://everruns.com/docs/framework/ag-ui/) or [serve](https://everruns.com/docs/framework/serve/) instead. ## Token lifecycle | Event | Effect | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ControlServer::start` | Issues a token and writes the token file. | | Every `token_ttl` (default 15 minutes) | Issues a new token. The old one stops opening connections at once; the client re-reads the file once when a token is refused, so rotation is invisible to it. Open attach streams continue. | | `rotate_token()` | Same as a scheduled rotation, on demand. Also re-opens a revoked server. | | `revoke()` | Deletes the token and the file. Every open attach stream and waiting send ends with `revoked`, and new connections are refused until `rotate_token()`. | | Dropping the server | Stops serving, ends open streams with `shutdown`, and removes the socket and token file. |
```rust
use std::time::Duration;
use everruns::control::{ControlRegistry, ControlServer};
let server = ControlServer::builder("/run/user/1000/myagent")
.token_ttl(Some(Duration::from_secs(60)))
.max_connections(8)
.start(registry)?;
// Someone copied the token somewhere it should not be:
server.revoke();
server.rotate_token()?;
```
## Cancel semantics `cancel` targets every unfinished turn the registry tracks for the session: the running turn and any message admitted behind it that has not started. It returns their identities, the same values `list` reports in `active_turns`. An identity is fixed when the message is sent: the turn id when `send` returned one, otherwise the accepted message’s id (the default Engine admits durably first, so expect message ids there). It does not change once the turn is assigned. Each target behaves like `TurnHandle::cancel`: * The turn stops at its next await point; running tool work is dropped and the session records `turn.cancelled`. * Messages that steered into that turn end with it, and messages still queued behind it are dropped rather than run afterwards. * The runtime records a short user message noting the cancellation (“User requested to cancel the work.”), so the next turn knows the work was stopped. Attached streams show it like any other user message. * The session stays usable; the next `send` starts a new turn. * A session with nothing tracked still running answers `no_active_turn`, including when the turn finished while the cancel was on its way. Disconnecting never cancels anything. Dropping an attach stream, a client that exits while waiting on `send_and_wait`, a revocation or a server shutdown all leave the session’s turns running. A client that leaves frees its connection slot at once, so abandoned waits cannot starve `cancel`. ## Protocol One newline-delimited JSON request per connection, answered by one or more JSON lines: `{"token": "...", "command": {"op": "list"}}`, with `send`, `cancel` and `attach` taking a `session` field. Errors are `{"type": "error", "code": "...", "message": "..."}`, with codes `unauthorized`, `bad_request`, `not_found`, `ambiguous`, `no_active_turn`, `session_closed`, `busy` and `failed`. Requests are capped at 256 KiB and must arrive within 10 seconds; connections beyond `max_connections` get `busy`. The client caps response lines at the same size and expects the first line within 30 seconds; later lines (attach events, a waited-for turn) have no deadline. Only one server may use a directory at a time: it holds an exclusive lock on `/lock` while it runs. The serde types in `everruns::control` are the wire format, so a client in another language can speak it.
---
# Session work and wakes
> Run immediate and scheduled background work with explicit delivery and restart semantics.
Source:
`everruns::work` lets an application request and handle work owned by a session without importing runtime registries, platform stores, or task-kind constants. The application chooses its own work kinds and JSON payloads.
```rust
use std::time::Duration;
use everruns::work::{TaskOutcome, TaskRequest, WakePolicy, WorkQueue};
use serde_json::json;
let queue = WorkQueue::in_memory();
let work = queue.for_session("session_123");
work.submit(
TaskRequest::new("thumbnail", json!({ "image": "cover.png" }))
.idempotency_key("thumbnail:cover.png")
.wake_policy(WakePolicy::OnCompletion),
).await?;
for delivery in queue.claim_due(Duration::from_secs(30), 16).await? {
// Route on delivery.task.kind and check cancellation before side effects.
queue.finish(
&delivery,
TaskOutcome::success(json!({ "path": "cover-thumb.png" })),
).await?;
}
```
The runnable version is [`session_work.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/session_work.rs). ## Contract * **Ownership:** every task and wake has one opaque `session_id`. A `SessionWork` handle fixes that owner for all requests and reads. * **Persistence and restart:** `WorkQueue::in_memory()` is process-local and database-free. State survives replacing the queue only while the same `Arc` is retained. It does not survive a process restart. A host that needs durable recovery supplies a `WorkBackend`. * **Scheduling:** `Immediate` work is claimable now; `At(SystemTime)` work is not claimable early. Both are one-shot requests. The host owns polling; recurring calendars and schedule runners stay host concerns. There is no hidden scheduler or database in the default build. * **Delivery:** task and wake claims are leased and at least once. If a process stops before settlement or acknowledgment, the item is claimable after the lease expires. Each retry has a new token and attempt; stale attempts cannot settle newer work. * **Idempotency:** task keys and direct-wake keys each have a session-scoped namespace. Repeating the same request returns the original task or wake. Reusing a key for different input fails with `IdempotencyConflict`. Workers should also deduplicate external side effects on the stable task id or submission key. * **Cancellation:** pending work cancels immediately. Running work records cancellation intent; the worker checks the latest task snapshot, stops cooperatively, then reports `TaskOutcome::Canceled`. A task may still succeed if it passes its safe cancellation point first. * **Wakes:** applications can request an immediate wake directly. A task can also create one atomic completion wake with `WakePolicy::OnCompletion`. Wakes use the same lease/retry/acknowledgment model as tasks. Durable platform scheduling, retention, distributed polling, and multi-host coordination belong in the host’s `WorkBackend`; they are not enabled by the offline Framework default. The in-memory backend retains accepted payloads until it is dropped and does not enforce admission quotas, so hosted providers must apply tenant authorization, payload limits, quotas, and retention at their own boundary.
---
# Batch Model Requests
> Submit independent model completions through native provider batch APIs, persist a credential-free ticket, and collect results in another process.
Source:
Use `everruns::batch::Batch` for offline classification, extraction, or evaluation where latency can be hours. A batch sends independent model requests to a provider’s batch API. It does not start an agent, stream tokens, run tools, or execute side effects. This is an opt-in **alpha** API; ordinary completions and sessions keep their existing behavior. ## Supported providers and hosts | Surface | Support | | --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Anthropic direct Messages API | Native Rust, `anthropic` Cargo feature; Message Batches submit, poll, collect, cancel | | Anthropic-compatible proxy | Explicit versioned `base_url`; must implement the Message Batches API, not just Messages | | OpenAI direct Responses API | Native Rust, `openai` feature; Files upload and Batches submit, poll, collect, cancel | | OpenAI direct Chat Completions API | Same lifecycle, using `OpenAI::from_env()?.api(OpenAIApi::ChatCompletions)`; tickets are separate from Responses tickets | | Gemini Developer API | Native Rust, `gemini` feature; inline batch submit, poll, collect, cancel; Files and Vertex AI modes are unsupported | | Mistral La Plateforme | Native Rust, `mistral` feature; inline request submission, polling, inline/file result collection and cancellation | | Groq Chat Completions | Native Rust, `groq` feature; Files/Batches submit, poll, collect and cancel; audio batches are outside this completion API | | Fireworks | Native Rust, `fireworks` feature; explicit account and artifact policy; dataset upload, native batch job submission, poll and collect. Cancellation returns `UnsupportedOperation` | | Custom endpoints | Must implement the complete corresponding native batch API; a chat-compatible endpoint alone is insufficient. Azure batch configurations are not qualified here | | OpenRouter, Bedrock, ChatGPT and remaining built-in drivers | Unsupported; returns `BatchError::Unsupported`, with no synchronous fallback | | Legacy endpoint-bound `into_boxed_driver()` / `DriverRegistry::create_chat_driver()` adapters | No batch hook: retain the configured `Provider` for ticket endpoint binding instead. Registry request-option/credential wrappers also fail closed | | Custom providers | Implement the optional `batch::BatchDriver` SPI and expose it through `ChatDriver::batch_driver` | | Native applications with memory, files, SQLite or another application database | Supported direct API; application owns ticket persistence and scheduling | | Engine memory/SQLite hosting, `serve`, Platform workers and eval service | No batch scheduling or ticket persistence integration; an embedding native application can separately use the direct API | | Wasm, Cloudflare Durable Objects and celld | No built-in batch driver in this version | The batch API is currently on `main` and is not included in the published `0.49.0` crate. Until a crate release includes it, use `main` with the provider features you need. Commit `Cargo.lock` to retain the resolved Git revision for reproducible builds:
```toml
[dependencies]
everruns = { git = "https://github.com/everruns/everruns", branch = "main", features = ["anthropic"] }
serde_json = "1"
tokio = { version = "1", features = ["macros", "rt"] }
```
For Core hosts using the stored-message conversion example, source `everruns-core` from the same branch and enable `host`, so both crates share the message contract:
```toml
[dependencies.everruns-core]
git = "https://github.com/everruns/everruns"
branch = "main"
default-features = false
features = ["host"]
```
## Submit now, collect later The batch owns one configured provider. Supply typed request values directly, or describe each request with a completion builder. Both paths share validation, submission and the same resumable tickets. ### Pass message lists directly `submit_requests` accepts a `Vec`. Each value contains its ID, existing messages, and `LlmCallConfig`, so code that already has a conversation can submit it without rebuilding it through `.system()` and `.user()` calls. The client stays available for polling, collection and cancellation.
```rust
use everruns::{
batch::{Batch, BatchRequest},
llm::{Message, MessageRole},
providers::anthropic::Anthropic,
LlmCallConfig,
};
let mut config = LlmCallConfig::new("claude-sonnet-5");
config.max_tokens = Some(64);
let requests = vec![
BatchRequest {
id: "review_001".into(),
messages: vec![
Message::text(MessageRole::System, "Classify as positive, negative, or neutral."),
Message::text(MessageRole::User, "Delivery arrived early."),
],
config: config.clone(),
},
BatchRequest {
id: "review_002".into(),
messages: vec![
Message::text(MessageRole::System, "Classify as positive, negative, or neutral."),
Message::text(MessageRole::User, "The replacement does not turn on."),
],
config,
},
];
let client = Batch::new(Anthropic::from_env()?, "tenant-42/anthropic-account-7");
let ticket = client.submit_requests(requests).await?;
let saved_ticket = serde_json::to_vec(&ticket)?;
```
`everruns::llm::Message` is the provider-message contract used by core. Stored `everruns_core::RuntimeMessage` records are a different type: core hosts reuse the existing [message conversion helpers](https://github.com/everruns/everruns/blob/main/crates/core/src/llm_conversions.rs). For a text-only core conversation:
```rust
use everruns::{batch::BatchRequest, LlmCallConfig};
use everruns_core::{RuntimeMessage, llm_conversions::llm_message_from_message};
let conversation = vec![
RuntimeMessage::system("Classify the review."),
RuntimeMessage::user("Delivery arrived early."),
];
let request = BatchRequest {
id: "review_001".into(),
messages: conversation.iter().map(llm_message_from_message).collect(),
config: LlmCallConfig::new("claude-sonnet-5"),
};
// Pass vec![request] to client.submit_requests(...).
```
For stored images or files, core hosts use the attachment-aware conversion helper with their resolved attachments. This direct API does not load session history or resolve stored file references itself. All request IDs must be unique, and each request needs messages. The entire list is validated before submission. Mixing an explicit list with buffered `.request(...)` builder entries is rejected to avoid silently discarding work. ### Completion builder Build each completion with a **bare model ID**, then give it a unique request ID. A completion carrying its own provider is rejected so a batch cannot silently route it to a different account.
```rust
use everruns::{Model, batch::Batch, providers::anthropic::Anthropic};
let model = Model::from("claude-sonnet-5");
let batch = Batch::new(Anthropic::from_env()?, "tenant-42/anthropic-account-7")
.request("review_001", model.completion()
.system("Classify as positive, negative, or neutral. Return one word.")
.user("Delivery arrived early.").max_tokens(64))?
.request("review_002", model.completion()
.system("Classify as positive, negative, or neutral. Return one word.")
.user("The replacement still does not turn on.").max_tokens(64))?;
let ticket = batch.submit().await?;
// Save immediately in access-controlled durable application storage.
let saved_ticket = serde_json::to_vec(&ticket)?;
```
### Resume and collect In another process, reconstruct the provider from current credentials and use exactly the same application scope and endpoint. Credential rotation is allowed within that provider account. The ticket contains a provider job ID, request IDs, scope, format version and endpoint fingerprint. It contains no credentials, prompts, result URLs, or results.
```rust
use everruns::{batch::{Batch, BatchStatus, BatchTicket}, providers::anthropic::Anthropic};
let ticket: BatchTicket = serde_json::from_slice(saved_ticket)?;
let client = Batch::new(Anthropic::from_env()?, "tenant-42/anthropic-account-7");
if client.poll(&ticket).await? == BatchStatus::Ended {
for item in client.collect(&ticket).await? {
match item.result {
Ok(response) => println!("{}: {}", item.id, response.text),
Err(error) => eprintln!("{}: {error:?}", item.id),
}
}
}
// Otherwise save the ticket and schedule another poll, usually ≥60 seconds later.
```
Collection checks that the job ended and maps results by request ID, returning them in submission order. Unknown, duplicate or missing IDs fail the whole collection. A request can succeed while another expires, is cancelled, or fails. Successful responses retain text, reasoning, function tool calls, finish reasons and token usage. Tool calls are data; nothing executes them. Validate model output before consuming it. Anthropic’s existing driver does not enforce `Completion::response_format`; batch submission rejects that option too. ## Choose a provider through the same API `Provider::supports_batch()` checks the driver’s local capability. It does not probe model eligibility, account permissions, quotas or a custom server’s API. Unsupported drivers retain the default `ChatDriver::batch_driver() == None`. Custom drivers implement the same optional SPI; Framework does not dispatch on provider names. With the `openai` feature:
```rust
use everruns::{Model, batch::Batch, providers::openai::OpenAI};
let provider: everruns::Provider = OpenAI::from_env()?.into();
assert!(provider.supports_batch());
let client = Batch::new(provider, "tenant-42/openai-account-7");
let ticket = client.request("review_001", Model::from("gpt-6.2-sol")
.completion().user("Classify: delivery arrived early.").max_tokens(64))?
.submit().await?;
```
For Gemini, enable `gemini`, use `Gemini::from_env()` from `everruns::providers::gemini`, select a Gemini model and its own tenant/account scope. `submit_requests(Vec)`, tickets and lifecycle operations stay the same. Requests in each OpenAI, Gemini, Mistral, Groq or Fireworks job must use one model. To select OpenAI Chat Completions explicitly, configure the connection with its typed API selector:
```rust
use everruns::{OpenAI, OpenAIApi, Provider};
let provider: Provider = OpenAI::from_env()?
.api(OpenAIApi::ChatCompletions)
.into();
assert!(provider.supports_batch());
```
### Mistral, Groq and Fireworks Mistral and Groq use the same lifecycle with their own features and provider values: `Mistral::from_env()` (`MISTRAL_API_KEY`) and `Groq::from_env()` (`GROQ_API_KEY`). Groq is also a first-class interactive Chat Completions driver, with model discovery and the shared message/tool codec. Only batch-compatible models are eligible; driver support does not imply account or model access. Fireworks separates its inference and management paths and requires an account. Configure exact trusted storage origins before submitting; obtain your account’s artifact origin from its storage configuration or a trusted existing dataset. For example, an account whose signed downloads use `storage.googleapis.com`:
```rust
use everruns::{batch::Batch, providers::fireworks::{Fireworks, FireworksBatchConfig}};
let config = FireworksBatchConfig::new("my-fireworks-account")?
.artifact_origin("https://storage.googleapis.com")?;
let client = Batch::new(Fireworks::from_env()?.batch(config), "tenant/fireworks-account");
```
Enable `fireworks`. The key comes from `FIREWORKS_API_KEY`; account and artifact policy stay explicit. Add each exact storage origin your account uses. Downloads carry only the signed URL, never provider authentication or configured headers; redirects and untrusted origins fail closed. A custom endpoint must serve both management `/v1` and inference `/inference/v1` paths on the configured origin. Changing the account requires a separate client even if its tenant scope was accidentally reused. Fireworks’ published API offers job deletion rather than a cancellation contract. `cancel` returns `BatchError::UnsupportedOperation`; it never deletes jobs or datasets. Manage deletion explicitly in the provider console after preserving needed results. Failed/cancelled/expired Fireworks jobs return `JobFailed`; this version does not collect partial datasets from those job states. ### Provider limits and terminal results | Driver | Submission boundary | Collection and cancellation | | ----------------------------------- | ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Anthropic | 100,000 requests, 256 MiB | Native result for every request; completed work remains collectable after cancellation | | OpenAI Responses / Chat Completions | 50,000 requests, 200 MB including this implementation’s small multipart envelope; one model | Merges output and error JSONL by `custom_id`. Cancelled/expired jobs may omit unprocessed requests: ticket IDs and validated provider counts identify these, preserving completed results | | Mistral | Inline submission only, at most 10,000 requests and 20 MB encoded body; one model, 24-hour job timeout | Native inline results or output/error files, correlated by `custom_id`; cancelled/expired omissions require verified counts and original ticket IDs | | Groq | 50,000 requests, 200 MB including the multipart envelope; one model, 24-hour processing window | Native output/error files and verified count mapping preserve completed requests on cancellation or expiry | | Fireworks | At most 50,000 requests and 100 MB including the multipart envelope; one model; provider default job window | Downloads up to 16 signed artifacts with an aggregate configured response-byte budget. Missing/duplicate/unknown request IDs fail closed | | Gemini Developer API | Inline only, 20 MB encoded body, at most 100,000 requests; one model | Maps `metadata.key`, including inline per-item errors. A failed/cancelled job without inline outputs returns `BatchError::JobFailed`; missing outputs are never invented | Account/model quotas can impose lower limits. Split jobs explicitly. Gemini’s Files mode and Vertex storage outputs are outside this implementation. OpenAI global validation failures also return `JobFailed`, separate from a list of per-item failures. Missing successful/error files or inconsistent counts fail collection rather than being relabelled as cancellation. OpenAI and Groq submission upload a `purpose=batch` file before creating a job. These are separate effects. An uncertain upload or job creation returns `SubmissionUncertain`; a definite rejection can still leave an uploaded file. The library does not automatically delete input/output files, cancel shared jobs, or follow provider-supplied URLs. Reconcile files and jobs in your own account, apply your retention policy and collect results before provider retention expires. Fireworks dataset creation, upload and job creation are separate external effects too. No stage is automatically replayed or deleted, including staging artifacts left after rejection. Persist intent before submitting and reconcile ambiguous responses through provider job/dataset listings before trying again. Mistral inline submission has one create operation; an ambiguous response still needs reconciliation. Provider retention and account quotas apply to all artifacts. See the current [Mistral batch lifecycle](https://docs.mistral.ai/studio/batch-processing), [Groq batch lifecycle](https://console.groq.com/docs/batch), [Fireworks batch lifecycle](https://docs.fireworks.ai/guides/batch-inference), [OpenAI batch lifecycle](https://developers.openai.com/api/docs/guides/batch) and [Gemini batch lifecycle](https://ai.google.dev/gemini-api/docs/batch-api). OpenAI batch function schemas explicitly use non-strict arguments, preserving optional properties without retaining original schemas in tickets. Structured text output remains available through OpenAI’s response format. Gemini batch rejects response formats, metadata, speed and verbosity options this encoder does not support. Stateful continuations, tool search, provider execution options and per-item credential/header overrides are rejected before submission. Function tool calls are returned as data, with no provider-hosted tool promotion. ## Runnable examples Run these examples from a repository checkout containing this change. Both examples call the real provider and incur its normal batch charges. Set the provider credential locally; do not put it in a ticket or command argument. **Classification across processes:** submit three reviews, then inspect or cancel the job and collect validated JSON classifications. Every command uses the same state directory and `BATCH_SCOPE`. `submit` requires a new directory, writes `intent.json` before the provider call, and saves `ticket.json` after acceptance. `collect` writes `results.json` without overwriting an earlier collection.
```bash
export BATCH_SCOPE=tenant-42/anthropic-account-7
cargo run -p everruns --features anthropic --example batch_classification -- submit /tmp/reviews-001
cargo run -p everruns --features anthropic --example batch_classification -- poll /tmp/reviews-001
# Run when the poll says Ended:
cargo run -p everruns --features anthropic --example batch_classification -- collect /tmp/reviews-001
# Or request cancellation, then poll and collect partial results:
cargo run -p everruns --features anthropic --example batch_classification -- cancel /tmp/reviews-001
```
Select a provider with `BATCH_PROVIDER` and enable its feature. Keep that selection, `BATCH_SCOPE` and state directory identical on later invocations:
```bash
# Set OPENAI_API_KEY locally first; each command can incur provider charges.
export BATCH_PROVIDER=openai
export BATCH_SCOPE=tenant-42/openai-account-7
cargo run -p everruns --features openai --example batch_classification -- submit /tmp/openai-reviews-001
cargo run -p everruns --features openai --example batch_classification -- poll /tmp/openai-reviews-001
cargo run -p everruns --features openai --example batch_classification -- collect /tmp/openai-reviews-001
# Gemini uses GEMINI_API_KEY (or GOOGLE_API_KEY).
BATCH_PROVIDER=gemini cargo run -p everruns --features gemini --example batch_eval
# Mistral and Groq use MISTRAL_API_KEY and GROQ_API_KEY respectively.
BATCH_PROVIDER=mistral cargo run -p everruns --features mistral --example batch_eval
BATCH_PROVIDER=groq cargo run -p everruns --features groq --example batch_eval
# Fireworks also needs explicit account and trusted storage origin.
export FIREWORKS_ACCOUNT_ID=my-fireworks-account
export FIREWORKS_ARTIFACT_ORIGIN=https://storage.googleapis.com
BATCH_PROVIDER=fireworks cargo run -p everruns --features fireworks --example batch_eval
```
The [classification example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_classification.rs) defaults to `claude-sonnet-5`, `gpt-6.2-sol` or `gemini-4-argon` for the selected provider; set `BATCH_MODEL` on submission to choose another eligible model. Protect the state directory: intents and results contain user content even though tickets do not contain prompts. **Batch eval:** a single process submits an explicit list of message-based requests for three capital-city cases, polls every 60 seconds, and scores exact-match accuracy, correlating each answer by ID. This example keeps its ticket only in memory; restarting submits a new job.
```bash
cargo run -p everruns --features anthropic --example batch_eval
```
See the [eval source](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_eval.rs) and the [example catalog](https://everruns.com/docs/framework/examples/). ## Retries, recovery and cancellation There are no automatic retries, resubmissions, background pollers, or cross-client batch pools. `BatchError::retryable()` identifies transient failures for retrying **that operation**. A failed poll or collection can use the same ticket again. For individual failed or expired items, select the work deliberately and create a new batch. Retain your source inputs separately; the ticket cannot reconstruct a prompt. **Uncertain submission is a separate outcome.** If a submit times out, loses its response, receives a server error, or returns an invalid acceptance response, it returns `SubmissionUncertain`. The provider may already be processing it. Do not automatically call `submit` again. Reconcile the intent and request IDs with the provider console/API first. If the process crashes between acceptance and saving the ticket, treat the surviving intent the same way. This API does not guarantee idempotent creation or exactly-once external effects; it cannot atomically commit a provider job and your database record. After reconciliation identifies the accepted job, `client.recover(job_id, original_request_ids).await?` checks access and returns a ticket without creating work. Supply IDs from the durable intent, then persist this ticket. The classification example exposes the same recovery path:
```bash
# Only after identifying this intent's accepted job in the provider console/API:
cargo run -p everruns --features anthropic --example batch_classification -- recover /tmp/reviews-001 msgbatch_YOUR_RECONCILED_JOB
```
`client.cancel(&ticket).await?` requests cancellation of the **whole job**. Cancellation is asynchronous: keep the ticket, poll until `Ended`, and collect available successes together with cancelled items, or handle the provider job-level failure described above. Completed or already running requests may still be billed. Dropping a client, ticket, or pending future does not cancel an external job. Retrying cancellation with the same ticket is safe; already ended jobs return `Ended`. ## Limits and isolation * Anthropic accepts at most 100,000 requests and 256 MiB per job. This driver rejects oversized submissions; split them explicitly. IDs are 1–64 ASCII letters, digits, underscores or hyphens. `max_tokens` must be positive. * Jobs can take up to 24 hours. Results are retained by Anthropic for 29 days from batch creation; collect them into your own storage before retention expires. Polling is not a streaming or interactive response path. * The client defaults to a 60-second timeout per HTTP request and a 256 MiB cap per response. Configure `Batch::timeout` and `max_response_bytes` for your workload. Streaming first-token/response limits, request-body capture, per-item headers and provider execution options are rejected by this batch driver. The timeout controls HTTP work, not the offline job’s lifetime. * Every client uses only its own provider credentials and endpoint. It rejects mismatched ticket scope, provider, endpoint, or format before network access. Choose a distinct stable scope for every tenant **and provider account**; the library cannot infer the account behind an API key. A scope is an application guard, not provider authorization. Never accept tenant identity or a ticket from an unauthenticated caller as proof of access. * Tickets are serialized routing data, not signed capabilities. Authorize their storage and use through your application. Do not pool work across tenants. Provider error bodies and endpoint URLs are excluded from batch errors. Result URLs are ignored except for Fireworks signed artifacts: those are fetched only from explicitly trusted origins with no provider credentials. Job operations stay on the configured provider origin. * Usage reports token counts, not a locally estimated bill. Consult the provider’s [current batch pricing and retention rules](https://platform.claude.com/docs/en/build-with-claude/batch-processing). For direct interactive calls, see [Direct Model Calls](https://everruns.com/docs/framework/direct-model-calls/). For sessions and their hosting guarantees, see [Hosting](https://everruns.com/docs/framework/hosting/).
---
# Portable and hosted capabilities
> Understand which capabilities run in the Framework and which require the Everruns Platform host.
Source:
The Framework advertises only capabilities that can run through its portable, in-process host contract. A capability reference remains an open value, so an application may retain configuration for an ID supplied by another host or plugin, but an unknown or hosted-only ID contributes no prompt, tools, or behavior in the default Framework runtime. ## Portable Framework capabilities Portable built-ins use services available from the Framework runtime or from an explicit application integration. Examples include files, session storage, current time, compaction, tool search, skills, and application-authored capabilities. Register application behavior with `#[everruns::tool]` or `everruns::capability` rather than depending on product internals. ## Hosted Platform capabilities Knowledge Bases and Knowledge Indexes, Memories, subagents and agent handoff, background/session tasks and schedules, user hooks, model scouting, OpenRouter workspace management, citations, and platform-management tools need hosted persistence or orchestration. Their implementations and narrow service contracts live in `everruns-capabilities`; the server and worker product presets register them explicitly. This boundary prevents the public Framework from promising tools whose stores, tenant scope, workers, or authorization services are absent. It does not change persisted capability IDs or JSON configuration. A specialized low-level host can depend on `everruns-capabilities`, install the required services, and select the hosted registry deliberately. Moving an agent to the Platform keeps its capability IDs and configuration; see [Moving to Platform or Cloud](https://everruns.com/docs/framework/moving-to-platform/). For application-owned behavior, continue with [advanced capabilities](https://everruns.com/docs/framework/advanced-capabilities/). For low-level host composition, see [custom backends](https://everruns.com/docs/framework/custom-backends/).
---
# Capability integrations
> Select filesystem, shell, web, Lua, and MCP implementation boundaries without pulling them into the Everruns kernel.
Source:
The Framework separates capability contracts from environment-backed implementations. `everruns-core` defines capability, tool, filesystem, egress, and MCP invocation contracts; focused crates own code that touches an interpreter, network transport, local process, or session filesystem. This keeps a custom host’s dependency and trust boundaries visible in `Cargo.toml`. It also prevents a core registry from silently granting an execution or network surface. ## Framework features | `everruns` feature | Default | Implementation | Effect boundary | | ------------------ | ------: | ---------------------------------------------- | ---------------------------------------------------------------------------------------- | | `filesystem` | Yes | `everruns-integrations` (`filesystem` feature) | Host-provided, session-scoped filesystem only | | `bashkit` | No | `everruns-integrations` (`bashkit` feature) | Sandboxed shell; HTTP remains capability-config and egress-policy gated | | `web-fetch` | No | `everruns-integrations` (`web-fetch` feature) | FetchKit requests through the host egress contract | | `lua` | No | `everruns-integrations` (`lua` feature) | Vendored Lua 5.4 sandbox; also requires `FEATURE_LUA=prod` at runtime | | `mcp` | No | `everruns-core` (`mcp` feature) | Remote HTTP MCP through the host egress contract | | `mcp-stdio` | No | `everruns-core` (`mcp` feature) | Adds local-process MCP servers and implies `mcp` | | `host-shell` | No | `everruns-core` (`host` feature) | `bash` tool over real host processes, bounded by a kernel policy; implies `host-compute` | \| `images` | No | `everruns-integrations` (`image-generation` feature) | [Image generation](https://everruns.com/docs/capabilities/image-generation/) (stills, GIFs, SVGs) through the host egress contract; keys from `OPENAI_API_KEY` and friends | | `duckduckgo` | No | `everruns-integrations` (`duckduckgo` feature) | DuckDuckGo web search through the host egress contract | | `a2a` | No | `everruns-capabilities` | Outbound A2A delegation to remote agents; implies `local` | | `otel` | No | `everruns-core` (`host` feature) | OpenTelemetry exporter; see [Observability](https://everruns.com/docs/observability/) | | `braintrust` | No | `everruns-core` (`host` feature) | Braintrust exporter; see [Observability](https://everruns.com/docs/observability/) | The default is offline: the filesystem capability can only use the session-filesystem implementation supplied by the host. Shell, web, Lua, MCP, and local-process transports require explicit features.
```toml
[dependencies]
everruns = { version = "0.34", features = ["bashkit", "web-fetch"] }
```
Enabling an implementation does not activate it on every agent. Add the matching capability reference to the agent, and retain the documented role, network-access, and runtime feature gates. In particular, Bashkit, web fetch, and Lua remain high-risk capabilities in the hosted product. ## Provider integrations Integration packages can expose typed values through `IntoCapability`. Brave Search supports the ordinary Framework builder:
```rust
use everruns::{Agent, OpenAI};
use everruns_integrations::brave_search::BraveSearch;
let agent = Agent::builder()
.instructions("Search and cite primary sources.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.capability(BraveSearch::from_env()?)
.build()?;
```
Depend on `everruns-integrations` with `default-features = false` and the `brave-search` feature. Add `brave-search-integration` to opt into Platform connector registration; leave it off for a Framework-only dependency. The Framework adapter reads `BRAVE_SEARCH_API_KEY` at construction; `BraveSearch::new` accepts an explicit application-owned key. Keys are retained privately by the client, never placed in capability JSON. Hosted execution continues to resolve connections and session secrets at tool execution time. Both paths use the same search schema and operation. Framework calls use the application’s direct HTTP client. ## Advanced host composition Advanced embedders select integrations on `everruns` and build the runtime registry through `everruns::batteries::runtime_capability_registry()`:
```toml
[dependencies]
everruns-core = "0.34"
everruns = { version = "0.34", default-features = false, features = ["filesystem", "web-fetch"] }
```
```rust
let registry = everruns::batteries::runtime_capability_registry();
let egress = everruns::batteries::runtime_egress_service();
assert!(registry.has("session_file_system"));
assert!(registry.has("web_fetch"));
assert!(!registry.has("bashkit_shell"));
```
If the host starts from a caller-owned registry, preserve it and apply the same feature-selected integrations with `everruns::batteries::compose_runtime_capability_registry(registry)`. Hosted server and worker composition uses `everruns_capabilities::capabilities::hosted_capability_registry_for_grade` with the platform’s `environment-capabilities` feature. That preset preserves the hosted catalog while keeping the implementations outside core. Depend directly on a focused crate when you need its public implementation types. The former core paths move as follows: | Former public path | New public path | | ------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | | `everruns_core::FileSystemCapability` and filesystem tools | `everruns_integrations_filesystem::*` | | `everruns_core::BashkitShellCapability`, `BashTool`, and adapter | `everruns_integrations_bashkit::*` | | `everruns_core::WebFetchCapability` and `WebFetchTool` | `everruns_integrations_web_fetch::*` | | `everruns_core::LuaCapability` and `LuaCodeModeCapability` | `everruns_integrations_lua::*` | | `everruns_core::McpCapability` and MCP capability-ID helpers | `everruns_core::mcp::*` | | `everruns_core::DirectEgressService` | `everruns_core::host::DirectEgressService` with `direct-egress` | | `everruns_core::SystemEmailConfig` and Resend types | `everruns_capabilities::*` | | `everruns_core::ModelScoutCapability` and `OpenRouterWorkspaceCapability` | `everruns_integrations_openrouter::*` | | `everruns_core::OpenRouterServerToolsCapability` | `everruns_integrations_openrouter::OpenRouterServerToolsCapability` | | `everruns_core::{HumanIntentCapability, InfinityContextCapability, SkillsCapability, AttachSkillCapability, ToolApprovalCapability}` | `everruns_core::builtins::*` | | `everruns_core::{OpenUiCapability, A2UiCapability}` | `everruns_core::builtins::*` with `ui-capabilities` | | `everruns_core::skill::ProcessCommandExecutor` | `everruns_core::host::ProcessCommandExecutor` with the host `process` feature | Continue with [Configure and author capabilities](https://everruns.com/docs/framework/advanced-capabilities/) for agent-level activation or [Custom backends](https://everruns.com/docs/framework/custom-backends/) for host-level storage and orchestration. ## GitHub The `github` feature adds `everruns::github::GitHub`: native repository and PR tools with application-owned credentials, read-only by default. See [GitHub tools](https://everruns.com/docs/framework/github/) for token providers, write options, and the runnable PR review example. For application-bound Slack, Cursor, Browserless, E2B, SQL and knowledge services, see [Application services](https://everruns.com/docs/framework/service-integrations/). These typed bindings supply the native tools with their real credential/storage dependencies.
---
# Code Mode with MCP
> Let an agent call a large MCP server's tools from Bashkit scripts, so their schemas stay out of the prompt and many calls take one step.
Source:
Attach an MCP server with hundreds or thousands of tools natively and every model request carries every tool schema. Code mode moves those tools into the sandboxed [Bashkit shell](https://everruns.com/docs/capabilities/bashkit-shell/) instead. The model sees one `bash` tool, finds operations with `tools search`, and writes scripts that call them, join the results with `jq`, and print only the answer. The intermediate payloads never enter the conversation.  The demo is the [Code Mode MCP Agent](https://github.com/everruns/everruns/tree/main/examples/code-mode-mcp-agent) example running `gpt-6.1-sol` against a synthetic MCP server with 1,000 tools (355 KiB of schemas). Each `❯ bash` block is a script the model wrote; each `↳ tools acme ...` line under it is an MCP tool that script called, with its input. The model’s tool list stayed at four tools and about 4 KiB, its largest prompt was about 3,500 tokens, and the answer matched totals computed from the server’s data. ## Declare it Code mode is three lines on the agent builder: the shell, the [Shell Code Mode](https://everruns.com/docs/capabilities/shell-code-mode/) capability in `cli` mode, and the MCP server. Enable the `bashkit` and `mcp-stdio` (or `mcp`) features.
```toml
[dependencies]
everruns = { version = "*", features = ["bashkit", "mcp-stdio", "openai"] }
serde_json = "1"
```
```rust
use everruns::{Agent, BashkitShell, CapabilityRef, Engine, McpServer, OpenAI};
use serde_json::json;
let agent = Agent::builder()
.name("acme-analyst")
.instructions(
"The company APIs are available only in the shell, through the `tools` \
command: find operations with `tools search`, read one with `--help`, \
then do the work in one script and print only what you need.",
)
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.capability(BashkitShell::new())
.capability(
CapabilityRef::new("shell_code_mode")
.config(json!({"mode": "cli", "rate_unlabeled_tools": false})),
)
.mcp_server(McpServer::stdio("acme", "acme-mcp-server"))
.build()?;
let session = Engine::new().create(agent);
let turn = session
.run("Which EU customers have overdue invoices? Give the top three by amount.")
.await?;
println!("{}", turn.response);
```
`mode: "cli"` removes every eligible tool from the model’s tool list and makes it a `tools ` command in the shell. `mode: "hybrid"` keeps the native route as well. `rate_unlabeled_tools: false` skips the decision service for tools that declare no read-only or destructive hint; leave it on when your server does not label its tools. Approvals still apply: a call that needs approval stops the script and asks. See [Shell Code Mode](https://everruns.com/docs/capabilities/shell-code-mode/) for every option, and [Large MCP catalogs](https://everruns.com/docs/features/large-mcp-catalogs/) for the limits on catalog size and discovery. ## What the model writes In a run of the example, after `tools search invoices customers`, the two operations’ `--help` and one script that sampled the data, the model wrote this script. It is copied unedited from that run; the two MCP calls below it ran inside the shell:
```bash
set -euo pipefail
fetch_all() {
local op="$1" base="$2" cursor='' page args all='[]'
while :; do
args=$(jq -cn --argjson base "$base" --arg cursor "$cursor" '$base + {limit:500} + (if $cursor == "" then {} else {cursor:$cursor} end)')
page=$(tools acme "$op" "$args")
all=$(jq -cn --argjson all "$all" --argjson page "$page" '$all + $page.items')
cursor=$(jq -r '.next_cursor // empty' <<< "$page")
[[ -n "$cursor" ]] || break
done
printf '%s' "$all"
}
customers=$(fetch_all crm-list-customers '{"region":"EU"}')
invoices=$(fetch_all billing-list-invoices '{"status":"overdue"}')
jq -n --argjson customers "$customers" --argjson invoices "$invoices" '[ $customers[] as $c | [$invoices[] | select(.customer_id == $c.id)] as $overdue | select(($overdue|length)>0) | {customer:$c.name,total_overdue_usd:($overdue|map(.amount_usd)|add)} ] | sort_by(-.total_overdue_usd) | .[:3]'
```
```text
↳ tools acme crm-list-customers {"limit":500,"region":"EU"}
↳ tools acme billing-list-invoices {"limit":500,"status":"overdue"}
```
A `tools` call prints the tool’s JSON reply as is, so a script pipes it straight into `jq`. When an MCP server answers in text, the text is printed unchanged. ## Watch the calls Each call a script makes is a `tool.nested_call` event on the session’s event stream, with the tool name, the command as the script spelled it, a shortened input, and a status. The model’s own `bash` calls stay ordinary `tool.started` and `tool.completed` events, so a host can tell the two apart:
```rust
let mut events = session.events();
// ... run the turn ...
while let Some(event) = events.try_recv()? {
if event.event_type() == "tool.nested_call" {
let data = &event.canonical_json()["data"];
println!("{} {}", data["command"], data["input_preview"]);
}
}
```
## Run the example From a checkout of the repository:
```bash
export OPENAI_API_KEY="your-key" # or OPENROUTER_API_KEY
cargo run -p everruns-code-mode-mcp-agent
```
Add `-- --interactive` to type your own question. The MCP server is the same binary, started again with `--mcp-server`, so nothing outside the repository is needed. After the turn the example checks that no MCP tool reached the model’s tool list, that the model called none directly, that the calls ran inside scripts, and that the answer is right. A failed check exits nonzero.
---
# Coordinator agents
> One session hands focused work to threads, hears back from them, and resolves them.
Source:
A coordinator is an agent that does not do the focused work itself. The person talks to one session. That session starts a **thread** for each piece of work, relays follow-ups to it, and reports back when it finishes. Each thread keeps a checklist and ends its work with a summary. Every report wakes the coordinator with an automatic update, so nobody has to poll. This is the same model as Platform Chat’s threads, running inside your own process.
```rust
use everruns::coordination::{self, Coordination, ThreadStatus};
use everruns::{Agent, Engine, LocalConfig, Model, WorkspacePolicy};
let agent = Agent::builder()
.name("launch-coordinator")
.instructions("Start one thread per piece of work. Tell me when each is ready.")
.model(Model::simulated("On it."))
.capability(Coordination::new().max_active_threads(4))
.workspace_policy(WorkspacePolicy::read_write())
.local(LocalConfig::new("./launch/.everruns").workspace("./launch"))
.build()?;
let engine = Engine::new();
let session = engine.create(agent);
session.run("Write the announcement and draft a pricing FAQ.").await?;
for thread in coordination::threads(&session).await? {
if thread.status == ThreadStatus::ReadyForReview {
println!("{}: {}", thread.title, thread.summary.unwrap_or_default());
}
}
```
Coordination needs the `local` feature and an agent built with [`.local(...)`](https://everruns.com/docs/framework/sessions/): threads, their assignments and their checklists live in the local profile’s SQLite store. ## What the coordinator does The capability gives the coordinator five tools: | Tool | Use | | ---------------- | --------------------------------------------------------------------------------------------- | | `start_thread` | Start a thread with a title and a brief that says what done looks like. | | `message_thread` | Relay a follow-up. After a thread finished, this opens a new unit of work on the same thread. | | `list_threads` | See every thread and where it stands. | | `get_thread` | Read one thread’s assignments and recent messages. | | `resolve_thread` | Close a thread once the person accepted its work, or reopen it. | Threads never talk to each other. Every route goes through the coordinator, so routing has one owner and one record. ## What a thread does A thread is an ordinary session of the same engine, running the coordinator’s agent. While it works an assignment it gets its own tools in place of the coordinator’s: `update_checklist`, `complete_assignment`, `ask_decision`, `report_to_coordinator` and `redirect_to_coordinator`. A thread turn that ends without `complete_assignment` flags the assignment as needing attention, so the coordinator always hears back. You can open a thread like any other session with [`Engine::resume`](https://everruns.com/docs/framework/sessions/) and its id from `coordination::threads`. ## Automatic updates When a thread finishes, asks a question or reports, the coordinator gets a user-role message marked as an automatic update. If the coordinator is idle, the update starts a turn. If it is mid-turn, the update steers that turn. Your application sees the coordinator’s answer in its history and events like any other reply. ## Status `coordination::threads` returns one entry per thread, from its latest unit of work: | Status | When | | ---------------- | ------------------------------------------------------------- | | `NeedsYou` | The thread asked a question, or its work failed. | | `Working` | The thread is working. | | `ReadyForReview` | The thread finished; the coordinator has not resolved it yet. | | `Open` | The work was cancelled and the thread was left open. | | `Resolved` | The coordinator resolved the thread. | ## Limits * Threads run the coordinator’s own agent. A local profile has no agent catalog, so another worker agent is refused. * A turn your application starts on a thread directly is not settled against its assignment. Send follow-ups through the coordinator. * The default caps are 8 threads working at once and 100 in total; set them with `max_active_threads` and `max_total_threads`. ## Example [`examples/coordinator-agent`](https://github.com/everruns/everruns/tree/main/examples/coordinator-agent) runs a launch coordinator with two threads that write their drafts to a shared workspace. `--offline` runs it with a deterministic simulated model; without it, the example uses OpenAI. ## Named members with separate behavior `AgentBuilder::member` composes a named agent with its own instructions, model and capabilities. Foreground delegation waits for its result; background delegation uses the Engine’s task services. Members remain separate sessions under the same host. The [named members example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/named_members.rs) uses a real lead model and a lighter reviewer model:
```bash
cargo run -p everruns --features openai,local --example named_members
```
---
# Custom Backends
> Decide when a Framework application should cross into low-level execution-host composition.
Source:
Most applications should use `everruns::Agent`, `Model`, and `Session`. That surface deliberately hides stored harness records, platform registries, backend stores, worker phases, and durable scheduling topology. For durable execution under Framework, implement [`SessionHosting` through the public hosting SPI](https://everruns.com/docs/framework/hosting/). Engine owns the runtime and recovery; the host supplies transactional session authority, epochs and independent activation. This path needs only an `everruns` dependency. To keep a host’s own capability implementations, provider resolution, retry budgets, MCP credentials or observers while Engine runs the sessions, use [host services](https://everruns.com/docs/framework/host-services/) instead of composing a runtime. Cross into separate low-level composition only when your application is itself an execution host, for example, a server, evaluation harness, research runtime, or specialized embedder that must replace storage or orchestration components. ## Host-level choices The low-level crates expose focused contracts for: * core agent, event, capability, and provider values; * the shared Input/Reason/Act kernel and sans-I/O turn planner; * runtime host phases, canonical event history, and in-memory reference stores; * local SQLite-backed task and schedule state; * portable session and agent definitions, plus explicitly selected durable deployment components. A separately composed low-level host depends on `everruns`, `everruns-core` with the `host` feature, and `everruns-contracts` for neutral service contracts. The low-level boundary is `everruns_core::host`; concrete integrations are selected on the facade. It is healthy for such a host to use low-level extension traits; the goal is not to re-export every backend through one facade. ## Two engine boundaries `everruns::Engine` is a concrete application object that owns the behavior registry, sessions, lifecycle, queries and activation through its hosting profile. It is the normal Framework entrypoint, not an extension trait. Applications do not implement it. `everruns_core::engine` is the shared execution kernel, enabled by the `engine` feature. Advanced hosts compose its `Execution` contract and serializable `TurnExecution` state machine, `InputAtom`/`ReasonAtom`/`ActAtom`, and phase values. The implementation lives in `everruns_core::host`, and the Platform’s workers drive the same turn loop under a session lease. Both use narrow contracts from `everruns-core`. The engine feature has no dependency on host effects, platform, server, worker, or durable crates. Do not copy state advancement or the phase loop into a custom backend; implement the execution boundary and keep deployment-specific service selection in the host. Where a turn runs is a separate interface: `everruns_core::host::TurnBackend` starts, cancels, and observes a session’s turns. `InProcessBackend` runs a turn in process. The legacy local/custom-backend facade lane uses `ActorRunner` under the session’s lease, and the Platform’s server marks the session runnable for a worker that leases it and runs the whole turn. Coherent Framework hosting instead enters Engine’s shared session actor path. `TurnBackend` is not the durable Framework extension point. A low-level host that records input through its own event store starts turns from it with `TurnInput::StoredMessage` and continues a parked turn with `TurnInput::RecordedToolResults`, which every backend serves. The trait is public and unsealed but experimental. The facade’s [backend conformance suite](https://github.com/everruns/everruns/tree/main/crates/everruns/tests/backend_conformance) defines the behavior every backend must match. See [Framework Architecture](https://everruns.com/docs/framework/architecture/) for the complete layer map and the distinction between immediate and durable execution. Conversation persistence is the one backend with a single write path. Replace it by implementing the canonical `EventLog`/`EventReader` SPI and passing it to `HostBackends::with_event_log`; the required snapshot, continuation, and polling behavior is specified in [Implementing a custom event log](https://everruns.com/docs/framework/events-and-cancellation/#implementing-a-custom-event-log). ## Security boundary Backend replacement does not relax tenant, credential, filesystem, or tool execution boundaries. Preserve event ordering, credential redaction, workspace containment, and cancellation behavior when adapting the host. A custom backend must fail explicitly when it cannot satisfy a required contract.
---
# Data services
> Use persistent session SQL, named memory directories, and application-owned citable knowledge retrieval in Framework agents.
Source:
Data services attach to real Engine tool contexts. They require no Server database. Choose storage directories outside the agent workspace so filesystem tools cannot rewrite service state. ## Persistent SQL Enable `sql-database` and add `SqlDatabase::open(directory)`:
```rust
use everruns::{Agent, Model, database::SqlDatabase};
let agent = Agent::builder()
.instructions("Use SQL to retain structured notes.")
.model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?))
.sql_database(SqlDatabase::open(".private/session-databases")?)
.build()?;
```
This enables `sql_execute`, `sql_query`, and `sql_schema`. Databases are isolated by Engine session identity; retaining a SQL directory alone does not retain Engine sessions. Reopen your Engine’s durable hosting profile and reconstruct its agent registrations to resume the same sessions. The local backend uses the Server’s SQLite authorizer, statement execution, validation, row/byte limits and timeouts. `ATTACH`, extension loading, unsafe pragmas and virtual tables are denied. File locking coordinates quota checks across independent store instances/processes. Database names cannot select host paths. This is an agent-tool boundary; it does not isolate arbitrary host shell processes from storage directories. ## Named memory
```rust
use everruns::{Agent, Model, MemoryDirectory};
let agent = Agent::builder()
.instructions("Read policy from /memory/policy; save notes in /memory/notes.")
.model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?))
.capability("session_file_system")
.memory(MemoryDirectory::directory("policy", "./policy").read_only())
.memory(MemoryDirectory::directory("notes", "./notes"))
.build()?;
```
Names are distinct path components below `/memory/`. The application decides which sessions share a directory. Read-only mounts reject creation, editing, deletion and directory creation through every filesystem tool, while keeping host file permissions unchanged. Directory bindings are trusted host configuration. They provide no containment for tools that execute arbitrary native host processes. ## Knowledge bases and indexes Enable `knowledge`. `Knowledge::documents` supplies a small deterministic keyword retriever; `Knowledge::directory` snapshots Markdown and text files with bounded enumeration and byte limits. Symlinks inside the snapshot are rejected.
```rust
use everruns::{Agent, Model, knowledge::{Knowledge, Document}};
let source = Knowledge::documents("support", vec![Document::new(
"Refund policy", "Refund requests are accepted within 30 days.",
"https://example.com/policy/refunds",
)]);
let agent = Agent::builder()
.instructions("Search policy before answering and cite the source URI.")
.model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?))
.knowledge(source)
.build()?;
```
`search_knowledge` returns ranked passages with immutable file/version identities, source URIs and optional locations. `browse_knowledge` pages through bound originals and metadata; read and grep them under `/knowledge//`. Only the application-bound source is accessible. For semantic/vector search, implement `Retriever::search` and return `Passage` values through `Knowledge::new`. Attach originals with `.catalog(documents)` when the provider should also support browsing and mounted reads. The retriever owns embeddings, refresh and provider billing. Result counts, snippets and citation metadata are bounded, and provider errors are sanitized. Enable `.automatic(true)` to retrieve source evidence before model execution; it is retained by input ID for replay in protected host service storage. The [complete data example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_data.rs) runs a real agent that retrieves policy, persists a SQL decision and writes named memory:
```bash
cargo run -p everruns --features openai,sql-database,knowledge --example service_data
```
---
# Deploying a Framework App
> Ship a Framework app as a binary or container: persistent storage for its data directory, provider keys from the environment, and when to use serve instead.
Source:
A Framework app is your own Rust binary. The `everruns` crate needs no database, message broker, or Everruns server, so deploying it is deploying that binary: build it and give it provider credentials. If sessions must outlive the process, select durable session hosting and preserve its storage and registered Agent builds. A persistent directory alone is not an independent recovery mechanism. ## Choose where state lives | Engine setup | State | Deploy with | | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- | | `Engine::new()` | In memory, lost when the process exits | Any binary or container. Nothing to persist | | Engine with `hosting::SqliteHosting` | Transactional session authority, canonical events, admission receipts and wakeup obligations | Persistent SQLite storage, retained Agent builds and Environment factories, and a running native supervisor | | An agent built with legacy `.local(LocalConfig::new(dir))` | Local events, session catalog, and SQLite task and schedule state under `dir`; agent files under `dir/workspace` unless `.workspace(path)` sets another root | A persistent volume and application-driven reattachment and recovery | `SqliteHosting` and `LocalConfig` need the `local` feature. New durable services should select `SqliteHosting` on `Engine::builder()`, register immutable `AgentBuild` identities and start `SqliteHosting::supervise` at startup, independently of incoming requests. Workspace files, compute and secrets need explicit services with their own persistence guarantees. See [session hosting](https://everruns.com/docs/framework/hosting/) and [Sessions](https://everruns.com/docs/framework/sessions/#persistence) for the recovery boundaries. ### Existing LocalConfig deployments `AgentBuilder::local` is deprecated. The following guidance and example apply to that legacy lane; use [legacy session cutover](https://everruns.com/docs/framework/sessions/#moving-legacy-sessions-to-sqlitehosting) to move existing sessions to coherent Engine hosting. For a legacy deployment that keeps local state: * **One process per data directory.** The local profile is built for one embedded process at a time. Do not point two replicas or two containers at the same directory. Scale by giving each instance its own directory and routing each session to the instance that owns it. If two processes do share one, a session’s turns still run in only one of them at a time: each turn holds the session’s lease in the local database. * **Trusted paths only.** Choose the data and workspace directories from deployment configuration, never from model output or request input. * **Persistent storage.** Mount the data directory from a volume. New state files are created owner-only on Unix; your backups and copies need the same protection, because the event log holds conversation content. * **Resume after a restart.** Rebuild each agent from the same trusted configuration, then call `engine.attach(session_id, agent)` and `engine.resume(session_id)`. See [Resume after a process restart](https://everruns.com/docs/framework/sessions/#resume-after-a-process-restart). ## Advanced local hosts Hosts built with `everruns::local::LocalRuntimeBuilder` reconcile orphaned background executions after constructing their runtime and start periodic maintenance. Recovery requires executor evidence that ownership is lost. A missing owner process leaves the execution outcome unverified; interrupted commands are never replayed. When composing `LocalBackends` directly, retain its recovery-wrapped task registry, bind `task_recovery` to the runtime’s task wake observer, then call `recover_all` and `start_maintenance`. Recovery wakes are delivered at least once across crashes; a durable observer must deduplicate by session, task ID and terminal attempt. ## Pass secrets through the environment Each provider type reads its key with `from_env`: `OpenAI::from_env()` reads `OPENAI_API_KEY` and `OPENAI_BASE_URL`, `Anthropic::from_env()` reads `ANTHROPIC_API_KEY`, and so on. The full per-driver table is under [Credentials](https://everruns.com/docs/framework/models-and-providers/#credentials). Set those variables from your platform’s secret store (a container secret, a Kubernetes `Secret`, a systemd credential), not in the image. `from_env` returns an error when a required variable is missing, so a misconfigured deployment fails at startup instead of on the first turn. When your application already resolves the key itself, pass it with `OpenAI::new(key)`. On AWS, `Bedrock::default_chain()` uses the instance or task role, so no key is stored at all. A legacy `LocalConfig` service entry point:
```rust
use std::time::Duration;
use everruns::{Agent, Engine, LocalConfig, OpenAI};
// Deployment configuration, not request input.
let data_dir = std::env::var("APP_DATA_DIR").unwrap_or_else(|_| "/data".into());
let agent = Agent::builder()
.instructions("Answer support questions.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.local(LocalConfig::new(data_dir))
.build()?;
let engine = Engine::new();
let session = engine.create(agent);
// ... accept requests and drive sessions ...
tokio::signal::ctrl_c().await?;
engine.shutdown(Duration::from_secs(10)).await;
```
`APP_DATA_DIR` is this example’s own setting; the Framework reads no variable for the data directory. `Engine::shutdown` drains and flushes event listeners such as the OpenTelemetry and Braintrust exporters, so call it before the process exits. See [Observability](https://everruns.com/docs/framework/observability/). ## Build a container A multi-stage build keeps the toolchain out of the runtime image. This follows the Dockerfile in [AgentCore app](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore):
```dockerfile
FROM rust:1-bookworm AS build
WORKDIR /src
COPY . .
RUN cargo build --release --locked --bin my-agent
FROM gcr.io/distroless/cc-debian12:nonroot
COPY --from=build /src/target/release/my-agent /my-agent
VOLUME /data
ENTRYPOINT ["/my-agent"]
```
The `everruns` crate’s HTTP clients use rustls, so the image needs no OpenSSL. It still needs CA certificates to reach provider APIs; the distroless `cc` image includes them. The `nonroot` user must be able to write the mounted data directory. Run it with the key from a secret and the data directory on a volume:
```bash
docker run --rm \
-e OPENAI_API_KEY \
-e APP_DATA_DIR=/data \
-v my-agent-data:/data \
my-agent
```
A volatile Engine needs no state volume. A container using `SqliteHosting`, `LocalConfig` or persistent workspace services needs storage that survives replacement. ## When to use serve instead [serve](https://everruns.com/docs/framework/serve/) is an experimental application framework on top of the Framework. Use it instead of a hand-written binary when you want: * the Everruns `/v1` session API, so the SDKs, CLI, and UI chat view can drive your agents, plus AG-UI and A2A endpoints; * a manifest of agents, tools, secrets, and models that a host can provision from; * a `start` command that requires every declared secret, and that fails a model with no real route instead of falling back to the simulator. Hosts configure `OPENAI_API_KEY` for `openai/…` models, optionally with `OPENAI_BASE_URL` for a compatible Responses API endpoint, or use `OPENROUTER_API_KEY` for provider-qualified model IDs. A serve binary listens on `0.0.0.0`, on `PORT` (default 3000). Its state goes to the SQLite file named by `DATABASE_URL`, else to `SERVE_DATA_DIR`, else to `.serve/` in the working directory, or, with `start --store s3://bucket/prefix`, to a bucket that a daemon on another machine can take over (see [Keep the data in a bucket](https://everruns.com/docs/framework/serve/#keep-the-data-in-a-bucket)). Its wire API is unauthenticated by default. Configure [serve authentication](https://everruns.com/docs/framework/serve/#authentication) or place it behind an authenticating host before exposing it. See [Serve on AgentCore](https://everruns.com/docs/framework/serve-agentcore/) for Amazon Bedrock AgentCore; the [celld adapter](https://everruns.com/docs/framework/serve-celld/) is a separate source-only reference, not a published deployment integration. Keep a plain Framework binary when the agent is embedded in an application that already owns its API, authentication, and storage. To move to a managed, multi-tenant runtime instead, see [Moving to Platform or Cloud](https://everruns.com/docs/framework/moving-to-platform/). ## Connect custom code to a control plane Use an experimental [custom agent worker](https://everruns.com/docs/how-to/custom-agent-workers/) when your binary should own private tools and model access while the self-hosted Platform owns user sessions, event history and tracing. The [data-agent walkthrough](https://everruns.com/docs/framework/examples/custom-data-agent/) includes container packaging and a real-model end-to-end smoke test.
---
# Direct Model Calls and Decisions
> Call a model once through the Framework's provider edge, or ask Decisions for a number instead of prose, without building an agent.
Source:
Some work is one prompt and one answer: classify a string, draft a summary, extract a field. That needs the provider edge — drivers, endpoints, credentials, retries, error classification — but none of the agent loop around it. `Model::complete` is the whole API for that case:
```rust
use everruns::{Model, OpenAI};
let model = Model::new("gpt-6.2-sol", OpenAI::from_env()?);
let answer = model.complete("Name the three primary colors.").await?;
println!("{answer}");
```
The model is the same value an [agent](https://everruns.com/docs/framework/agents/) takes, reached through the same [`Provider`](https://everruns.com/docs/framework/models-and-providers/), which can also be asked [which models it offers](https://everruns.com/docs/framework/models-and-providers/#model-catalogs). Nothing is persisted: a direct completion owns no session, no history, and no workspace. Reach for an agent as soon as the work needs tools, multiple turns, durability, or events. When the answer should be a number rather than prose, such as a probability, a choice among options, or a graded score, use [Decisions](#decisions) instead. ## Testing without a provider `Model::simulated` returns canned responses from an in-process simulator (`everruns-llmsim`). It is a **test double**, not a local model: it runs no inference and is not a way to use Everruns without a model provider. It exists so tests and examples can assert on agent behavior without a network call or an API key. Real work always goes through a provider — see [Supported providers](https://everruns.com/docs/framework/models-and-providers/#supported-providers).
```rust
use everruns::Model;
let answer = Model::simulated("4").complete("What is 2 + 2?").await?;
assert_eq!(answer, "4");
```
See [Testing and simulation](https://everruns.com/docs/framework/testing-and-simulation/) for scripted multi-response simulators. ## System messages, context, and controls `Model::completion` describes the call before sending it. Messages append in call order; each control maps to one provider request field and stays unset unless assigned, so the provider keeps its own defaults.
```rust
use everruns::{Model, ReasoningEffort};
let response = model
.completion()
.system("Answer with a single word.")
.user("What is the capital of France?")
.max_tokens(16)
.reasoning_effort(ReasoningEffort::Low)
.send()
.await?;
println!("{}", response.text);
println!("{:?} tokens", response.metadata.total_tokens);
```
`send` returns the full `LlmResponse` — text, reasoning artifacts, tool calls, and call metadata. `text()` returns only the answer text. Replay prior turns with `.assistant(...)`: the completion carries no history of its own, so context is whatever the call passes. When the model is a bare provider-visible id, attach the provider on the completion instead of the model:
```rust
use everruns::{Model, OpenAI};
let answer = Model::from("gpt-6.2-sol")
.completion()
.provider(OpenAI::from_env()?)
.user("Summarize this in one line: ...")
.text()
.await?;
```
## Streaming `stream()` returns the provider’s events as they arrive, ending with a `Done` event carrying the call’s metadata:
```rust
use everruns::{LlmStreamEvent, Model};
use futures::StreamExt;
let mut stream = model.completion().user("Write a haiku.").stream().await?;
while let Some(event) = stream.next().await {
if let LlmStreamEvent::TextDelta(delta) = event? {
print!("{delta}");
}
}
```
What the stream guarantees: * Every `TextDelta` carries text. Assistant text is their concatenation. * `ReasoningDelta` streams reasoning live; the `ReasoningItem` that follows repeats the whole block, plus any replay state. Show the deltas and store the item, or use the item alone; adding both duplicates the reasoning. * The stream ends with exactly one `Done` or `Error`. `Error` keeps the provider’s message, error code, and HTTP status, including errors a gateway reports inside a `200` stream. * `Done` carries `finish_reason` as the provider sent it, or `None` when it sent none. It is never filled in as `stop`, so a caller can treat a missing reason as a failure. * Drivers retry `429` and transient `5xx` responses before the first event: up to 2 retries and 30 seconds by default. That time counts against any timeout you put around the call. To own retries yourself, build the driver with `.with_retry_config(everruns::llm::LlmRetryConfig::no_retry())`. The full contract is on `LlmStreamEvent` in the API reference. ## Errors `CompletionError` separates configuration mistakes from provider failures: * `MissingProvider` — the model names an id but nothing says how to reach it. * `NoMessages` — the completion was sent empty. * `Call(..)` — the provider call failed, carrying the `AgentLoopError` and its full `LlmError` decision. The first two are caught before any request leaves the process. ## Going lower `Completion` is a thin value-first layer over `Provider`, which is public. Applications that already hold a `Provider` — or implement their own [`ChatDriver`](https://everruns.com/docs/framework/models-and-providers/#custom-providers) — can call it directly with `everruns::llm`’s `Message` and `MessageRole`, plus the crate-root `LlmCallConfig` and `LlmResponse` re-exports:
```rust
use everruns::llm::{Message, MessageRole};
use everruns::{LlmCallConfig, Provider};
let response = provider
.chat_completion(
vec![Message::text(MessageRole::User, "What is 2 + 2?")],
&LlmCallConfig::new("gpt-6.2-sol"),
)
.await?;
```
That surface is the driver boundary itself: every field of `LlmCallConfig`, including tool definitions, is available, and nothing is defaulted for you. When all you have is a base URL, `DriverId::for_base_url` names the driver whose vendor serves it (OpenAI, Azure OpenAI, OpenRouter, Anthropic, Gemini, Fireworks). It returns `None` for a gateway or self-hosted server; those speak the OpenAI-compatible wire, so use `DriverId::OpenAICompletions`. The runnable version of this section is [`direct_llm.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_llm.rs), which runs offline without an API key. ## Decisions Some questions have typed answers. *Is this claim supported by the source? How severe is this complaint? Which queue does this ticket belong in?* A chat model answers those in prose, so the call site ends up with a prompt asking for JSON, a parser, and a fallback for when the parse fails. A decision service answers them as numbers instead, and the decision stays in your code. This is the counterpart to a direct model call: the same shape, a different contract. | | `Model` | `Decisions` | | ------------------- | ----------------- | ---------------------------- | | you send | messages | state plus typed questions | | you get back | text | calibrated numbers | | decides the outcome | the model’s words | your threshold, in your code | | streams | yes | no — one round trip | ### Quick start
```bash
cargo add everruns --features typesafe
cargo add tokio --features macros,rt-multi-thread
export TYPESAFE_API_KEY=... # a key from typesafe.ai
```
```rust
use everruns::{Decisions, TypeSafeAI};
#[tokio::main]
async fn main() -> Result<(), Box> {
let decisions = Decisions::new("jev-latest", TypeSafeAI::from_env()?);
let text = "CONGRATULATIONS! You've WON $1,000,000. Click here to claim your prize now!";
let spam = decisions.probability("Is this message spam?", text).await?;
println!("spam: {spam:.2}");
if spam > 0.9 {
println!("quarantined");
}
Ok(())
}
```
```text
spam: 0.98
quarantined
```
One number, and your own `> 0.9` decides — the model reports how likely, not what to do. The same call answers `0.03` for “Standup moved to 10am.” and `0.74` for a bare “Claim your prize now!”; the middling ones are what a threshold is for. `--features typesafe` adds `TypeSafeAI` and the `Jev` capability. Without it `everruns` names no vendor: `Decisions::new` takes any `DecisionsService`. ### OpenAI’s Decisions API [OpenAI’s Decisions API](https://developers.openai.com/api/docs/guides/decisions) (public beta, model `gpt-6-luna`) answers the same questions. `OpenAIDecisions` in `everruns-integrations` is a `DecisionsService`, so it drops into the same `Decisions`:
```bash
cargo add everruns
cargo add everruns-integrations --features openai-decisions
export OPENAI_API_KEY=...
```
```rust
use everruns::Decisions;
use everruns_integrations::openai_decisions::OpenAIDecisions;
let service = OpenAIDecisions::new(std::env::var("OPENAI_API_KEY")?);
let decisions = Decisions::new("gpt-6-luna", service);
let urgent = decisions
.probability("Does this convey urgency?", "My card was charged twice, fix it today.")
.await?;
println!("urgent: {urgent:.2}");
```
A runnable version that routes a support ticket with all three primitives is [`openai_decisions_triage`](https://github.com/everruns/everruns/blob/main/crates/integrations/examples/openai_decisions_triage.rs):
```bash
OPENAI_API_KEY=... cargo run -p everruns-integrations --features openai-decisions --example openai_decisions_triage
```
All three primitives are native and calibrated, and one request sends every question in one call. Score questions take at most ten levels. On Everruns Platform, set `UTILITY_DECISION_DRIVER=openai` with `UTILITY_OPENAI_API_KEY` to answer guardrail `jev` checks with it (see [environment variables](https://everruns.com/docs/sre/environment-variables/)). ### Three primitives A decision asks one or more questions about the same state. Each is one of three shapes:
```rust
use everruns::Decisions;
let answers = decisions
.about("I've been on hold for two hours and my card was charged twice.")
.noul("urgent", "Does this convey urgency?")
.score(
"severity",
"How severe is the problem the writer describes?",
[
"A minor annoyance",
"A real problem with their account",
"Serious harm requiring immediate action",
],
)
.choice(
"queue",
"Which team should handle this message?",
["billing", "technical", "sales"],
)
.send()
.await?;
let urgent: f64 = answers.probability("urgent")?;
let queue: &str = answers.selected("queue")?;
```
* **`noul`** — whether something holds, as the probability of yes. A value near 0.5 means yes and no are near-equally likely, not “medium”. ([Noul](https://docs.typesafe.ai/primitives/noul)) * **`choice`** — exactly one option from your set, with the distribution behind it. Needs at least two options. ([Choice](https://docs.typesafe.ai/primitives/choice)) * **`score`** — a position along levels you define, lowest first. Needs at least two levels. ([Score](https://docs.typesafe.ai/primitives/score)) The three are System One’s own, so TypeSafe’s [Primitives](https://docs.typesafe.ai/primitives) documents what each answer means and how to choose between them, and [State](https://docs.typesafe.ai/concepts/state) covers what to put in the `about(...)` value. Everruns names them the same way rather than inventing synonyms. Questions in one call are answered **in parallel inside a single request**, so asking five costs one round trip, not five. TypeSafe calls leaning on that [speculative fan-out](https://docs.typesafe.ai/patterns/fan-out): ask the questions you *might* need, and let your code decide which ones mattered. ### Ids are yours; instructions are the model’s The id labels the answer for your code and is never sent to the model. A question whose meaning lives in its id asks nothing:
```rust
// Wrong: the model never sees "is_the_joke_funny".
.noul("is_the_joke_funny", "?")
// Right: the instructions carry the question.
.noul("funny", "Would a general audience laugh at this joke?")
```
The same applies to score levels. Describe concrete situations — “A minor annoyance” reads on its own where “2 out of 5” does not. ### Read the tail, not the average For “is there any serious hit here” rules, read the probability mass at or above a level rather than the weighted score. Something probably fine but possibly awful must not average into fine:
```rust
// Not: answers.score("severity")? > 1.5
let serious = answers.tail("severity", 2)?;
if serious > 0.3 {
println!("escalated");
}
```
### Errors `DecisionsError` separates configuration mistakes from service failures, the same split [`CompletionError`](#errors) makes: * `MissingService` — the decisions was built without a service to reach. * `NoQuestions` — the decision was sent with nothing to ask. * `Unconfigured` — the service exists but the deployment never configured its credential, so it would answer nothing. * `NoSuchAnswer(id)` — you read an id that was not asked, or read an answer as the wrong shape (a `score` as a probability). * `Call(..)` — the service call failed, carrying the `AgentLoopError`. The first three are caught before any request leaves the process. `Unconfigured` is worth handling separately: a guardrail treats it as fail-open, but a direct caller usually wants to know the number never arrived rather than read a confident-looking default. ### Going lower `Decision` is a thin value-first layer over `DecisionsService`, which is public. Applications that already hold a service — or implement their own, over a different vendor or a local model — can call it directly with `everruns`’s `DecisionRequest`, `DecisionQuestion`, and `DecisionAnswer` re-exports:
```rust
use everruns::{DecisionQuestion, DecisionRequest, DecisionsService};
let outcome = service
.evaluate(
DecisionRequest::new("Claim your prize now!")
.ask("spam", DecisionQuestion::noul("Is this message spam?")),
)
.await?;
```
That surface is the contract itself: every question type and the full `DecisionOutcome`, including usage, with nothing defaulted for you. Implementing `DecisionsService` is also how a different decision service (another vendor, or a fine-tuned local model) plugs into the same `Decisions`, guardrails included. ### Choosing a model The model is named up front, the way `Model::new` names one: the service is transport, and the model is the thing that answers. There is no default to inherit without noticing, because a threshold calibrated against one version is not evidence about the next. Ids are the provider’s own, so they are spelled the way the vendor spells them. `jev-latest` is TypeSafe’s alias for the current Jev, so it tracks whatever the current version is; an exact id like `jev-1.14.0` pins one, so a vendor update cannot move your thresholds under you. Bare `jev` is not an id the API knows — nothing here rewrites what you pass. Ask for the alias and read back what answered, which is the id to pin once a threshold is calibrated:
```rust
let version = answers.model(); // "jev-1.14.0" for a "jev-latest" request
```
A single call can name a different model with the same method on the request builder, and it wins for that call:
```rust
let answers = decisions
.about("...")
.noul("urgent", "Does this convey urgency?")
.model("jev-1.14.0")
.send()
.await?;
```
A deployment that must pin a model does so by never exposing the knob in the config an agent writes — not by the type being unable to carry one, because there will be other decision services and other models. A deployment running the Everruns platform configures a separate `UTILITY_TYPESAFE_API_KEY` for its [guardrails](https://everruns.com/docs/capabilities/guardrails/) — a different account from the one an embedding application holds. ### Giving an agent the decisions Everything above is agentless: your code asks, your code decides. The other half is letting an *agent* classify as part of its own work — checking a claim against a source before citing it, rating a draft before sending it. `Jev` is the same decisions as a capability, so an agent gets it as a tool:
```rust
use everruns::{Agent, Engine, Jev, Model, OpenAI};
let agent = Agent::builder()
.name("reviewer")
.instructions(
"You review copy. When asked how something reads, measure it with \
jev_decision and report the numbers rather than judging by eye.",
)
.model(Model::new("gpt-6.2-sol", OpenAI::from_env()?))
.capability(Jev::from_env()?)
.build()?;
let session = Engine::new().create(agent);
let turn = session
.run("Rate this subject line for pushiness: 'Act now before it is too late'")
.await?;
```
The agent calls `jev_decision`, writing its own questions about whatever it is looking at, and gets the same calibrated numbers back. It is the identical tool the hosted [TypeSafe integration](https://everruns.com/docs/integrations/typesafe/) gives platform agents — same name, same schema — so behavior matches whether you embed the Framework or run on Everruns. Which one to reach for: | | you decide | the agent decides | | ------------ | -------------------------------------- | --------------------------------- | | **who asks** | your code writes the questions | the model writes the questions | | **use** | `Decisions` | the `Jev` capability | | **good for** | a policy check, a routing rule, a gate | verification inside a longer task | ### What stays with an agent A decision owns no session, no history, and no workspace, and runs no tools. Reach for an [agent](https://everruns.com/docs/framework/agents/) as soon as the work needs any of those. Typed output guarantees the interface, not the truth: validate thresholds against your own data and consequences. ### Testing without a credential `Decisions::simulated` returns a fixed number from an in-process stub. It is a **test double**, not a local decisions: it runs no inference, reads nothing from the state you pass it, and is not a way to classify without a provider. It exists so tests and examples can assert on the code around a decision without a network call or an API key. Real work always goes through a decision service: `TypeSafeAI` above, or your own `DecisionsService`.
```rust
use everruns::Decisions;
let p = Decisions::simulated(0.93)
.probability("Does this convey urgency?", "Two hours on hold.")
.await?;
assert!(p > 0.9);
```
Because the answer is fixed, a simulated decision proves your threshold logic runs — never that a real decision service would return that number. The runnable version of this section is [`direct_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_decisions.rs). It uses the stub by default so it runs with no key; pass `--live` (with `--features typesafe` and `TYPESAFE_API_KEY` set) to send the same questions to a real decision service. [`agent_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/agent_decisions.rs) does the same for the agent path. ### Shared provider accounts for decisions `Decisions::from_registry` accepts a `ProviderRegistry` and an explicit `ModelSpec`. Select `ModelSpec::on("typesafe", "jev-1.14.0")` for direct TypeSafe, or `ModelSpec::on("openrouter", "jev-1.14.0")` for OpenRouter. The OpenRouter driver maps that explicit alias to `typesafe/jev-1.14`; other IDs remain opaque. Providers hold authentication once, so their chat and decision drivers use the same account. Returned model IDs report the snapshot that actually answered. `Jev::with_provider(provider, model_spec)` supplies that same account to a Framework capability. Use its exact provider key in the model spec. Existing `Jev::new` and `Jev::with_client` conveniences remain available for applications holding a direct TypeSafe key. ## Offline batches For independent requests that can wait hours, use [Batch Model Requests](https://everruns.com/docs/framework/batch-model-requests/) to submit through Anthropic Message Batches and persist a ticket for later collection. ## Model refusals The Responses driver reports explicit model refusals as `LlmErrorKind::Refusal` for streamed calls, native JSON completions, and batch results. These are terminal decisions and are not automatically retried. The diagnostic retains the provider’s reason, bounded to 16 KiB; treat it as provider content when displaying or logging it. Engine sessions apply their configured error disclosure policy and expose the stable `model_refusal` category instead of treating refusal text as a successful answer.
---
# Durable Object hosting reference
> Run ordinary Everruns Rust agents with durable admission, leases, event logs and alarm-driven recovery owned by a Durable Object.
Source:
The source-only `everruns-durable-object` reference demonstrates the [pluggable session hosting SPI](https://everruns.com/docs/framework/hosting/) beneath `everruns::Engine`. It is independently resolved and is not a published Framework dependency or a first-class Cloudflare service integration. Application code defines ordinary Framework Agents and uses the same Session API on native execution and Wasm inside an object. The reference and deployment APIs remain experimental; their guarantees cover the failure domains exercised by the qualification tests.  Create an Engine with `ObjectHosting::new(client)` and register trusted Agent builds under immutable `AgentBuild` identities. `create_registered` asynchronously commits a session binding at a caller-retained session ID. `Session::send_with_id` commits input, its deduplicated receipt and an independent wakeup obligation. Retain those identities across uncertain transport responses. Admission precedes turn assignment. Reopen an admission by its message identity to observe pending assignment, a durable wait, suspension or finalized outcome. Dropping a handle or SSE connection does not cancel accepted work. Approval and tool answers name their exact logical turn; cancellation names a retained input. Acceptance of cancellation is separate from reaching a cancelled terminal state. Framework start/completion hooks run on the executor under the same ownership epoch. Invocation records conservatively handle uncertain external effects; explicitly idempotent hooks can opt into recovery. A turn result becomes final only after its awaited Framework completion hooks settle. ## Native and isolate placements The [native example](https://github.com/everruns/everruns/tree/main/crates/durable-object/examples) registers an ordinary Agent on Engine and supplies that Engine to an authenticated `Executor`. Activation validates the authoritative binding and retained registry. The [Wasm application](https://github.com/everruns/everruns/tree/main/examples/durable-object-agent) uses the same [Agent definition](https://github.com/everruns/everruns/blob/main/crates/durable-object/examples/shared/agent.rs), which imports only `everruns`. Its Workers `/v1` adapter calls Engine for creation, admission and queries, and uses bounded canonical pages for SSE replay. JavaScript handles transport, storage and alarms; Rust owns agent and tool policy. [serve on celld](https://everruns.com/docs/framework/serve-celld/) selects this hosting profile and exposes its same Engine for native activation. The celld isolate bootstrap imports the shared Wasm application. There is no application runtime factory or celld step machine to implement. ## Hosting boundaries Keep API, authority and executor credentials separate. Tenant scope comes from trusted host configuration. Privileged authority credentials stay on trusted hosts; browsers use the user API. Production native authority/executor endpoints require HTTPS. Retain exact builds and Environment references while sessions need them; unavailable behavior suspends rather than silently selecting the latest. Every canonical/session-state mutation checks an unexpired epoch at the authority. An old owner cannot commit after replacement. External effects still need their own idempotency or reconciliation: a started tool with at-most-once recovery is reported interrupted when its effect is uncertain, and is not blindly repeated. This does not promise exactly-once external effects. Workspace and compute services are explicit retained Environment factories. A journal does not preserve ephemeral container files, secrets, background tasks, channels or schedules. Unsupported requirements must be refused before admission. The adapter’s default secret backend refuses plaintext storage. See the [adapter setup and tests](https://github.com/everruns/everruns/tree/main/crates/durable-object) for workerd, Cloudflare and replicated celld qualification. A test qualifies its specific revision, placement and failure domain; local fleet results do not establish production bucket or network-partition guarantees.
---
# Evaluation and traces
> Project portable session traces and run shared rule, judge and decision scorers in offline cases or bounded online listeners.
Source:
Portable trace projection uses the same pure event fold as the Server’s trace index:
```rust
use everruns::trace::Trace;
let trace = Trace::project(&session.events_after(0).await?)?;
println!("{}", serde_json::to_string_pretty(&trace)?);
```
Supply a complete, sequence-ordered prefix from one session. Mixed sessions, missing stored sequences and duplicate/out-of-order events are rejected. The result contains turns and model/tool/approval/delegation steps with bounded previews. Full model prompts and tool bodies are not retained in the projection. Treat trace previews as application data when exporting them. The indexed Server API and trace viewer are host features. ## Offline evaluation Enable `evaluation`. `Evaluator` validates and runs the same deterministic scoring rules the Server uses. Supported rules include contains/not-contains, regex, tool use and argument matching, call/turn/token bounds, JSON schema and citation faithfulness. File checks use explicitly collected session workspace evidence.
```rust
use everruns::evaluation::{Evaluator, Scorer};
let evaluator = Evaluator::new(vec![
Scorer::Contains { text: "42".into(), weight: 1.0 },
Scorer::TurnsWithin { max: 1, weight: 1.0 },
])?;
```
`run(engine, agent, cases, concurrency)` creates a fresh real session per case, runs each input in order, captures evidence and returns results in case order. Concurrency is explicitly bounded to 1–32. Each case has a per-input deadline, and timed-out turns are cooperatively cancelled. Results retain session IDs, exact build registrations, evidence, ordered scores and failures for debugging. For a durable Engine, use `run_registered(engine, build, cases, concurrency)` with an application-retained `AgentBuild`; re-register it after restart. Use `.judge(Model)` for rubric/citation judges and `.decisions(Decisions)` for typed yes/no grading. Judge prompt construction and response parsing are shared with Server. Providers and credentials are explicitly selected by the application. Hosted model IDs require a custom `Grader`; they are never silently translated into another account’s model. Missing evidence/models and provider failures are errors. `Grader` also supports application-specific external evidence. The [complete evaluation example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_evaluation.rs) runs a real arithmetic agent, prints scores and projects its trace:
```bash
cargo run -p everruns --features openai,evaluation --example service_evaluation
```
## Online scoring Install `OnlineScoring::new(evaluator, sink)` with `Engine::builder().listener(...)`. Implement `ScoreSink` to retain/export grades. It uses the same evaluator as offline runs and executes on the Engine’s bounded observer queue, outside turn execution. The listener bounds retained turns and evidence size; overflow, unsuccessful turns and grading failures are reported to the sink. Monitor Engine observer drop statistics. Online scoring is observational and must not be used as a durable audit ledger; replay canonical events for complete historical grading. `Engine::shutdown(deadline)` flushes listeners within a caller-selected deadline. Server evaluation definitions, history/reporting storage, durable scoring jobs and dashboards remain host concerns. ## Compaction observation The Framework exposes the same pure operational assessment used by hosted compaction monitoring, without an extra feature or capability:
```rust
use everruns::{SessionEvent, CompactionVerdict, observe_compaction};
fn inspect(event: &SessionEvent) {
if let Some(observation) = observe_compaction(event) {
if observation.verdict == CompactionVerdict::Failed {
eprintln!("{}", observation.reason);
}
}
}
```
Call it from an application-owned Engine listener and retain findings in your own sink. It does not make model calls or change the session. A pass reports execution metadata, not independent durability or semantic validation. Old events without the runtime’s history-evidence permission are ineligible for content assessment. Hosts own consent, tenant boundaries, bounded evidence reads, judge selection and spending admission. Organization opt-in, durable jobs, reports and review UI are Platform features; built-in and custom harnesses share their runtime instrumentation.
---
# Event triggers
> Route authenticated events through shared filters, changes-only checks, durable replay protection and recoverable Engine admission.
Source:
Enable `triggers` to use the Server’s filtering, change detection, template rendering and GitHub normalization without its management APIs. An application owns HTTP authentication, subscriptions and scheduling; `Dispatcher` owns the event-to-Engine pipeline.
```rust
use everruns::triggers::{Dispatcher, SqliteStore, Trigger, Filter, Routing};
let dispatcher = Dispatcher::new(engine, SqliteStore::open("private-triggers.sqlite")?, Trigger {
key: "deployment-watcher/v1".into(),
build,
message: "Deployment {{payload.version}} completed for {{payload.service}}".into(),
filter: Filter::default(),
routing: Routing::PerSubject,
})?;
```
Register the exact `AgentBuild` with the Engine before dispatch. Change the trigger key when changing its configuration; accidental reuse is rejected. Routing can create one session per event, per subject, or one shared session. Missing subjects under `PerSubject` use an event-specific session. Filters combine dotted paths and accepted values. Numeric/boolean scalars match their string representations using Server semantics. `ChangesOnly` remembers the last value per subject: the first event establishes the baseline, repeats are filtered, and a change passes. Events missing the selected value do not erase it. The latest 500 subjects are retained. ## Recovery and replay protection Before calling the Engine, a transaction freezes the rendered input, session and input IDs in a durable outbox. Concurrent retries and recovery use `Session::send_with_id` to admit the same input once. A delivery reports `Dispatched` after admission, not after the agent finishes; use its session/input IDs to inspect or wait for the admission. Call `recover()` at application startup and from your scheduling loop. It retries unadmitted deliveries. Engine hosting and the outbox must both survive process loss. At most 200 unadmitted deliveries are accepted per trigger; recent payload history is bounded. Permanent receipt tombstones retain replay protection even after delivered/filtered payload history is pruned. Retire the database explicitly when that retention is no longer required. `StateStore` lets applications replace SQLite. Its compare-and-swap must atomically update state and insert any new receipt, and receipt tombstones must outlive history pruning. Store state outside the agent workspace, with application-level access restrictions. ## Sources `Event` accepts already authenticated application events. Never construct it directly from a public webhook without authentication. `github(secret, GitHubRequest, own_bot)` verifies `X-Hub-Signature-256` over the exact body before parsing, suppresses the selected bot’s own events, and shares GitHub/CI context normalization with Server. Restrict subscribed event types and repositories in the ingress/filter configuration. GitHub signatures cover the body, without a timestamp or delivery header. The adapter derives replay identity from the verified body, so changing the unsigned delivery header cannot bypass deduplication. Identical signed bodies are treated as the same event. The header remains informational `github.delivery_id` metadata; event/repository routing must also validate the expected payload shape. `mcp(secret, McpRequest)` uses the shared Standard Webhooks verifier, enforces the timestamp window and expected live subscription, then builds MCP template context. The application owns subscribe/renew/unsubscribe calls and passes only the active subscription identity. Oversized bodies and invalid signatures are rejected before dispatch. The [complete trigger example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_triggers.rs) uses a real model and persistent local storage. Run it twice with the same event ID to see durable duplicate suppression:
```bash
cargo run -p everruns --features openai,local,triggers --example service_triggers -- deploy-123
```
---
# Events and Cancellation
> Subscribe to live Framework session events, replay durable events, cancel a turn, and work with the canonical event protocol.
Source:
Subscribe before sending a message to observe its live event projection:
```rust
use everruns::{Agent, Engine, OpenAI};
let agent = Agent::builder()
.instructions("Be concise.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.build()?;
let session = Engine::new().create(agent);
let mut events = session.events();
let pending = session.send("Start.").await?;
while let Some(event) = events.recv().await? {
println!("{}", event.event_type());
if event.kind.is_terminal() {
break;
}
}
let turn = pending.wait().await?;
assert!(turn.success);
```
Known events have typed `SessionEventKind` values. Unknown canonical event types are preserved as `Other` with their payload, so the projection does not silently drop information. The feed is live and non-blocking: `send` starts execution in the session’s background actor, a slow or dropped consumer does not stop the turn, and the feed is not a durable replay API. Use [lifecycle hooks](https://everruns.com/docs/framework/lifecycle-hooks/) instead when application work must be awaited at an execution boundary or its failure must affect the run. See [Canonical events](#canonical-events) below for canonical envelopes, live-delta memory bounds, explicit lag handling, ordering, and the durability boundary. Use [Session history and resume](https://everruns.com/docs/framework/sessions/#history-and-resume) to rebuild a bounded persisted transcript after live lag or a process restart. ## Replay durable events Durable events carry an increasing per-session `sequence()` starting at 1. `session.events_after(n)` reads a stable snapshot of durable events after sequence `n`. On a branch-aware authority such as `SqliteHosting`, it follows the active conversation branch, so sequences can have gaps. A backlog above 100,000 events returns `HistoryError::HistoryTooLarge` rather than an unbounded result. Replay works after `Engine::resume` in a new process when the hosting profile preserves the session and its events. `session.events_from(n)` returns an `EventStream` that replays that backlog and then continues live, deduplicating events already replayed. Live-stream lag is reported explicitly; reconnect to recover from the last sequence seen. Ephemeral deltas have no sequence and pass through live only:
```rust
let last_seen = 2;
let mut stream = session.events_from(last_seen).await?;
while let Ok(Some(event)) = stream.try_recv() {
println!("{:?} {}", event.sequence(), event.event_type());
}
```
## Cancel a turn A message receipt exposes the specific accepting turn, so live applications can cancel without racing against whichever turn is active later:
```rust
let pending = session.send("Start.").await?;
pending.turn().cancel().await?;
let cancelled = pending.wait().await?;
assert!(!cancelled.success);
```
`run_with` retains cancellation-token convenience for request/response calls:
```rust
use everruns::{CancellationToken, RunOptions};
let cancel = CancellationToken::new();
let options = RunOptions::new().cancel_token(cancel.clone());
cancel.cancel();
let turn = session.run_with("Stop before starting.", options).await?;
assert!(!turn.success);
```
Cancellation is cooperative. Cancelling drops the in-flight turn future and tears down tool work through the same runtime path; it does not kill the host process or provide an independent transaction boundary. ## Cancel one tool call To stop one running tool call and let the turn go on, for example when a user stops a long shell command, cancel it by its tool call id:
```rust
if session.tool_call_running("call_42") {
session.cancel_tool_call("call_42");
}
```
The call’s future is dropped, its tool context cancellation fires, and it completes with status `cancelled` and an error the model reads, so the agent can react and continue. Other calls in the same batch finish as usual. Calls register in a process-wide registry; a host that wants its own puts a `ToolCallCancels` (`everruns_core::engine::tool_cancel`) in the tool context extensions. ## Observability To send every session’s events to OpenTelemetry or Braintrust, or to your own listener, register it on the Engine; see [Framework observability](https://everruns.com/docs/framework/observability/). ## Canonical events `Session::events()` installs an in-process subscriber without exposing runtime event buses or core event types. Subscribe before `Session::run()` so the stream sees the turn from its first event.
```rust
use everruns::prelude::*;
let agent = Agent::builder()
.instructions("Answer concisely.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.build()?;
let engine = Engine::new();
let mut session = engine.create(agent);
let mut events = session.events();
let observer = tokio::spawn(async move {
let mut canonical_events = Vec::new();
while let Some(event) = events.recv().await? {
// Recording for replay, so take the canonical envelope: it withholds
// nothing. `as_json()` carries the reviewed projection instead.
canonical_events.push(event.canonical_json().clone());
// Typed rendering for common terminal/service UI concerns.
match event.kind {
SessionEventKind::TextDelta { delta } => print!("{delta}"),
SessionEventKind::ToolStarted { tool_name, .. } => {
eprintln!("starting {tool_name}");
}
SessionEventKind::ToolCompleted {
tool_name,
success,
..
} => eprintln!("{tool_name}: {success}"),
SessionEventKind::TurnFailed { error } => eprintln!("failed: {error}"),
SessionEventKind::TurnCancelled => eprintln!("cancelled"),
_ => {}
}
}
Ok::<_, EventStreamError>(canonical_events)
});
let turn = session.run("hello").await?;
drop(session); // closes the subscriber once buffered events are drained
let recorded = observer.await??;
assert!(!recorded.is_empty());
```
### One protocol, two views `SessionEventKind` is a convenience projection for application renderers. It promotes assistant output lifecycle and deltas, model reasoning/generation, tool lifecycle/progress/output, and turn terminal states. It is non-exhaustive, so match it with a fallback arm. `SessionEvent` exposes two surfaces, and which one you want depends on whether you are rendering or recording. `SessionEvent::as_json()` and `SessionEvent::data()` return the **reviewed** form: the event envelope — id, type, timestamp, optional persisted sequence, correlation context and tags, with reviewed public provenance in metadata — and a `data` payload holding only the fields promoted onto `SessionEventKind`. Nothing else reaches it, so a field added inside the runtime cannot become part of the Framework’s public surface, or travel to wherever your application forwards these envelopes, without being promoted first. This is the form to log, forward, or expose to clients. `SessionEvent::canonical_json()` returns the **canonical** envelope with the complete payload: prompts, tool arguments, tool results, structured assistant messages, and the payloads of event types this version does not recognize. Nothing observable is lost — this is the form for recording, auditing, and replay. It follows the runtime’s internal shape rather than the Framework’s reviewed surface, so treat what you read from it as unstable, and do not forward it anywhere the conversation itself should not go. Live `output.message.delta` envelopes omit the redundant `data.accumulated` prefix on both surfaces; retaining every growing prefix in a slow subscriber’s buffer would use quadratic memory. Concatenate the typed `TextDelta::delta` values to reconstruct streamed text, or use the subsequent `output.message.completed` event for the complete message. When a provider stream breaks after some text, the runtime retries it under the same message id and sends `OutputStarted` for that id again first: drop the text collected for the message and start over. Model-generation accounting — model, provider, token counts, cost, and duration — is promoted onto `SessionEventKind::ModelGeneration`, so tracking spend never requires the unstable surface. This bridge does not define a second wire schema. The canonical event contract, compatibility rules, and lifecycle semantics remain documented in [Events](https://everruns.com/docs/explanation/events/). ### Durability, observation, and derived history These roles are deliberately separate: * `EventLog` is the host’s sole durable conversation write authority. It stores complete canonical event envelopes and provides bounded cursor replay. * `EventSink` is the host’s post-commit, nonblocking live-delivery boundary. `Session::events()` exposes that observation path as an ergonomic `EventStream` subscriber. Neither sink nor subscriber is durable or authoritative. * `EventHistory` is one read-only message projection rebuilt from `EventLog` replay. It is an index/view, never a second writable message store. Framework applications read that bounded projection through [`Session::history()`](https://everruns.com/docs/framework/sessions/#history-and-resume). It pages messages from a stable event-log snapshot; it does not maintain or write an independent transcript. Rebuild a transcript in persisted sequence order from `input.message`, `output.message.completed`, and relevant `tool.completed` events. An `output.message.replaced` event alone creates no history message; the subsequent completed message contains the safe replacement. If a crash leaves a replacement without completion, replay correctly omits that incomplete output. The Framework stream exposes canonical payloads, subject to the live-delta exception above, and introduces no independent writable message history. Canonical recordings can contain user messages, agent instructions, model inputs, tool arguments, and tool results. Treat them as application data with the same access controls and retention policy as the session itself; do not log them indiscriminately. Model and tool text is untrusted: terminal renderers should strip or escape control sequences, and web renderers should escape it as content rather than interpreting it as markup or commands. Provider credentials are not part of the event protocol. ### Implementing a custom event log An advanced host can store canonical events itself. `everruns-core` (`host` feature) exposes `EventReader` and `EventLog` as a public SPI: an external crate implements both against its own storage and supplies the result to composition through `HostBackends::with_event_log`. No in-crate access is required, cursors and pages are built with `EventCursor::continuation`, `EventCursor::after`, and `EventPage::new`, which validate the shared invariants. Three request shapes are distinguished by `EventReadRequest::cursor()`: * no cursor is an initial read that captures the session’s current high-watermark and reports it as `EventPage::snapshot_high_watermark()`; * a cursor whose `snapshot_high_watermark()` is `Some` is a continuation pinned to that snapshot, so appends committed later stay invisible and paging neither skips nor duplicates; * a cursor whose `snapshot_high_watermark()` is `None`, built by `EventCursor::after`, is a poll that captures a fresh snapshot and therefore does observe those later appends.
```rust
use async_trait::async_trait;
use everruns_core::events::{Event, EventRequest};
use everruns_contracts::typed_id::EventId;
use everruns_core::host::{
EventCursor, EventDurability, EventLog, EventLogError, EventPage, EventReadRequest,
EventReader,
};
#[async_trait]
impl EventReader for MyEventLog {
async fn read_page(&self, request: EventReadRequest) -> Result {
let session_id = request.session_id();
let current_high = self.high_watermark(session_id);
let (after, snapshot) = match request.cursor() {
None => (0, current_high),
Some(cursor) => {
if cursor.session_id() != session_id {
return Err(EventLogError::CrossSessionCursor {
detail: "cursor belongs to another session".into(),
});
}
match cursor.snapshot_high_watermark() {
Some(snapshot) if snapshot > current_high => {
return Err(EventLogError::ExpiredCursor {
detail: "cursor snapshot is not available".into(),
});
}
// Pinned continuation, then the polling form.
Some(snapshot) => (cursor.after_sequence(), snapshot),
None => (cursor.after_sequence(), current_high),
}
}
};
let limit = request.limit().get();
let mut events = self.events_in(session_id, after, snapshot, limit + 1);
let has_more = events.len() > limit;
if has_more {
events.pop();
}
let next_cursor = has_more
.then(|| {
let last = events.last().and_then(|event: &Event| event.sequence).unwrap_or(after);
EventCursor::continuation(session_id, last, snapshot)
})
.transpose()?;
EventPage::new(events, next_cursor, snapshot)
}
}
#[async_trait]
impl EventLog for MyEventLog {
async fn append(&self, request: EventRequest) -> Result {
if request.is_ephemeral() {
return Err(EventLogError::InvalidAppend {
detail: "ephemeral events are sink-only".into(),
});
}
// The log owns identity: assign the event id and the next per-session
// sequence, persist, and return the finalized canonical envelope.
let sequence = self.next_sequence(request.session_id);
let event = request.into_event(EventId::new(), sequence);
self.persist(&event)?;
Ok(event)
}
fn durability(&self) -> EventDurability {
EventDurability::CrashDurable
}
}
```
The contract an implementation must uphold: * an accepted append owns id and sequence assignment and returns the finalized canonical `Event`, visible to the next read of that session; * durable sequences are unique and strictly increasing per session, and need not be contiguous, gaps are expected when a reader projects an append-only physical log into a filtered logical event sequence; * a continuation stays pinned to the first page’s high-watermark and cannot observe concurrent appends; a poll cursor can; * cursor/session mismatches and inconsistent positions return the typed `EventLogError` variants above rather than panicking; * the log is append-only. There is no truncate, rewind, or mutation contract, and `EventHistory` remains a read-only projection rather than a second writable message store. The [external event-log example](https://github.com/everruns/everruns/tree/main/crates/everruns/tests/fixtures/external-consumer/event-log) provides a complete implementation in a standalone application. ### Ordering and bounded delivery A subscriber receives events in channel arrival order and each session has its own stream. The canonical `sequence` field is a replay position, not a live delivery counter: durable events carry `Some(sequence)` and live-only ephemeral events such as streaming deltas carry no sequence. Persisted sequences increase monotonically per session and may have gaps. Ephemeral events do not consume replay positions. The live stream has a bounded buffer and never applies backpressure to the agent turn. A dropped or slow subscriber cannot stall model or tool execution. If a subscriber falls behind, `recv()` and `try_recv()` return `EventStreamError::Lagged { missed }`; loss is never hidden. The next receive can continue from the oldest retained event, but the renderer must treat its live projection as incomplete. Streaming deltas are provisional and sink-only. Completed assistant/tool events are authoritative, and an output-replacement event means accumulated text for that message must be discarded. Durable events reach the live sink only after their log append commits; ephemeral events go directly to the sink and never enter history. After live lag, a Framework application can rebuild its persisted transcript with bounded [`Session::history()` pages](https://everruns.com/docs/framework/sessions/#history-and-resume). That projection excludes ephemeral deltas by design. Applications that need raw durable envelopes rather than derived messages can provide and read an `EventLog` through the advanced `everruns-core` (`host` feature) SPI; neither recovery path relies on the in-process subscriber. ### Cancellation and failure Pass a `CancellationToken` through `RunOptions` to stop a turn. Cancellation produces both a `Turn` with `TurnStopReason::Cancelled` and a correlated `turn.cancelled` event carrying the same `turn_id`. Runtime failures remain available through `Session::run()`’s outcome/error semantics and the event stream. Subscribe before running and continue draining the stream after the run resolves to retain the terminal failure event and its full structured payload. The complete runnable example is [`canonical_events.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/canonical_events.rs):
```bash
cargo run -p everruns --example canonical_events
```
---
# Runnable Examples
> Complete Framework programs, maintained and compiled with the everruns crate.
Source:
The [`crates/everruns/examples` catalog](https://github.com/everruns/everruns/tree/main/crates/everruns/examples) contains the maintained public examples. Each imports the `everruns` facade. ## Complete agents The root-level [`examples`](https://github.com/everruns/everruns/tree/main/examples) catalog contains eight Framework walkthroughs. Each folder includes the program, instructions, fixtures where applicable, and recording scripts. Run them from a repository checkout: their dependencies point to the workspace crates. `cargo run` uses a real provider and can incur charges. CI tests offline tool behavior and recording logic; it does not establish the quality of a live model’s answer. | Example | Provider and model | What it does | | -------------------------------------------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------- | | [Support Agent](https://everruns.com/docs/framework/examples/support-agent/) | OpenAI `gpt-6.2-sol` | Chooses between MFA recovery, lockout, and browser troubleshooting from facts and policy. | | [Everruns Support Agent](https://everruns.com/docs/framework/examples/everruns-support-agent/) | Anthropic `claude-opus-5-5` | Searches and reads citable official documentation snapshots. | | [Coding Review Agent](https://everruns.com/docs/framework/examples/coding-review-agent/) | Anthropic `claude-sonnet-5` | Reads a refund contract and executes a fixed regression before reporting a defect. | | [Research Agent](https://everruns.com/docs/framework/examples/research-agent/) | OpenRouter `z-ai/glm-5.2` | Searches and fetches primary sources before writing a cited brief. | | [Incident Commander Agent](https://everruns.com/docs/framework/examples/incident-commander-agent/) | Meta Model API `muse-spark-1.3` | Investigates fixture telemetry and persists an evidence-backed incident update. | Start with Support for typed tools, Research for reusable capabilities, or Code Review for restricted execution. Each walkthrough shows the agent builder and session loop, explains expected behavior, and documents what remains a fixture. These are in-memory sessions. For durability itself, use the session-history and workspace examples below. Importable hosted Platform definitions live separately as TOML folders in [`examples/agents`](https://github.com/everruns/everruns/tree/main/examples/agents). See [file-based agents](https://everruns.com/docs/how-to/define-agents-as-files/) for loading, importing and serving them. ## Execution runtimes | Example | Provider and model | What it does | | -------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | [Bashkit Repo Agent](https://everruns.com/docs/framework/examples/bashkit-repo-agent/) | OpenAI `gpt-6.2-sol` | Cuts a release in a real repository with the sandboxed Bashkit shell as its only tool, then verifies the result on disk. | | [Code Mode MCP Agent](https://everruns.com/docs/framework/code-mode/) | OpenAI `gpt-6.2-sol` (or through OpenRouter) | Answers a question over a 1,000-tool MCP server by calling its tools from Bashkit scripts, with none of their schemas in the prompt. | | [Host Shell Agent](https://everruns.com/docs/framework/examples/host-shell-agent/) | OpenAI `gpt-6.2-sol` | Fixes a failing Rust test suite by compiling and running it through real processes inside a kernel-enforced boundary. | | [Foreman](https://everruns.com/docs/framework/examples/foreman-agent/) | TypeSafe `jev-latest` over an Everruns session, Codex, or yolop | Supervises a live coding session with nine decision questions per reading, and stops, verifies, or finishes it from a deterministic policy. | ## Terminal hosts | Example | Provider and model | What it does | | -------------------------------------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`attached-cli`](https://github.com/everruns/everruns/tree/main/examples/attached-cli) | OpenAI `gpt-6.2-sol` | Serves a running session to a second terminal, which lists, watches, sends to and cancels it through [attached control](https://everruns.com/docs/framework/attach-control/). | ## Talking to other companies’ agents | Example | Provider and model | What it does | | ------------------------------------------------------------------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`poppy-personal-agent`](https://github.com/everruns/everruns/tree/main/examples/poppy-personal-agent) | OpenAI `gpt-6.2-sol` | Acts as a person’s own agent over the [Personal Agent Protocol](https://everruns.com/docs/features/poppy/): finds a company by its domain, asks its agent about an order, and reports the answer. | ## Serving agents These run agents as HTTP services with [Serve](https://everruns.com/docs/framework/serve/). | Example | What it does | Command | | ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | | [`serve/hello`](https://github.com/everruns/everruns/tree/main/examples/serve/hello) | The smallest serve app: one agent, one tool | `cargo run -p serve-example-hello` | | [`serve/ag-ui`](https://github.com/everruns/everruns/tree/main/examples/serve/ag-ui) | An agent streamed to CopilotKit or any AG-UI client | `cargo run -p serve-example-ag-ui` | | [`serve/a2a`](https://github.com/everruns/everruns/tree/main/examples/serve/a2a) | Two agents over [A2A](https://everruns.com/docs/framework/a2a/): a researcher served by serve, and a writer that delegates to it. Offline by default | `cargo run -p serve-example-a2a --bin researcher`, then `--bin writer -- "tide pools"` | | [`serve/voice`](https://github.com/everruns/everruns/tree/main/examples/serve/voice) | A hotel front desk people [call from the browser](https://everruns.com/docs/framework/voice/), with a greeting and a speaking style | `OPENAI_API_KEY=… cargo run -p serve-example-voice` | ## Durable hosts [Durable multiplayer chat on Cloudflare](https://everruns.com/docs/framework/examples/durable-chat/) runs the portable kernel inside a SQLite-backed Durable Object, with shared rooms, forks, scheduled follow-ups, and a recoverable history-search child. This is an experimental advanced host example. ## Core crate catalog | Example | Demonstrates | Command | | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | [`capability_configuration.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/capability_configuration.rs) | Typed Compaction and ToolSearch, a code-defined Definition, and a dynamic third-party reference through one entrypoint | `cargo run -p everruns --example capability_configuration` | | [`model_catalog.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/model_catalog.rs) | Listing a provider’s models, reading their metadata, and running the selection | `cargo run -p everruns --example model_catalog` | | [`ask_user.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/ask_user.rs) | Structured questions answered by a host responder, and the unattended defaults path, fully offline | `cargo run -p everruns --example ask_user` | | [`ag_ui_axum.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/ag_ui_axum.rs) | An AG-UI 1.0 run served over axum | `cargo run -p everruns --features ag-ui-axum --example ag_ui_axum` | | [`image_generation.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/image_generation.rs) | An agent on GPT-6.2 Sol that draws an animated GIF with GPT Image 2.6 Flare and saves it next to you | `OPENAI_API_KEY=… cargo run -p everruns --features openai,images,local --example image_generation` | | [`workspace_policy.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/workspace_policy.rs) | Safe workspace scopes and trusted starter files, fully offline | `cargo run -p everruns --example workspace_policy` | | [`direct_llm.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_llm.rs) | One-shot, builder, and streamed model calls with no agent, fully offline | `cargo run -p everruns --example direct_llm` | | [`direct_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_decisions.rs) | Typed questions and calibrated answers with no agent, fully offline | `cargo run -p everruns --example direct_decisions` | | [`agent_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/agent_decisions.rs) | An agent that classifies with its own questions, offline by default | `cargo run -p everruns --example agent_decisions` | | [`live_session.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/live_session.rs) | Non-blocking send, automatic steering, and optional waiting, fully offline | `cargo run -p everruns --example live_session` | | [`hello.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/hello.rs) | Small live-provider agent | `cargo run -p everruns --features openai --example hello` | | [`production_agent.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/production_agent.rs) | Tools, files, and production-style setup | `cargo run -p everruns --features openai --example production_agent` | | [`github_monitor.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/github_monitor.rs) | Typed tools and an offline simulation mode | `cargo run -p everruns --features openai --example github_monitor -- --simulate` | | [`session_work.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/session_work.rs) | Offline session work, leased delivery, and completion wakes | `cargo run -p everruns --example session_work` | | [`session_history.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/session_history.rs) | Offline durable resume and bounded history pages | `cargo run -p everruns --features local --example session_history` | | [`engine_sessions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/engine_sessions.rs) | Concrete Engine ownership, isolated sessions, and engine-scoped resume | `cargo run -p everruns --example engine_sessions` | | [`workspace_heads.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/workspace_heads.rs) | Isolated Git workspace heads, Environment binding, and durable reopening | `cargo run -p everruns --features local --example workspace_heads -- /path/to/repo /path/to/state` | | [`canonical_events.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/canonical_events.rs) | Offline bounded recording and typed rendering of live events | `cargo run -p everruns --example canonical_events` | | [`explicit_communication.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/explicit_communication.rs) | Offline agent with explicit communication: sent messages apart from working notes | `cargo run -p everruns --example explicit_communication` | | [`framework_observability.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/framework_observability.rs) | Engine-wide OpenTelemetry and optional Braintrust export with explicit shutdown flushing | `cargo run -p everruns --features otel,braintrust --example framework_observability` | | [`subagents.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/subagents.rs) | Public facade composition for delegated work | `cargo run -p everruns --features openai --example subagents` | | [`observe_and_cancel.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/observe_and_cancel.rs) | Live events and cancellation | `cargo run -p everruns --features openai --example observe_and_cancel` | | [`advanced_capability.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/advanced_capability.rs) | Code-defined capability through the unified `capability(...)` entrypoint | `cargo run -p everruns --features openai --example advanced_capability` | | [`lifecycle_hooks.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/lifecycle_hooks.rs) | Awaited agent, turn, tool, and completion handlers | `cargo run -p everruns --features openai --example lifecycle_hooks` | Live-provider modes use `gpt-6.2-sol` and require `OPENAI_API_KEY`. `capability_configuration`, `canonical_events`, `direct_llm`, `engine_sessions`, `explicit_communication`, `live_session`, `session_work`, `workspace_heads`, `workspace_policy`, and `session_history` are fully offline; the GitHub monitor also offers a simulated GitHub flow:
```bash
cargo run -p everruns --features openai --example github_monitor -- --simulate
```
For copyable command details and behavior notes, use the [example catalog](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/README.md) next to the source. Examples that demonstrate low-level host internals remain advanced-host examples, not alternative Framework entrypoints. ## Provider-native batches The [batch guide](https://everruns.com/docs/framework/batch-model-requests/) shows offline work through Anthropic, OpenAI, Gemini, Mistral, Groq and Fireworks. Set `BATCH_PROVIDER` to the provider name, enable its Cargo feature and configure its API key. Both examples can incur charges. | Example | Behavior | Run | | ---------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | [`batch_classification.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_classification.rs) | Persist submission intent and ticket; resume classification, collect or cancel where supported | `cargo run -p everruns --features anthropic --example batch_classification -- submit /tmp/reviews-001` | | [`batch_eval.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_eval.rs) | Submit typed message lists, poll and score three eval cases | `cargo run -p everruns --features anthropic --example batch_eval` | ## Custom platform workers The [Custom Data Agent Worker](https://everruns.com/docs/framework/examples/custom-data-agent/) packages a Framework agent with private SQLite tools behind the restricted Platform worker protocol. Its complete smoke runner uses a real model, the ordinary session APIs and SSE, and checks replica replacement and credential rotation. This advanced host example needs a self-hosted CP; its shared history lives there. The batch examples also accept `BATCH_PROVIDER=mistral`, `groq` or `fireworks` with the matching Cargo feature. Fireworks requires account and artifact-origin configuration; see [Batch Model Requests](https://everruns.com/docs/framework/batch-model-requests/). ## Native GitHub PR review [`github_review.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/github_review.rs) reads a real PR with the native GitHub capability and prints an evidence-based review. It requires `OPENAI_API_KEY` and `GITHUB_TOKEN` and does not write to GitHub. See [GitHub tools](https://everruns.com/docs/framework/github/).
```bash
cargo run -p everruns --features github,openai --example github_review -- OWNER/REPO NUMBER
```
## Application services | Example | Required features | What it runs | | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------------------------------------- | | [OAuth MCP](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/mcp_oauth.rs) | `openai,mcp-auth` | Browser login, callback validation, authenticated MCP tool use | | [Slack actions](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/slack_actions.rs) | `openai,slack-actions` | React and upload into one authorized thread | | [Data services](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_data.rs) | `openai,sql-database,knowledge` | Retrieve policy, retain a SQL decision and write named memory | | [Native services](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_credentials.rs) | `openai,cursor,browserless,e2b` | Use one selected vendor with explicit credentials and resource cleanup | | [Event triggers](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_triggers.rs) | `openai,local,triggers` | Recoverable event dispatch and duplicate suppression across restarts | | [Evaluation and traces](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_evaluation.rs) | `openai,evaluation` | Run real cases, print scores and project canonical trace events | Every example uses a real model and reports failed turns. See [Application services](https://everruns.com/docs/framework/service-integrations/) for configuration and host responsibilities.
---
# Bashkit Repo Agent
> Modify and verify a disposable repository through a sandboxed shell.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/bashkit-repo-agent). Cut a release in a disposable repository through the sandboxed [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/), then verify every claimed change directly from the host. This is a real `gpt-6.2-sol` agent, not a scripted turn.  ## What you learn How to mount a narrow read-write workspace, give an agent one execution capability, accept an interactive task, and treat independent host assertions as the success condition. ## Scenario and expected outcome The bundled fixture is a two-crate Cargo workspace with three changelog fragments. The agent must cut release `0.2.0`: update both package versions and the path-dependency pin, create today’s changelog section, preserve the `0.1.0` history, and remove the folded fragments. After the turn, Rust code reopens the real working copy and fails the process if any invariant is false. A confident model answer cannot make a broken release pass. ## Run it Clone the repository and run from its root:
```bash
export OPENAI_API_KEY="your-key"
cargo run -p everruns-bashkit-repo-agent
```
Type the task interactively:
```bash
cargo run -p everruns-bashkit-repo-agent -- --interactive
```
The default workspace is temporary and removed on exit. Pass a scratch directory to inspect the result afterwards:
```bash
cargo run -p everruns-bashkit-repo-agent -- /tmp/release-run
cargo run -p everruns-bashkit-repo-agent -- --interactive /tmp/release-run
```
The configured model is `gpt-6.2-sol`. Provider access and funded credits are required; missing credentials, unsuccessful turns, and failed disk assertions exit nonzero. Never pass a repository you care about: the example materializes its fixture into the target and the agent may change anything inside the mount. ## Build the agent Read-write access is explicit—the default workspace policy is read-only.
```rust
pub fn build(provider: impl Into, workspace: &Path) -> Result {
Agent::builder()
.name("bashkit-repo-agent")
.instructions(include_str!("resources/instructions.md"))
.provider(provider)
.model(MODEL)
.max_iterations(12)
.workspace(workspace)
.workspace_policy(WorkspacePolicy::read_write())
.capability(BashkitShell::new())
.build()
}
```
## Run and verify The shared observer displays a bounded shell timeline and waits for a successful turn. The host then verifies the mounted files itself.
```rust
// OPENAI_API_KEY, declared by the OpenAI driver itself.
let agent = agent::build(everruns::OpenAI::from_env()?, &workspace)?;
let engine = Engine::new();
let session = engine.create(agent);
demo::run(&session, &request).await?;
verify_release(&workspace, &release_date)?;
```
Use `session.send_and_wait(&request).await?` when a live tool timeline is not needed. ## How Bashkit behaves Bashkit interprets shell scripts in-process against `/workspace`. The model gets no host filesystem outside that mount, network, Git credentials, or subprocess execution. Commands, loops, output, and script size are bounded. Repository text is treated as untrusted data rather than agent instructions. ## Validate it
```bash
cargo test -p everruns-bashkit-repo-agent
bash examples/bashkit-repo-agent/demo/record.sh --check
```
These checks validate argument handling, fixture state, host assertions, and the live-demo contract without provider credentials. They do not grade model quality. ## Demo and recording The screencast types the release task into the interactive binary, displays real `bashkit_shell` calls, and ends with host-side checks over the mutated files. It does not replay prepared output. Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/bashkit-repo-agent/demo/transcript.txt) at your own pace. With VHS, ffmpeg, a VHS-compatible browser, and funded OpenAI credentials:
```bash
bash examples/bashkit-repo-agent/demo/record.sh
```
The script uses `OPENAI_API_KEY` when exported, otherwise Doppler project `everruns-dev`, config `dev`. It updates the GIF and transcript only after the model turn and host verification succeed. ## Adapt it safely Replace the fixture, request acceptance criteria, and verifier together. Keep the mount disposable and narrow, start read-only unless mutation is required, and verify consequential claims outside the model/tool boundary. ## Boundaries This demonstrates one sandboxed repository mutation, not a general coding agent. It cannot fetch dependencies, run native programs, commit, or push. The Framework session is in-memory.
---
# Coding Review Agent
> Tool-based code inspection, a tightly scoped execution tool, and distinguishing test failure from tool failure.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/coding-review-agent). Review a refund implementation against a written contract, then execute a bundled regression test before reporting a defect. A proposed test is not treated as proof: the agent gets actual compiler and test output. ## What you learn Tool-based code inspection, a tightly scoped execution tool, and distinguishing test failure from tool failure. ## Scenario and expected outcome Two refunds of 1,000 cents are requested against a 1,000-cent payment. Each individual call caps its own amount, but the implementation retains no cumulative refunded balance. The regression fails with `left: 2000` and `right: 1000`. The review should connect that observed failure to the missing cumulative-refund state, describe the over-refund risk, and propose a minimal fix. It must not claim the code was changed or fixed. ## Run it Install Rust/Cargo (including rustc), clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient.
```bash
git clone https://github.com/everruns/everruns.git
cd everruns
export ANTHROPIC_API_KEY="your-key"
cargo run -p everruns-coding-review-agent
```
The configured model is `claude-sonnet-5`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question:
```bash
cargo run -p everruns-coding-review-agent -- "Read the contract and test. Reproduce the cumulative refund bug and explain the smallest safe fix."
```
## Build the agent This builder combines the agent instructions, tools, and model. Tools/capabilities supply evidence and actions; the model chooses how to use them.
```rust
let agent = Agent::builder()
.name("coding-review-agent")
.instructions(include_str!("instructions.md"))
.provider(everruns::Anthropic::from_env()?)
.model(MODEL)
.max_iterations(12)
.tool(tools::inspect_change())
.tool(tools::run_regression())
.build()?;
```
## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline.
```rust
let engine = Engine::new();
let session = engine.create(agent);
println!("MODEL: {MODEL}");
demo::run(&session, question).await?;
```
This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `inspect_change` can read only `sample_payment.rs`, `contract.md`, and `regression.rs`. `run_regression` invokes `rustc --test` on that fixed trusted fixture, runs the temporary binary, and returns its exit code and assertion output. Compilation and execution have timeouts; temporary files are removed automatically. ## Validate the behavior
```bash
cargo test -p everruns-coding-review-agent
python3 examples/coding-review-agent/src/render_demo.py --check
```
The offline test compiles and runs the bundled regression and asserts the observed failure. `cargo test -p everruns-coding-review-agent` passes because it verifies reproduction of the intentionally buggy fixture. A failing subprocess is expected evidence, not a failing example test. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording  Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/coding-review-agent/src/demo.txt) for the full recorded output. The GIF shows excerpts from the run. With credentials exported and Python 3, VHS, ffmpeg, and a VHS-compatible browser installed:
```bash
cd examples/coding-review-agent
bash src/record.sh
```
The script captures a successful run, generates correctly wrapped pages and page durations, and renders `src/demo.gif`. It preserves the previous transcript when the provider run fails. To replay an existing transcript without another model call, run `(cd src && python3 render_demo.py && vhs demo.tape)`. `src/demo.txt` retains the displayed output; `.demo-pages/` is generated and ignored. Inspect results before sharing: public/demo data is safe here, but adapting tools may expose private data. ## Adapt it Replace the fixture with a trusted review checkout and a restricted test selection. Use a sandbox before accepting arbitrary repositories, generated code, or model-selected commands. Add a second verified run after applying a fix in a separate approved workflow. ## Boundaries This is one deliberately buggy, trusted fixture—not a general-purpose coding agent. The execution tool takes no shell command or user-supplied path, and cannot edit code. Rust including `rustc` must be installed.
---
# Custom Data Agent Worker
> Package a Framework agent with private database tools and connect it to an Everruns control plane while keeping the ordinary chat and tracing surfaces.
Source:
This example turns an ordinary Framework agent into an experimental customer-hosted worker. Its `sales_summary` function reads its own SQLite database. An OpenAI `gpt-6.2-sol` model chooses the tool arguments and composes the answer. Users talk to the Everruns control plane, not to the worker directly. Follow [Run a custom agent worker](https://everruns.com/docs/how-to/custom-agent-workers/) for complete provisioning, TLS, credential rotation and smoke-test instructions. The [source and README](https://github.com/everruns/everruns/tree/main/examples/custom-data-agent) contain the executable and container packaging. ## Agent code and invocation The guide includes the [complete private database tool and Agent builder](https://everruns.com/docs/how-to/custom-agent-workers/#define-the-custom-agent-and-tool), [database seed code](https://everruns.com/docs/how-to/custom-agent-workers/#4-start-the-data-agent), and [ordinary session API calls](https://everruns.com/docs/how-to/custom-agent-workers/#invoke-it-through-the-cp) to create a conversation and send a message. The Agent is listed in **Agents**; operator provisioning binds that identity to the worker. Starting this executable does not register an Agent automatically. For Slack, use the same Agent’s **Channels and triggers** tab and Direct communication. Read the [channel support and limitations](https://everruns.com/docs/how-to/custom-agent-workers/#connect-the-same-agent-to-slack) before deployment: basic CP reply routing does not install native Slack tools, approvals or delegated-task behavior into this implementation, and live custom-worker Slack qualification is still required. ## Follow the code | Source | What it owns | | -------------------- | ------------------------------------------------------------------------------------------------------------------- | | `src/main.rs` | Worker lease loop, CP journal reads, Framework agent/tool composition, canonical event forwarding and cancellation. | | `src/transport.rs` | Application-owned OpenAI transport with a fixed HTTPS origin, redirect refusal and deployment proxy support. | | `smoke.py` | End-to-end verification against a real CP, built-in worker and model, using either a process or container. | | `Dockerfile` | Source-built release image. | | `Dockerfile.runtime` | Package an already-built Linux binary against a compatible runtime for smoke/CI use. | The worker opens the database with SQLite’s read-only flag. Model input supplies only the region, bound as a query parameter. It cannot choose a file path or send arbitrary SQL. For a real data service, apply your application’s own authorization before querying and return only fields that session readers may see. ## State and events Each lease reads the canonical CP event log and selects the next unconsumed user input. The example reconstructs conversation context in a fresh in-memory Engine. The CP remains the durable record; a second private Engine checkpoint is not claimed. Completed final answers can finish after a crash without rerunning the model. Other interrupted reads can replay under the same CP turn, with a bounded attempt count. The adapter subscribes before sending to the Framework. It forwards generations, tool calls, streaming output and terminal events, replaces local session/input correlation with CP correlation and reconstructs streamed text prefixes omitted by the facade. A lost observer event is an error, not a successful acknowledgment. The server scopes trace ids to the deployment/session/input and sends events through the normal EventService, so existing SSE, work and trace consumers apply. Every exit path cancels local execution. Renewal happens during execution and cancellation is checked against the input it targets. The worker acknowledges only the completed input; input arriving later remains queued. ## Run and verify After provisioning the CP binding and private database:
```bash
cargo run -p everruns-custom-data-agent
```
Credentials and endpoints come from the deployment environment, as documented in the how-to. The [full smoke test](https://everruns.com/docs/how-to/custom-agent-workers/#5-run-the-full-smoke-test) checks three real-model turns, live SSE, process/container replacement, changed private data, credential revocation and rotation. Offline tests cover the SQL boundary, streaming projection and fixed-origin transport. Neither fixture-only tests nor a successful compilation establish an end-to-end deployment. This is a deployment example for advanced hosts. It uses the internal protocol and canonical event contracts in addition to the public application facade. Pin the server and worker to the same revision. It is not native durable Engine hosting, a general-purpose SQL agent, a sandbox, or a Cloud image-hosting service.
---
# Durable multiplayer chat on Cloudflare
> Build a shared chat room with ordinary Everruns Agents and Engine sessions inside Cloudflare Durable Objects, with forks, reminders and history search.
Source:
This walkthrough recreates [Earendil’s multiplayer chat tutorial](https://earendil.com/posts/pi-durable-on-cloudflare/) with ordinary Everruns Framework Agents. Conversations and search children run inside native Cloudflare Durable Objects. SQLite journals and durable alarms keep their work independent of connected browsers. The [chat example](https://github.com/everruns/everruns/tree/main/examples/cloudflare-durable-chat) uses the shared Engine session-hosting profile and OpenRouter driver. This is experimental hosting; read [Durable Object hosting](https://everruns.com/docs/framework/durable-objects/) for its guarantees. ## See the demo [Open the Cloudflare demo](https://everruns-durable-chat.everruns.workers.dev/), create a conversation, and copy the room link into a second browser. [Durable multiplayer chat with Everruns on Cloudflare](https://www.youtube-nocookie.com/embed/eFmwod7GTrY?rel=0) [Watch on YouTube](https://www.youtube.com/watch?v=eFmwod7GTrY).  Alice and Bob see the same committed conversation. **Copy room link** invites another browser; **Fork here** starts a new conversation from a committed message. ## Run the chat Clone the whole repository because the example uses local workspace crates. Install Rust 1.96 and Node.js 24, then:
```sh
git clone https://github.com/everruns/everruns.git
cd everruns/examples/cloudflare-durable-chat
rustup target add wasm32-unknown-unknown
cargo install wasm-bindgen-cli --version 0.2.129 --locked
npm ci
npm run build
cp .dev.vars.example .dev.vars
```
Set `OPENROUTER_API_KEY` in `.dev.vars`. Start the local Worker:
```sh
npm run dev
```
The default model is [Muse Spark 1.3 Contributor](https://openrouter.ai/meta/muse-spark-1.3-contributor) through OpenRouter. Provider charges apply; Contributor uses submitted data for training. `MODEL` and `OPENROUTER_BASE_URL` are deployment settings. Open the local URL, choose a name and create a conversation. Copy the room link into another browser. Both participants see committed messages. Record a decision in one conversation and ask about it in another to exercise the search child. Ask for a reminder in 30 seconds, close the browsers, and return after the deadline. ## Framework execution, application services  A room is addressed through `/rooms//...`. It stores the conversation list, command receipts and reminders. Each conversation and history-search child has its own Durable Object, SQLite journal and alarm. The app registers two ordinary Agents: `durable-chat` and `history-search`. Both use `Engine::builder().hosting(ObjectHosting::new(...))`. The child has only the search tool; the main agent has scheduling and history-search tools. Framework owns the model/tool loop, lifecycle, event history, interruption recovery, iteration limits and completion metadata. `ObjectHosting` connects Engine to the Durable Object’s storage and alarms. Engine runs as WebAssembly, with the Worker providing storage, network requests and wakeups. Each session keeps the model selected when it was created; changing `MODEL` affects new sessions. The shared OpenRouter driver handles Responses request encoding, streaming, provider options, retries, error mapping and usage metadata. The host adapter supplies `fetch`, bounded response bytes, cancellation, endpoint policy and the server-side credential. ## Accept commands without losing work Every create, fork, send and schedule cancellation includes a stable `request_id`. The room stores a command fingerprint and receipt. A matching retry returns the same receipt; reusing the ID for another command is rejected. Before accepting a mutation, the room arms its alarm. It then commits the receipt and outbox intent in one SQLite document update, without yielding between them. The room alarm delivers pending intents through Framework. Message delivery uses `Session::send_with_id` with an identity derived from the durable outbox entry. A crash after admission but before deleting the outbox entry resends the same identity, so Framework deduplicates it. The room receipt acknowledges a durable delivery obligation. Framework’s own admission receipt is the authority for session execution. A room outbox is application delivery state; it does not replace the actor journal. ## Fork a committed prefix A fork reads canonical events through Framework and selects committed user and assistant messages through the chosen boundary. It creates another registered session and uses the validated empty-history seed port to import that prefix. Pending tools, schedules and execution ownership are not inherited. A tool-call message cannot be used as a fork boundary. The room’s event projection serves rendering, search and fork selection. Framework constructs model context from the session’s canonical history. ## Schedule reminders without a browser The scheduling tool persists an instruction, deadline and optional recurrence. Its receipt is keyed by Framework’s tool execution identity, so retrying the same operation returns the existing schedule. The room alarm advances a due occurrence and records its delivery intent in the same write. Delivery enters the target session through normal Framework admission. An occurrence has a stable message identity. Repeated delivery after a crash does not create another admission. Missed recurring ticks collapse into one follow-up. Cancellation disables future occurrences; it does not retract an already admitted instruction. Schedules are an explicit application service here. A durable session journal alone does not provide a scheduler or room directory.  ## Recover a history-search child `search_history` creates a child keyed by the parent tool execution identity. That identity distinguishes different assistant tool-call messages even when a provider reuses a call ID. The child is an ordinary Framework session with its own durable admission and alarm. It can query room history through `search_conversations`, returning conversation IDs and sequence numbers.  **Decision lookup** retrieves the choice recorded in **Launch plan**. The sidebar also shows **Fork of Launch plan** and the independent **History search** children. The tool result carries source conversation IDs and sequence numbers. A parent interrupted while waiting can repeat this idempotent tool. It finds the same child and reads its reconstructible admission outcome; it does not create another child. Framework handles the interrupted tool and actor recovery. The room only stores application ownership and delivery state. Search children and search-generated assistant answers are excluded from future search evidence. Retrieved transcripts are untrusted evidence, not instructions. ## Deploy and verify
```sh
npm run build
npm test
npm run check:bundle
npx wrangler deploy --dry-run
npx wrangler login
npm run deploy
npx wrangler secret put OPENROUTER_API_KEY
```
The public Worker handles browser requests. Room and session Durable Objects communicate through private bindings, and the OpenRouter API key stays on the server. Public access and model costs This example has no sign-in, room access controls or per-user quotas. Anyone with its URL can open rooms and send messages using the configured model budget. Add access and usage controls before sharing your own deployment. The test suite covers shared conversations, forks, interrupted history searches and scheduled reminders across Worker restarts. It runs locally with workerd and simulated model responses. See [Durable Object hosting](https://everruns.com/docs/framework/durable-objects/) for hosting guarantees and upgrade compatibility, and [Cloudflare’s platform limits](https://developers.cloudflare.com/workers/platform/limits/) for deployment constraints.
---
# Everruns Support Agent
> Search and read citable documentation before answering Framework questions.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/everruns-support-agent). Answer Framework questions by searching and reading a small, inspectable corpus of official documentation. The agent retrieves evidence before answering instead of routing keywords to prewritten responses.  ## What you learn A two-step search/read tool interface, citable evidence, and an explicit boundary around the available knowledge. ## Scenario and expected outcome The default question asks how to resume a durable session after restarting a process. The agent should search for persistence and session history, read the relevant pages, then explain the local catalog, persisted `SessionId`, and agent reattachment requirements with source URLs. It must not imply that `Engine::new()` survives a restart. When the corpus lacks an answer, it should say so instead of inventing one. ## Run it Install Rust/Cargo, clone the repository, and run from its root. This folder is self-contained **within the workspace**: its Cargo manifest references local Framework crates, so copying the folder alone is not sufficient.
```bash
git clone https://github.com/everruns/everruns.git
cd everruns
export ANTHROPIC_API_KEY="your-key"
cargo run -p everruns-framework-support-agent
```
The configured model is `claude-opus-5-5`. Provider access and funded credits are required. Keep keys in your environment, not in source control. Missing credentials, provider errors, and unsuccessful turns exit nonzero. Try another question:
```bash
cargo run -p everruns-framework-support-agent -- "How do I register a custom provider?"
cargo run -p everruns-framework-support-agent -- "How can a tool return a structured error?"
```
Or type a question interactively:
```bash
cargo run -p everruns-framework-support-agent -- --interactive
```
## Build the agent Tools retrieve evidence; Opus decides what to search, read, and explain.
```rust
pub fn build(provider: impl Into) -> Result {
Agent::builder()
.name("everruns-support-agent")
.instructions(include_str!("resources/instructions.md"))
.provider(provider)
.model(MODEL)
.max_iterations(12)
.tool(tools::search_docs())
.tool(tools::read_doc())
.build()
}
```
## Send, observe, and wait The shared demo helper subscribes before sending, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. Use `session.send_and_wait(&question).await?` when a live tool timeline is unnecessary.
```rust
// ANTHROPIC_API_KEY, declared by the Anthropic driver itself.
let agent = agent::build(everruns::Anthropic::from_env()?)?;
let engine = Engine::new();
let session = engine.create(agent);
println!("MODEL: {}", agent::MODEL);
demo::run(&session, &question).await?;
```
This engine is in-memory. The agent can explain durable sessions from its corpus, but the example itself does not persist its session. ## How the tools work `search_docs` ranks matches against five bundled pages and returns page IDs, public URLs, and matching excerpts. `read_doc` accepts only those page IDs and returns the complete snapshot. It never accepts an arbitrary filesystem path. ## Validate the behavior
```bash
cargo test -p everruns-framework-support-agent
bash examples/everruns-support-agent/demo/record.sh --check
```
Tests cover content-based search, empty and unmatched queries, complete citable reads, arbitrary-path rejection, and interactive input validation. They do not grade the model’s answer; compare a live response with the expected outcome above. CI runs the offline checks without provider credentials. Live model behavior is evaluated separately. ## Demo and recording Watch the screencast or read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/everruns-support-agent/demo/transcript.txt) to see the tool calls and answer. With credentials exported and VHS, ffmpeg, and a VHS-compatible browser installed:
```bash
bash examples/everruns-support-agent/demo/record.sh
```
Export `ANTHROPIC_API_KEY` before recording. The script updates `demo/demo.gif` and `demo/transcript.txt` after a successful run. ## Adapt it Replace the bundled pages with a versioned documentation index while retaining separate search and read operations. Attach source/version metadata, restrict reads to authorized documents, and treat retrieved text as untrusted evidence rather than instructions. ## Boundaries The corpus is a five-page snapshot from 2026-09-08, not a live search of the docs site. Provenance is recorded in the [corpus README](https://github.com/everruns/everruns/blob/main/examples/everruns-support-agent/src/resources/docs/README.md), and the snapshot can lag current APIs.
---
# Foreman
> A decision service supervising a coding agent it never has to stop.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/foreman-agent). A fast decision service watching a slow coding agent, and a policy in ordinary Rust deciding what to do about the numbers. A Framework port of [thruwire/foreman](https://github.com/thruwire/foreman), which placed [TypeSafe’s Jev](https://docs.typesafe.ai/introduction) above a Codex worker and asked whether semantic supervision can run *while* the work happens.  ## What you learn How to run a worker session and observe it at the same time: a [`Decisions`](https://everruns.com/docs/framework/direct-model-calls/#decisions) turning bounded evidence into nine probabilities in one request, and a deterministic policy that owns every threshold, every limit, and the closed vocabulary of things the supervisor may do. ## The two loops The worker keeps its own loop — an Everruns session, or an external CLI in a child process. `session.send` returns a receipt immediately, and a child process is simply left running; either way the supervisory loop reads the evidence beside the live work. Activity is debounced to a floor, and only a worker finishing bypasses it. Nothing stops for the factory to think, and a test holds that claim: it counts readings taken while a worker’s turn is unresolved and fails if supervision waits its turn. ## What it assesses Five questions describe the job (`implementation_complete`, `tests_sufficient`, `requirements_satisfied`, `needs_verification`, `ready_to_finish`) and four describe the floor right now (`meaningful_progress`, `worker_stuck`, `work_off_track`, `needs_human`). Each is a Noul — the probability that a yes/no statement is true — and all nine ride one request, because questions in a decision are answered independently and in parallel. ## What it may do about them The decision service only estimates. The policy decides, safety and hard limits before productivity: escalate when a person is needed or the iteration ceiling is reached, stop a worker that is off track or stuck, retry once after a stop, finish when the completion thresholds hold and verification is resolved, start one independent verifier when a check is warranted, otherwise start or continue work. Thresholds and limits are Foreman’s defaults and are overridable through `FOREMAN_*` environment variables. ## What it looks at Never the repository. One bounded snapshot per reading: worker status, elapsed time, tool calls and output tails, `git status`, a bounded `git diff`, the untracked paths a diff cannot show, recent session events, verification results, and the previous assessment and decision. An unbounded observation would make supervision as slow as the work it is watching. That snapshot goes to the decision service on every reading, so a bounded slice of the repository leaves the machine on every run — point `--repo` at a private repository only if that is acceptable for it. `demo` works on a fixture it materializes itself, so it carries nothing of yours. The repository content in an observation is also untrusted input to the decision service, and that it can only produce a number is the point: the decision service never names an action, and every action the policy can take is in one readable file. ## Run it Foreman’s own two entry points, and they mean the same things here:
```bash
cargo run -p everruns-foreman-agent --bin foreman -- demo
foreman run --repo ./my-project --job "Add rate limiting, and test it."
```
Both commands use the same worker, decision service, and credentials. They differ in the repository and job they use: | Command | Repository | Job | Needs | | --------------- | ------------------------------------------- | ----------------------- | --------------------------------- | | `foreman demo` | a bundled fixture, in a temporary directory | one it ships with | `TYPESAFE_API_KEY` + the worker’s | | `foreman run …` | yours, named by `--repo` | yours, named by `--job` | the same | `demo` exists because a fixed starting state makes the ending checkable: the job names a rate schedule, so at the end the repository either prices by weight or it does not. Those checks run at the bottom of the run and read the files, not the supervisor’s opinion of them. `demo` writes its fixture into a temporary directory unless `--repo` says otherwise, and `run` never writes a fixture at all — `--repo` is your project, and the only thing that touches it is the worker. It will be modified. ## One supervisor The supervisor uses `Decisions` to ask a model nine questions in one request, within a fixed budget. Every run requires provider credentials; there is no offline mode. ## The supervisor runs the tests itself It already runs `git` rather than asking the worker what changed, and `--tests` applies the same reasoning to the suite: a worker reporting its own green tests is a claim, and an independently run result is a fact. The command runs in a networkless Docker container with no inherited host environment, over a read-only repository mount copied to isolated temporary storage. Docker is therefore required when `--tests` is used. The result lands in the observation as `test_results` — the field Foreman declares and never fills — and `tests_sufficient` moves on it. The suite runs once as a baseline before any worker starts, then whenever the floor is quiet; between times the last result is carried with its age, because a stale pass should not read as a fresh one. ## Who does the work `--worker` picks the crew. All three are watched identically, because what the supervisor reads is a bounded observation and the strongest evidence in one — the repository’s own diff — is gathered by the host either way. | `--worker` | What runs | Independent verification | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- | | `session` (default) | An Everruns session on the [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/), `meta/muse-spark-1.3-contributor` through OpenRouter | A second session under the default read-only workspace policy | | `codex` | `codex exec --cd … --sandbox workspace-write --color never --json …`, the line Foreman itself runs | The same CLI with `--sandbox read-only` | | `yolop` | `yolop -C … -p …`, its one-shot print interface | Mission only — yolop publishes no read-only mode | Anything else is a template: `--worker-command "mycli --cd {repo} --task {mission}"`, where both placeholders are substituted as whole arguments so no shell sees either. A session is observed through its own canonical event stream, which arrives already typed. A CLI offers none of that, so an external worker is observed through stdout and stderr, and a JSONL line that names its own `type` counts as a step. ## The floor The bundled fixture is a small shell project that prices every parcel at one flat rate; the job is to replace that with weight tiers and cover the boundaries. Shell, deliberately: `bash tests/run.sh` needs no framework, no interpreter and no network, so the same suite runs inside the Bashkit sandbox, on the host, and inside an external agent. On the session crew the coding worker mounts it read-write and the verifier mounts the same directory under the default read-only policy, so “independent check” is a property of the mount rather than a request in a prompt. The supervisor runs `git` itself rather than asking the worker what it did, which is also why an external CLI is supervised just as well as a session. ## What the Framework changes The architecture is Foreman’s; the runtime underneath it is not. A worker is a session rather than a subprocess, so stopping one is a cooperative turn cancellation instead of a signal to a process group. Evidence is the canonical event stream rather than parsed JSONL. A retry is a fresh session over the same workspace. Read-only verification is a workspace policy. Steering exists — sending into a live turn applies at the next iteration boundary — and is deliberately left out of the policy’s vocabulary so the comparison with the original stays honest. ## Limits This is an architectural experiment, and porting it does not make it a proven one. Decisions accuracy for this use is unproven and the thresholds are uncalibrated: false positives stop useful workers, false negatives let bad work continue. Observations are bounded and therefore incomplete. One coding worker runs at a time, a verifier reports evidence rather than proof, and the session state is in memory.
---
# Host Shell Agent
> Fix a failing Rust test suite by compiling and running it, inside a kernel-enforced boundary.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/host-shell-agent). Fix a failing Rust test suite by compiling and running it, through the [Host Shell](https://everruns.com/docs/capabilities/host-shell/) capability: real processes on this machine, bounded by a kernel policy. The agent uses `gpt-6.2-sol`. ## What you learn How to give an agent the machine it is running on without giving it the machine: one execution capability, a kernel-enforced write and network boundary, and independent host assertions as the success condition. This is the example the sandboxed [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/) cannot be. Compiling a crate and running its test binary needs a real toolchain and real child processes; an in-process interpreter over a virtual filesystem has neither. ## Scenario and expected outcome The bundled fixture is a small zero-dependency crate whose `chunk_count` drops the partial final chunk, so two tests fail. The agent must run the suite, read the failure, fix the source, and re-run until it passes. Before the agent starts, the example probes the same containment provider the capability uses and prints what it found: a write inside the workspace succeeds, a write outside it is refused by the kernel, an outbound socket is refused by the kernel. Not a description of the boundary, the boundary. After the turn, Rust code re-runs `cargo test` from the host and fails the process if the suite is still red, if an assertion was edited away, or if a test was marked `#[ignore]`. A confident model answer cannot make a red suite pass. ## Run it Clone the repository and run from its root:
```bash
export OPENAI_API_KEY="your-key"
cargo run -p everruns-host-shell-agent
```
Type the task interactively, or point it at a directory you keep:
```bash
cargo run -p everruns-host-shell-agent -- --interactive
cargo run -p everruns-host-shell-agent -- /tmp/chunker
```
With no directory, the fixture is materialized into a fresh temporary one and discarded afterwards. ## Requirements Kernel containment needs macOS, or Linux with Landlock ABI v3 fully enforced (Linux 6.2 or a backport). On a kernel that cannot enforce it, the provider fails closed and the example says so instead of running uncontained. On Linux the policy is applied by a helper process, and this example is its own: `main` routes a `__sandbox-exec` re-exec into the containment worker and names that through `SandboxLauncher::ReexecSelf`, which is what a single-binary embedder does. Nothing else needs to be on disk.
---
# Incident Commander Agent
> Multi-tool evidence gathering, distinguishing correlation from causation, and a narrowly scoped append-only side effect.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/incident-commander-agent). Investigate a fictional checkout alert using contrasting metrics, deployment history, logs, and a runbook; then persist an evidence-backed status update. The agent must investigate before recommending action. ## What you learn Multi-tool evidence gathering, distinguishing correlation from causation, and a narrowly scoped append-only side effect. ## Scenario and expected outcome Checkout errors rise from 0.2% to 18.4% after v43 reduces the payment timeout from 2,000 ms to 200 ms. Catalog traffic and database metrics stay normal; sampled logs show requests timing out just beyond the new deadline. Read metrics, deployments, logs, and the runbook. Identify the timeout reduction as a likely cause, not proven causation. Propose on-call ownership and approval for rollback consideration. Record that conclusion locally without claiming any production mitigation occurred. ## Run it Install Rust/Cargo, clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient.
```bash
git clone https://github.com/everruns/everruns.git
cd everruns
export MODEL_API_KEY="your-key"
cargo run -p everruns-incident-commander-agent
```
The configured model is `muse-spark-1.3`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question:
```bash
cargo run -p everruns-incident-commander-agent -- "Investigate checkout and explain what evidence argues against a database-wide incident. Record a concise update."
```
## Build the agent This builder combines the agent instructions, tools, and model. Tools/capabilities supply evidence and actions; the model chooses how to use them.
```rust
let agent = Agent::builder()
.name("incident-commander-agent")
.instructions(include_str!("instructions.md"))
.provider(everruns::Meta::from_env()?)
.model(MODEL)
.max_iterations(12)
.tool(tools::inspect_evidence())
.tool(tools::record_incident_update())
.build()?;
```
## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline.
```rust
let engine = Engine::new();
let session = engine.create(agent);
println!("MODEL: {MODEL}");
demo::run(&session, question).await?;
```
This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `inspect_evidence` exposes only four named fixture categories. `record_incident_update` appends a non-empty update of at most 500 UTF-8 bytes to this example’s `src/incident.log`, normalizing newlines. There is deliberately no production rollback tool. ## Validate the behavior
```bash
cargo test -p everruns-incident-commander-agent
python3 examples/incident-commander-agent/src/render_demo.py --check
```
Tests verify that updates survive multiple writes, oversized/empty updates are rejected before a file is created, and evidence access is scoped. A live run should show all evidence reads followed by a persisted update that matches the observed facts. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording  Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/incident-commander-agent/src/demo.txt) for the full recorded output. The GIF shows excerpts from the run. With credentials exported and Python 3, VHS, ffmpeg, and a VHS-compatible browser installed:
```bash
cd examples/incident-commander-agent
bash src/record.sh
```
The script captures a successful run, generates correctly wrapped pages and page durations, and renders `src/demo.gif`. It preserves the previous transcript when the provider run fails. To replay an existing transcript without another model call, run `(cd src && python3 render_demo.py && vhs demo.tape)`. `src/demo.txt` retains the displayed output; `.demo-pages/` is generated and ignored. Inspect results before sharing: public/demo data is safe here, but adapting tools may expose private data. ## Adapt it Replace each fixture with a read-only monitoring/deployment API scoped to the relevant service. Keep investigation separate from action. Add explicit approval and an audit trail before introducing any production mutation. ## Boundaries All telemetry is fictional. The log is a real local append-only artifact, ignored by Git; it contains exercise text and does not change production. Filesystem permissions and rotation are the application’s responsibility.
---
# Research Agent
> Composing the first-party Brave Search and WebFetch capabilities, handling external evidence, and reporting uncertainty.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/research-agent). Search for relevant material, open primary sources, and synthesize a short, cited research brief. The model must read source content rather than treating search snippets as sufficient evidence. ## What you learn Composing the first-party Brave Search and WebFetch capabilities, handling external evidence, and reporting uncertainty. ## Scenario and expected outcome The default question asks what durable execution guarantees about retries and external side effects. A useful answer must distinguish replaying recorded results from the risk of retrying an unrecorded side effect. Read at least two primary sources successfully before answering. Cite those pages, distinguish documented guarantees from inference, and explain why idempotency can still be necessary. If a source is inaccessible, try another and disclose the gap. Search snippets alone do not meet the task. ## Run it Install Rust/Cargo, clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient.
```bash
git clone https://github.com/everruns/everruns.git
cd everruns
export OPENROUTER_API_KEY="your-key"
export BRAVE_SEARCH_API_KEY="your-key"
cargo run -p everruns-research-agent
```
The configured model is `z-ai/glm-5.2`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question:
```bash
cargo run -p everruns-research-agent -- "Compare retry guarantees in Temporal Activities and Restate. Read official sources and give two findings plus one caveat."
```
## Build the agent This builder combines the agent instructions, tools, and model. Tools/capabilities supply evidence and actions; the model chooses how to use them.
```rust
let agent = bound_external_calls(
Agent::builder()
.name("research-agent")
.instructions(include_str!("instructions.md"))
.provider(everruns::OpenRouter::from_env()?)
.model(MODEL),
)
.capability(BraveSearch::from_env()?)
.capability(everruns::WebFetch::new())
.build()?;
// Search and fetched pages are untrusted model input. Keep one injected result from
// multiplying paid provider requests or scheduling a batch of outbound calls.
fn bound_external_calls(builder: AgentBuilder) -> AgentBuilder {
builder
.max_iterations(MAX_ITERATIONS)
.parallel_tool_calls(false)
}
```
## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline.
```rust
let engine = Engine::new();
let session = engine.create(agent);
println!("MODEL: {MODEL}");
demo::run(&session, question).await?;
```
This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `BraveSearch::from_env()` provides `brave_web_search`. `WebFetch::new()` provides `web_fetch`, enabled through the Framework `web-fetch` Cargo feature. Search chooses candidates; fetch retrieves page content using the existing integration’s egress controls. Download-to-file is not enabled. ## Validate the behavior
```bash
cargo test -p everruns-research-agent
python3 examples/research-agent/src/render_demo.py --check
```
Offline tests cover evidence-preview rendering and recording pagination; they do not perform web research. Validate a live run by checking successful `web_fetch` results for at least two primary sources and matching the final citations to pages actually read. Network/provider behavior and factual quality are not guaranteed by a green offline test. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording  Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/research-agent/src/demo.txt) for the full recorded output. The GIF shows excerpts from the run. With credentials exported and Python 3, VHS, ffmpeg, and a VHS-compatible browser installed:
```bash
cd examples/research-agent
bash src/record.sh
```
The script captures a successful run, generates correctly wrapped pages and page durations, and renders `src/demo.gif`. It preserves the previous transcript when the provider run fails. To replay an existing transcript without another model call, run `(cd src && python3 render_demo.py && vhs demo.tape)`. `src/demo.txt` retains the displayed output; `.demo-pages/` is generated and ignored. Inspect results before sharing: public/demo data is safe here, but adapting tools may expose private data. ## Adapt it Narrow the research question and source policy, add an evidence store if results need to survive sessions, and validate citation coverage before publishing important findings. Treat fetched text as untrusted data, not as instructions. ## Boundaries Requires both OpenRouter and Brave Search credentials plus outbound HTTPS. Search and fetch can fail; six agent iterations cap the loop, not the bill. Word limits are instructions, not a hard output validator. This is a small research workflow, not an exhaustive literature review.
---
# Support Agent
> Typed read-only tools, separating facts from policy, and choosing different answers for different inputs.
Source:
[Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/support-agent). Diagnose a sign-in problem by combining account facts with an explicit recovery policy. The interesting decision is whether the user needs MFA recovery, must wait for a lockout, or should try a clean browser session.  ## What you learn Typed read-only tools, separating facts from policy, and choosing different answers for different inputs. ## Scenario and expected outcome The default customer reset their password, but has MFA enabled and neither an authenticator nor recovery codes. A password reset alone cannot solve this case. For `cust_mfa`, direct the user to verified identity recovery, explicitly noting that resetting a password does not disable MFA. Never request passwords or recovery codes. For `cust_locked`, explain the 15-minute wait. For `cust_browser`, suggest a private window and an escalation if it still fails. ## Run it Install Rust/Cargo, clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient.
```bash
git clone https://github.com/everruns/everruns.git
cd everruns
export OPENAI_API_KEY="your-key"
cargo run -p everruns-support-agent
```
The configured model is `gpt-6.2-sol`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question:
```bash
cargo run -p everruns-support-agent -- "cust_locked reset their password but cannot sign in. What should they do?"
cargo run -p everruns-support-agent -- "cust_browser cannot sign in after a reset. What next?"
```
Or enter a question interactively:
```bash
cargo run -p everruns-support-agent -- --interactive
```
## Build the agent Tools supply evidence; the model chooses how to use it.
```rust
pub fn build(provider: impl Into) -> Result {
Agent::builder()
.name("support-agent")
.instructions(include_str!("resources/instructions.md"))
.provider(provider)
.model(MODEL)
.max_iterations(12)
.tool(tools::lookup_customer())
.tool(tools::read_support_policy())
.build()
}
```
## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline.
```rust
// OPENAI_API_KEY, declared by the OpenAI driver itself.
let agent = agent::build(everruns::OpenAI::from_env()?)?;
let engine = Engine::new();
let session = engine.create(agent);
println!("MODEL: {}", agent::MODEL);
demo::run(&session, question).await?;
```
This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `lookup_customer` reads one of three fictional customer records. It returns facts, not a prewritten recommendation. `read_support_policy` returns the bundled recovery rules. The model combines the two; no tool disables MFA or changes a real account. ## Validate the behavior
```bash
cargo test -p everruns-support-agent
bash examples/support-agent/demo/record.sh --check
```
Tests cover the distinct account states and rejection of unknown IDs. They do not grade the model’s recommendation: compare a live response with the expected outcomes above. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording Watch the screencast or read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/support-agent/demo/transcript.txt) to see the tool calls and answer. With credentials exported and VHS, ffmpeg, and a VHS-compatible browser installed:
```bash
bash examples/support-agent/demo/record.sh
```
Export `OPENAI_API_KEY` before recording. The script updates `demo/demo.gif` and `demo/transcript.txt` after a successful run. ## Adapt it Replace the fixture lookup with your authorized customer-data service. Keep policy separate from account facts, scope lookups to the authenticated customer, and return only fields needed for the support decision. Add human approval before any account mutation. ## Boundaries All customers, policy rules, and support.example.com URLs are fictional. This is a read-only support exercise, not a live help desk.
---
# Typed Extractions
> Extract typed values and evidence from documents without an agent loop.
Source:
Extractions ask for named fields in text or images and return typed values, evidence and uncertainty. They are separate from Decisions (classification) and structured chat (schema-shaped generated output). The Framework exposes the same contract as the Server. No agent, session or tool loop is required. The upstream OpenRouter API is **alpha**. Providers differ in supported fields and assets; an unsupported request fails rather than silently switching to chat or inventing confidence. ## Framework
```rust
use everruns::{Extractions, ExtractionField};
async fn example() -> Result<(), Box> {
let key = std::env::var("OPENROUTER_API_KEY")?;
let provider = everruns_drivers::openrouter::provider("openrouter", key);
let result = Extractions::new("cloudflare/clairic", provider)
.about("Invoice INV-42. Total USD 120.00.")
.field("invoice_number", ExtractionField::text())
.field("total", ExtractionField::number())
.send().await?;
println!("{:?}", result.answers);
Ok(())
}
```
Enable the driver’s `openrouter` feature. Run the synthetic invoice example:
```bash
OPENROUTER_API_KEY=... cargo run -p everruns --features utility-llm --example direct_extractions
```
This sends a paid provider request. Set `EXTRACTION_MODEL` to another extraction model supported by your account. Do not use real private documents in tests without authorizing their transmission. For direct Cloudflare access, construct `everruns_drivers::cloudflare::provider` with your account ID and API token and select `@cf/cloudflare/clairic`. It uses Workers AI directly, without an OpenRouter credential. The Cloudflare driver accepts text and images, not PDFs. Run `cargo run -p everruns --features cloudflare --example direct_cloudflare_extractions` with `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN` to try the synthetic invoice directly (provider charges may apply). Other upstream vendors are available through OpenRouter; they are not advertised as direct integrations here. ## Server and console Configure an organization-owned provider key and add or sync an extraction-capable model. In Platform, first enable **Extractions** in **Settings → Features** for the organization. Open **Extractions** in the console to supply text and field specifications, then inspect values and evidence. Inline assets are supported through the API; the initial console does not include an asset upload widget. Set `EVERRUNS_URL` to the API base URL, for example `http://localhost:27100/api` for a local stack.
```bash
curl "$EVERRUNS_URL/v1/extractions" \
-H "Authorization: Bearer $EVERRUNS_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"model_id":"YOUR_MODEL_ID","input":{"state":"Invoice INV-42","fields":{"invoice_number":{"type":"text"}}}}'
```
The API requires model-management permission and an enabled extraction model in the authenticated organization. Read-only credentials cannot make paid calls. This initial endpoint supports organization BYOK providers; hosted/billed models are rejected until their extraction accounting path is available. Model/provider overrides inside `input` are rejected. Credentials stay on the server. Requests carry no arbitrary remote document URL: assets are inline base64. OpenRouter transport additionally caps fields at 128, state at 2 MiB and encoded assets at 12 MiB as client-side safeguards, not vendor limit claims. The contract caps assets at 32 and their aggregate encoded data at 32 MiB before decoding. Driver-specific limits may be lower; the Server limits the whole request body to 8 MiB. ## Values, absence and evidence Field types are `text`, `number`, `date`, `time`, `datetime`, `duration`, `list`, `estimate` and `forecast`. `ExtractionField` validates type-specific options before network access. The outcome preserves provider usage, `truncated`, optional state text and every returned field’s value. * `value: null` is a valid absent answer. Do not replace it with `guess` automatically. * Confidence and `p_null` are optional provider observations, not calibrated guarantees. Missing confidence means unknown, not zero or one. * `span` and per-item `spans` use Unicode **code points**, not byte or JavaScript UTF-16 offsets. In JavaScript use `Array.from(state).slice(start, end).join('')`. A list may have null span entries. * Use returned `state_text` when present. Truncation means the model did not read the whole input. * Estimates and forecasts preserve distribution statistics; they are not exact source facts. * Image/document answers may have no textual span. Do not fabricate evidence. ## Deployment-owned utility With the Framework’s `utility-llm` feature, `utility_extractions::utility_extractions_from_env()` selects the explicit `UTILITY_EXTRACTION_MODEL` and `UTILITY_OPENROUTER_API_KEY`. With no extraction model configured, it is disabled. Tenant extraction endpoints never fall back to these deployment credentials. ## Document conversion and Markkit Keep conversion separate from inference. Convert a local document to Markdown with Markkit, review the text, then pass that text as extraction state. This preserves Markkit’s deterministic offline boundary and keeps provider credentials in Everruns. The example accepts a reviewed Markdown file from a separate converter:
```bash
EXTRACTION_INPUT_FILE=reviewed-invoice.md OPENROUTER_API_KEY=... \
cargo run -p everruns --features utility-llm --example direct_extractions
```
This explicitly sends that file’s text to the provider. For scanned documents, use a provider-supported asset input instead: conversion quality and OCR are not guaranteed by the extraction contract. ## Verification Unit and mock-HTTP tests cover typed validation, malformed answers, Unicode spans, auth/tenant checks and sanitized errors. The live qualification harness is opt-in, uses synthetic documents and requires separate credentials and an explicit spending decision. A mock pass does not prove vendor availability or extraction accuracy. After explicitly approving provider spending and setting the provider credentials, run the synthetic-document live qualification harness:
```bash
EVERRUNS_EXTRACTIONS_LIVE=1 EXTRACTIONS_LIVE_PROVIDER=openrouter \
cargo bench -p everruns-drivers \
--no-default-features --features openrouter,cloudflare \
--bench extractions_live
```
Select `openrouter` or `cloudflare` with `EXTRACTIONS_LIVE_PROVIDER`; do not supply credentials for a provider you do not intend to call. See [OpenRouter’s extraction contract](https://openrouter.ai/docs/guides/community/extractions) and [Cloudflare Clairic](https://developers.cloudflare.com/workers-ai/models/clairic/) for current limits.
---
# GitHub tools
> Read pull requests and repository files, search GitHub, and explicitly enable write tools in a Framework application.
Source:
Give an agent GitHub tools with `everruns::github::GitHub`. The capability uses the same GitHub operations as the Platform, with credentials your application supplies. It requires no Everruns server or database. Until these APIs are included in the next crate release, use the repository’s `main` branch or run the examples from a source checkout:
```toml
[dependencies]
everruns = { git = "https://github.com/everruns/everruns", branch = "main", features = ["github", "openai"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
```
The `github` feature is optional. It adds the GitHub HTTP integration without enabling other vendors or the hosted capability catalog. ## Read a pull request Set `GITHUB_TOKEN` with repository read access and `OPENAI_API_KEY`, then run:
```rust
use everruns::{Agent, Engine, OpenAI, github::GitHub};
#[tokio::main]
async fn main() -> Result<(), Box> {
let agent = Agent::builder()
.instructions("Read the requested pull request and its diff. Report concrete bugs with evidence.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.capability(GitHub::from_env()?)
.max_iterations(12)
.build()?;
let engine = Engine::new();
let result = engine.create(agent)
.send_and_wait("Review pull request 123 in my-org/my-repo.")
.await?;
println!("{}", result.response);
Ok(())
}
```
The [complete PR reviewer example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/github_review.rs) takes the repository and PR number as arguments:
```bash
cargo run -p everruns --features github,openai --example github_review -- OWNER/REPO NUMBER
```
It runs a real model against GitHub and prints its review. Its GitHub capability is read-only, so it cannot post comments or reviews. ## Tool selection and write controls | Configuration | Available tools | | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | Default | `get_github_pull_request`, `get_github_pull_request_diff`, `search_github_code`, `read_github_file`, `search_github_issues` | | `.allow_writes(true)` | Also `upsert_github_comment`, `submit_github_pull_request_review`, `upsert_github_issue` | | `.allow_writes(true).allow_pull_requests(true)` | Also `create_github_pull_request` | | `.allow_writes(true).allow_merge(true)` | Also `mark_github_pull_request_ready`, `merge_github_pull_request` | Enable writes only for an application that should modify GitHub:
```rust
use everruns::github::GitHub;
let github = GitHub::from_env()?
.allow_writes(true)
.private_issues_only(true)
.allow_pull_requests(true);
```
These options decide which tools exist; instructions cannot enable an absent tool. The GitHub token also needs the corresponding repository permissions. `private_issues_only(true)` checks repository visibility before filing an issue. Reviews can comment or request changes, and cannot approve. PR creation defaults to a draft and returns an existing open PR from the same head branch. Comments and findings carry markers for updates and duplicate suppression. Finding/review matching trusts bot-authored records. Use a GitHub App installation token for repeatable bot writes; personal-token runs do not gain a bot identity. ## Tokens and refresh `GitHub::new(token)` keeps a token in application memory. `GitHub::from_env()` reads `GITHUB_TOKEN` once at startup and rejects empty values. Neither stores credentials in capability configuration or includes them in debug output. For short-lived installation tokens or rotating credentials, implement `GitHubTokenProvider`. It is called on each operation with the repository and requested permissions. An application can mint the least-privileged token it needs; a static personal token retains its original permissions.
```rust
use everruns::capability::async_trait;
use everruns::github::{GitHub, GitHubTokenProvider, GitHubTokenRequest};
struct ApplicationTokens;
#[async_trait]
impl GitHubTokenProvider for ApplicationTokens {
async fn token(&self, request: &GitHubTokenRequest) -> Result {
// Replace this with your application's refresh or installation-token service.
// request.repository and request.permissions describe this operation's needs.
let _ = request;
std::env::var("GITHUB_TOKEN").map_err(|error| error.to_string())
}
}
let github = GitHub::with_token_provider(ApplicationTokens);
```
Credential-provider failures are returned as generic errors; raw provider errors and GitHub error response bodies are not exposed to the model. A failed provider does not fall back to process or session credentials. `.api_base("https://github.example.com/api/v3")` selects a trusted GitHub Enterprise API root. It receives the credential, so configure it in application code. Search availability follows your GitHub installation and token permissions. ## Platform boundary The Framework adapter shares tool schemas, validation, diff limits, review policy, and vendor operations with the Platform. The Platform still owns GitHub App setup, webhooks/triggers, user-versus-agent identity selection, and connection storage. Managed sandbox checkout requires an attested installation credential and a sandbox service; this Framework capability does not advertise that tool. See [capability integrations](https://everruns.com/docs/framework/capability-integrations/) for other optional integrations and [moving to Platform](https://everruns.com/docs/framework/moving-to-platform/) for deployment differences.
---
# Host Services
> Compose an embedding host's own capabilities, provider resolution, retry budgets, MCP credentials, request metadata and observers into Engine sessions.
Source:
An embedding host often owns more than an agent definition: a curated set of capability implementations with guards and context controls, its own provider and credential resolution, provider recovery budgets, refreshing MCP tokens, request attribution and event observers. `everruns::hosting::services` composes those into the runtimes `Engine` builds, so the host keeps its behavior and still uses Engine’s sessions, durability and turn runner. Stability: experimental, together with [session hosting](https://everruns.com/docs/framework/hosting/). ## What host services can and cannot change `HostServices` is an immutable value attached to an Agent with `AgentBuilder::host_services`. Engine applies it every time it constructs that Agent’s runtime: for new sessions, resumption, durable activations and context inspection. | Host services contribute | Engine keeps owning | | ------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- | | Capabilities written against the Core `Capability` contract, with pre-tool hooks, model views, slash commands and instructions | Session identity, admission, leases and the turn runner | | Provider resolution through a host `ProviderStore` and `DriverRegistry` | The Environment-selected session filesystem and its containment | | Provider retry and stream-stall budgets | Canonical history: observers only observe | | Call-time MCP credentials | Framework capability ids, aliases and tool names | | The outbound boundary for runtime traffic (MCP servers, web tools) | The address checks the MCP transport makes before a request reaches that boundary | | Credential-free provider-request metadata and OpenRouter attribution | Request correlation ids such as `session_id` and `turn_id` | There is no callback that receives a runtime builder, and no runtime handle to drive. A host capability whose id, alias or tool name collides with a Framework capability or another input fails `AgentBuilder::build`, so a host cannot replace, for example, the filesystem capability by registering the same id. Capability tools reach files only through the session filesystem in their `ToolContext`, which is the one the session’s Environment selected. ## Add a host capability Implement `Capability` for the host’s policy. This one blocks one tool before it runs and adds a reminder to every model call:
```rust
use std::sync::Arc;
use everruns::hosting::services::{
Capability, HostServices, ModelViewContext, ModelViewProvider, PolicyGate,
PreToolUseDecision, PreToolUseHook, RuntimeMessage, ToolCall, ToolContext,
ToolDefinition, async_trait,
};
use everruns::{Agent, Engine, OpenAI};
use serde_json::Value;
struct NoForcePush;
#[async_trait]
impl PreToolUseHook for NoForcePush {
async fn before_exec(
&self,
tool_call: ToolCall,
_definition: &ToolDefinition,
_context: &ToolContext,
) -> PreToolUseDecision {
if tool_call.name == "git_push" && tool_call.arguments["force"] == true {
return PreToolUseDecision::Block {
tool_call,
reason: "force pushes are disabled by policy".into(),
user_message: None,
};
}
PreToolUseDecision::Continue(tool_call)
}
}
struct Reminder;
impl ModelViewProvider for Reminder {
fn apply_model_view(
&self,
mut messages: Vec,
_config: &Value,
_context: &ModelViewContext<'_>,
) -> Vec {
messages.push(RuntimeMessage::system("Keep changes reviewable."));
messages
}
}
struct RepoPolicy;
impl Capability for RepoPolicy {
fn id(&self) -> &str { "repo_policy" }
fn name(&self) -> &str { "Repository policy" }
fn description(&self) -> &str { "Guards repository operations." }
fn system_prompt_addition(&self) -> Option<&str> {
Some("Never rewrite published history.")
}
fn pre_tool_use_hooks(&self) -> Vec> {
vec![Arc::new(PolicyGate(NoForcePush))]
}
fn model_view_provider(&self) -> Option> {
Some(Arc::new(Reminder))
}
}
let agent = Agent::builder()
.instructions("You maintain this repository.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.host_services(HostServices::new().capability(RepoPolicy))
.build()?;
let session = Engine::new().create(agent);
let context = session.inspect().await?;
assert!(context.instructions.contains("Never rewrite published history."));
```
`PolicyGate` marks the hook as a policy decision, so it runs after any hook that rewrites arguments and decides on the call that will execute. A blocked call never reaches its tool; the model receives the reason as the call’s result. Commands a capability contributes appear in `SessionContext::commands`. `HostServices::capability` registers and activates. `HostServices::register` only registers, and the Agent activates it with an ordinary `.capability("repo_policy")` reference or through its Harness. ## Resolve providers through the host With `HostProviders`, the host’s `ProviderStore` owns endpoints and credentials and the Agent names its model with `Model::on(provider_key, model_id)`. The store is asked under the calling session’s identity on every provider call, so rotated credentials and provider metadata reach the next call without rebuilding anything.
```rust
use everruns::hosting::services::{HostProviders, HostServices, ProviderStore};
use everruns::{Agent, DriverRegistry, Model};
let mut drivers = DriverRegistry::new();
everruns::drivers::openai::register_driver(&mut drivers);
let agent = Agent::builder()
.instructions("You are concise.")
.model(Model::on("team-openai", "gpt-6.2-sol"))
.host_services(HostServices::new().providers(HostProviders::new(store, drivers)))
.build()?;
```
Selection is never implicit. Building fails with `ConflictingProviders` when a host-resolved model also carries an Agent provider, or when `Model::on` names a different provider than `.provider(..)`. A provider key the host cannot serve fails runtime construction before any model call, rather than falling back to another driver. ## Attribute provider requests `RequestMetadata` is a small set of credential-free strings that every provider request of the Agent’s sessions carries: turn model calls, out-of-band completions, and sessions Engine rebuilds or resumes, on both the in-process and the hosted lanes. `openrouter_attribution` sets the entries the OpenRouter driver sends as its `HTTP-Referer` and `X-Title` headers; other providers receive them as ordinary request metadata.
```rust
use everruns::hosting::services::{HostServices, RequestMetadata};
use everruns::{Agent, Engine, OpenAI};
let metadata = RequestMetadata::new()
.entry("app", "my-app")
.entry("my_app_version", env!("CARGO_PKG_VERSION"))
.openrouter_attribution("https://my-app.example", "My App");
let agent = Agent::builder()
.instructions("You are concise.")
.provider(OpenAI::from_env()?)
.model("gpt-6.2-sol")
.host_services(HostServices::new().request_metadata(metadata))
.build()?;
let session = Engine::new().create(agent);
let context = session.inspect().await?;
assert_eq!(context.request_metadata["app"], "my-app");
```
`SessionContext::request_metadata` shows the entries as assembled for the next request. The runtime adds its own correlation ids to each request after them. Building fails with `InvalidRequestMetadata` when an entry claims one of those ids (`RequestMetadata::RESERVED_KEYS`: `session_id`, `harness_id`, `turn_id`, `exec_id`, `org_id`, `agent_id`, `model_id`), when a key is not 1 to 64 ASCII letters, digits, `_`, `.`, `-` or `:`, when a value is longer than 512 bytes or holds control characters, or when there are more than nine entries. Providers and request capture see these values, so never put a secret in them. ## MCP readiness Cold MCP discovery waits for complete tool catalogs by default. Interactive hosts can opt into a short readiness budget with the `mcp` feature enabled:
```rust
use std::time::Duration;
use everruns::hosting::services::HostServices;
let services = HostServices::new()
.mcp_readiness_budget(Duration::from_millis(250));
```
The budget applies once to the whole eager discovery pass, not separately to every server. Expiry does not turn a pending connection into an error or expose partial catalogs. Discovery continues with bounded concurrency, queue size, response size and its existing hard deadline. Completed tools are available on subsequent turns, including when an Engine rebuilds its runtime between turns. Unrevealed deferred servers still connect only when revealed. With this option enabled, `session.mcp_discovery_statuses().await` reports `Pending`, `Ready`, `AuthenticationRequired`, or `Failed` for eager and revealed servers. It does not initiate discovery. `Pending` also covers servers whose discovery has not started. Authentication failures retain their reconnect hold; transient failures retain the existing retry cooldown. The shared cache remains scoped by session and attachment configuration. Reuse the same `HostServices` value for runtimes that should share completion state. ### MCP status, reconnect and doctor A host can drive an MCP status display, a reconnect action and doctor output from the session alone:
```rust
use everruns::{McpDiscoveryStatus, Session, SessionControlError};
async fn render(session: &Session) -> Result<(), SessionControlError> {
let readiness = session.mcp_readiness().await?;
for server in &readiness.servers {
let state = match server.status {
McpDiscoveryStatus::Ready => format!("ready, {} tools", server.tool_count),
McpDiscoveryStatus::Pending if server.deferred => "deferred".into(),
McpDiscoveryStatus::Pending => "connecting".into(),
McpDiscoveryStatus::AuthenticationRequired => "sign-in required".into(),
McpDiscoveryStatus::Failed => "failed".into(),
};
println!("{}: {state} {}", server.name, server.reason.as_deref().unwrap_or(""));
}
Ok(())
}
async fn reconnect(session: &Session, name: &str) -> Result<(), SessionControlError> {
let after = session.probe_mcp(Some(name)).await?;
if let Some(server) = after.server(name) {
println!("{name}: {:?} at {:?}", server.status, server.last_probe);
}
Ok(())
}
async fn follow(session: &Session) -> Result<(), SessionControlError> {
let mut changes = session.watch_mcp_readiness().await?;
while changes.changed().await.is_some() {
render(session).await?;
}
Ok(())
}
```
* `mcp_readiness` reads, and never starts discovery. It returns one entry per configured server that discovers tools, including unrevealed deferred ones. Each entry carries the status, `deferred`, `probing` (an attempt is running), `tool_count`, a bounded failure or authentication `reason` and `last_probe` (when the last attempt completed). A server the runtime cannot serve, for example a stdio server in a build without `mcp-stdio`, reports `Failed` with a reason. * `probe_mcp(Some(name))` reconnects one server now, and `probe_mcp(None)` reconnects every eager and revealed server. The probe asks your `McpAuthProvider` again and skips the authentication hold and the retry cooldown. It waits up to the discovery deadline and returns the updated readiness. If an attempt is already in flight, the probe returns that attempt’s result. A transient failure keeps the previous tools for turns. Unknown names, and servers that do not discover tools, are rejected. * `watch_mcp_readiness` wakes when readiness may have changed. Every snapshot also carries a `revision` that you can poll and compare instead. Revisions are shared by sessions that use the same `HostServices`, so re-read readiness after each change. For hosted sessions, set `mcp_readiness_budget` so that readiness survives runtime rebuilds. For doctor output, print the full snapshot: `McpReadiness` and its entries implement `Debug`. `mcp_discovery_statuses` remains as the short status list for eager and revealed servers. ## Budgets, MCP credentials and observers
```rust
use std::time::Duration;
use everruns::hosting::services::{
HostEvent, HostEventListener, HostServices, LlmRetryConfig, McpAuthProvider,
McpAuthRequest, McpAuthResult, McpCredential, async_trait,
};
use everruns::Engine;
struct Vault;
#[async_trait]
impl McpAuthProvider for Vault {
async fn authorization(
&self,
request: &McpAuthRequest<'_>,
) -> McpAuthResult