Framework: build and run agents inside a Rust application with the everruns crate --- # Everruns Documentation > Tutorials, how-to guides, reference, and explanation for building and operating Everruns agents. Source: # Documentation Everruns is a **durable agentic harness engine** built on Rust. It provides APIs for managing agents, sessions, and long-running tasks, with real-time SSE event streaming and PostgreSQL-backed durability. Popular[Framework quickstart](https://everruns.com/docs/framework/quickstart/)[Docker Compose](https://everruns.com/docs/getting-started/docker-compose/)[Tools](https://everruns.com/docs/how-to/equip-agents-with-tools/)[Sessions API](https://everruns.com/docs/api/operations/tags/sessions/) ## Choose how you run Everruns [Start here compares the three →](https://everruns.com/docs/getting-started/introduction/) [Framework](https://everruns.com/docs/framework/quickstart/) Build and run agents inside a Rust process with the everruns crate. Nothing else to deploy. * [Quickstart](https://everruns.com/docs/framework/quickstart/) * [Agents and tools](https://everruns.com/docs/framework/agents/) * [Sessions](https://everruns.com/docs/framework/sessions/) * [Models and providers](https://everruns.com/docs/framework/models-and-providers/) `$ cargo add everruns` [Self-hosted](https://everruns.com/docs/getting-started/docker-compose/) Run the Platform (API, workers, PostgreSQL, UI) on infrastructure you control, starting with Docker Compose. * [Docker Compose quickstart](https://everruns.com/docs/getting-started/docker-compose/) * [Configure a model provider](https://everruns.com/docs/getting-started/docker-compose/#configure-llm-provider) * [Environment variables](https://everruns.com/docs/sre/environment-variables/) * [Architecture](https://everruns.com/docs/explanation/architecture/) `$ docker compose up -d` [Everruns Demo](https://everruns.com/docs/getting-started/cloud/) Use the Platform at app.everruns.com without running it. Explore with available promotional credit for built-in models, or bring your own provider keys. * [Try the demo](https://everruns.com/docs/getting-started/cloud/) * [Personal access tokens](https://everruns.com/docs/getting-started/cloud/#3-create-a-personal-access-token) * [Credit and the spend cap](https://everruns.com/docs/getting-started/cloud/#credit-and-the-spend-cap) * [Bring your own keys](https://everruns.com/docs/getting-started/cloud/#bring-your-own-keys-optional) `https://app.everruns.com` ## What kind of help do you need? [LearnTutorialsStep-by-step lessons that end with a running agent. Start here if you're new.](https://everruns.com/docs/tutorials/run-an-agent/)[DoHow-to guidesTask-oriented recipes for common problems, when you know what you want to build.](https://everruns.com/docs/how-to/)[UnderstandExplanationBackground and design rationale: durable execution, sessions, and harnesses.](https://everruns.com/docs/explanation/)[Look upReferenceAPI endpoints, event types, the capability catalog, CLI flags, and environment variables.](https://everruns.com/docs/api/) --- # Everruns Framework > Build and run agents inside a Rust application with the application-facing everruns crate. Source: The **Everruns Framework** is the application-facing [`everruns`](https://docs.rs/everruns) crate. Use it to describe agents, attach models and tools, run multi-turn sessions, observe events, and embed agent execution directly in a Rust process. ```rust use everruns::{Agent, Engine, OpenAI}; let agent = Agent::builder() .instructions("Answer in one short sentence.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let engine = Engine::new(); let turn = engine.create(agent).send_and_wait("Say hello.").await?; println!("{}", turn.response); ``` No database, server or worker is required: an agent runs inside your process. A model provider is: pick one from [Supported providers](https://everruns.com/docs/framework/models-and-providers/#supported-providers), or use the [test simulator](https://everruns.com/docs/framework/testing-and-simulation/) when writing tests. ## Choose the right surface | Surface | Use it for | | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Framework** | Rust applications that build and run agents in process through `everruns` | | **Advanced host crates** | Low-level execution-host composition through `everruns-core` (`host` feature) and focused siblings | | **SDKs** | Remote clients that call a running Everruns server | | **Platform (self-hosted)** | The control plane, server, workers, UI, and durable deployment you run yourself, for example with [Docker Compose](https://everruns.com/docs/getting-started/docker-compose/) | | **Everruns Demo** | The Platform operated for you at [app.everruns.com](https://app.everruns.com), with a built-in model provider and a starter credit; bring-your-own keys are optional. See the [Everruns demo quickstart](https://everruns.com/docs/getting-started/cloud/) | Normal library users should start with the Framework. Hosts that must replace storage or orchestration cross into [custom backends](https://everruns.com/docs/framework/custom-backends/). ## Start here * [Quickstart](https://everruns.com/docs/framework/quickstart/), install the crate and run one turn against a live model provider. * [Architecture](https://everruns.com/docs/framework/architecture/), understand Agent, Engine, Session, and the shared immediate/durable execution kernel. * [Upgrade notes](https://everruns.com/docs/framework/upgrade-notes/), the code changes each breaking release needs. ## Core APIs * [Agents and tools](https://everruns.com/docs/framework/agents/), instructions, files, workspaces, MCP, plugins, context inspection, and typed function tools through `everruns::tool`. * [Models and providers](https://everruns.com/docs/framework/models-and-providers/), the model/provider split, every driver that ships today, each driver’s environment variables, provider model catalogs, and custom providers. * [Direct calls and decisions](https://everruns.com/docs/framework/direct-model-calls/), one prompt and one answer without an agent, or a calibrated number rather than prose. * [Sessions](https://everruns.com/docs/framework/sessions/), independent multi-turn conversations, bounded history, typed resume, and engine-lifetime or crash-durable persistence. * [Events and cancellation](https://everruns.com/docs/framework/events-and-cancellation/), observe a live turn, stop work cooperatively, and record bounded canonical event envelopes. * [Observability](https://everruns.com/docs/framework/observability/), export every session to OpenTelemetry or Braintrust, or register your own event listeners. * [Workspaces and environments](https://everruns.com/docs/framework/workspaces-and-environments/), bind sessions to isolated or shared heads and configure read and write scopes with secure defaults. * [Lifecycle hooks](https://everruns.com/docs/framework/lifecycle-hooks/), run awaited application behavior at execution boundaries. * [Answer agent questions](https://everruns.com/docs/framework/ask-user/), implement `AskUser` so your application answers the agent’s structured questions. * [Session work and wakes](https://everruns.com/docs/framework/background-work/), immediate and scheduled work with explicit delivery and restart semantics. * [Coordinator agents](https://everruns.com/docs/framework/coordination/), one session that hands work to threads and hears back from them. ## Extend * [Capabilities](https://everruns.com/docs/framework/advanced-capabilities/), configure the optional standard policy bundle and open references, or package typed tools with stable metadata and lifecycle context. * [GitHub tools](https://everruns.com/docs/framework/github/), read PRs, explore repositories, and explicitly enable writes with application-owned credentials. * [Capability integrations](https://everruns.com/docs/framework/capability-integrations/), opt into filesystem, shell, web, Lua, and MCP implementation boundaries. * [Portable and hosted capabilities](https://everruns.com/docs/framework/capability-boundaries/), understand the Framework/Platform implementation boundary. * [Agent blueprints](https://everruns.com/docs/framework/agent-blueprints/), contribute a code-defined specialist agent from a capability. * [Host services](https://everruns.com/docs/framework/host-services/), compose a host’s own capabilities, provider resolution, retry budgets, MCP credentials and observers into Engine sessions. * [Session controls](https://everruns.com/docs/framework/session-controls/), run slash commands, hot-install capabilities, switch model, effort and MCP servers, and query context on a live session. * [Attach to a running session](https://everruns.com/docs/framework/attach-control/), let a second local process list, watch, drive and cancel a host’s sessions. * [Custom backends](https://everruns.com/docs/framework/custom-backends/), cross into low-level host composition deliberately. * [Testing and simulation](https://everruns.com/docs/framework/testing-and-simulation/), deterministic tests without credentials. * [Runnable examples](https://everruns.com/docs/framework/examples/), complete programs maintained with the crate. ## Expose and deploy * [Deploy a Framework app](https://everruns.com/docs/framework/deployment/), ship the binary or a container with a persistent data directory and environment-provided keys. * [Move to Platform or Cloud](https://everruns.com/docs/framework/moving-to-platform/), what carries over to a hosted agent and what has to be rebuilt. * [Serve](https://everruns.com/docs/framework/serve/) (experimental), attribute macros, file-layout discovery and a manifest, served over the Everruns server `/v1` API. * [Tutorial: Serve an Agent and Call It from Code](https://everruns.com/docs/framework/serve-tutorial/) (experimental), build a serve app, protect it with an API key, and call it with the SDK’s `AgentClient`. * [Serve AG-UI](https://everruns.com/docs/framework/ag-ui/), stream a session to CopilotKit or any AG-UI 1.0 client from your own HTTP server. * [A2A](https://everruns.com/docs/framework/a2a/), serve Framework agents to other agents over A2A 1.0, and delegate work to remote A2A agents. * [Serve on AgentCore](https://everruns.com/docs/framework/serve-agentcore/) (experimental), deploy a serve app to Amazon Bedrock AgentCore Runtime. * [Serve on celld](https://everruns.com/docs/framework/serve-celld/) (experimental), run a serve app durably on celld, self-hosted Durable Objects. [Application services](https://everruns.com/docs/framework/service-integrations/) covers authenticated integrations, data services, durable event triggers, and portable evaluation/trace APIs. --- # A2A > Serve Framework agents to other agents over A2A 1.0, and let a Framework agent delegate work to remote A2A agents. Source: [A2A](https://a2a-protocol.org) is an open protocol for agents to call each other over HTTP: a JSON-RPC endpoint plus a public Agent Card that says what the agent is and how to reach it. A Framework app can use it in both directions: * **Serve**: other agents, the official `a2a` CLI, or the A2A Inspector call your agents. This comes from the `a2a` feature of [`everruns-serve`](https://everruns.com/docs/framework/serve/). * **Call**: your agent hands work to a remote A2A agent with `spawn_agent`. This comes from the `a2a` feature of `everruns` and the `a2a_agent_delegation` capability. Both speak A2A 1.0. The routes match an Everruns [A2A endpoint](https://everruns.com/docs/features/a2a/), so a caller can move between a serve app and Everruns by changing the base URL. ## Serve your agents over A2A Turn on the feature: ```toml [dependencies] everruns-serve = { version = "*", features = ["a2a"] } ``` Every top-level agent in the app is then served at: | Route | What it is | | ---------------------------------------------------------- | -------------------------------------------------------------- | | `POST /v1/channels/{agent}/a2a` | A2A 1.0 JSON-RPC. Requests need the `A2A-Version: 1.0` header. | | `GET /v1/channels/{agent}/a2a/.well-known/agent-card.json` | The Agent Card, built from the agent’s name and `description`. | Nothing else changes in the agent. A `description` is worth setting, because it is what other agents read on the card: ```rust use serve::prelude::*; /// Researches a topic and answers with short factual notes. #[agent] fn researcher() -> Agent { Agent::builder() .model("openai/gpt-6.2-sol") .description("Researches a topic and answers with short factual notes.") .instructions(md!("instructions.md")) .build() } ``` Try it with the `a2a` CLI against a dev server: ```bash CARD=http://localhost:3000/v1/channels/researcher/a2a/.well-known/agent-card.json a2a card get -a "$CARD" a2a send -a "$CARD" "Tide pools" a2a send -a "$CARD" --stream "Coral reefs" a2a task list -a "$CARD" ``` Pass the full card URL: the card lives under the agent’s route, not at the server root. ### How A2A maps onto serve * Each A2A `contextId` is one serve session. A follow-up message with the same `contextId` continues the conversation, and the session survives a restart. * Each A2A task is one turn. A task goes `working`, then delivers the final reply as a `response` artifact, then `completed` (or `failed`). * `SendMessage`, `SendStreamingMessage`, `GetTask`, `ListTasks`, `CancelTask`, and `SubscribeToTask` are supported. * A pending tool approval or `ask_user` question keeps the task `working` until it is answered through serve’s `/v1` routes. ### Limits * Only A2A 1.0 is served. Requests without `A2A-Version: 1.0` are rejected. * Tasks are kept in memory: after a restart, old task ids are unknown, while their contexts continue. * The endpoint takes the same credentials as the other serve routes ([Authentication](https://everruns.com/docs/framework/serve/#authentication)). With auth configured, the card’s `securitySchemes` and `securityRequirements` list one scheme per method (`apiKey` for static keys, `oidc` for an OpenID Connect issuer, `oauth2Bearer` for introspection), any one of which is enough; with none, the card lists no scheme and the endpoint is open. * Push notifications are not offered. Use streaming or `GetTask`. For a hosted endpoint with API keys, push notifications, and A2A 0.3 support, use an Everruns [A2A endpoint](https://everruns.com/docs/features/a2a/). ## Call remote A2A agents Turn on the feature and add the capability, listing the agents your agent may call: ```toml [dependencies] everruns = { version = "*", features = ["a2a", "local", "openai"] } ``` ```rust use everruns::{Agent, CapabilityRef, Engine, LocalConfig, OpenAI}; use serde_json::json; let agent = Agent::builder() .name("writer") .instructions( "Before writing, ask the `researcher` agent for notes with `spawn_agent` \ (target type `external_a2a`, id `researcher`, mode `foreground`). \ Then write one paragraph from its notes.", ) .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability(CapabilityRef::new("a2a_agent_delegation").config(json!({ "agents": [{ "id": "researcher", "name": "Researcher", "description": "Researches a topic and answers with short factual notes.", "base_url": "https://agents.example.com/v1/channels/researcher/a2a" }] }))) // Delegated runs are recorded in session storage. .local(LocalConfig::new("./writer-state")) .build()?; let session = Engine::new().create(agent); ``` The model sees one `spawn_agent` tool with an `external_a2a` target. It can only reach the agents in `agents`; it never supplies a URL. * `base_url` is where the Agent Card is resolved (`{base_url}/.well-known/agent-card.json`). * `mode: "foreground"` waits for the remote task and returns its reply as the tool result. `mode: "background"` returns a `task_id` at once and wakes the session when the remote task finishes. * `result_schema` on `spawn_agent` requires a structured result and validates the remote agent’s first data part against it. * URLs are checked before each call: localhost, private ranges, and metadata addresses are refused, resolved addresses are pinned, and redirects are not followed. Set `"allow_local_urls": true` on an agent entry only while developing against a server on your machine under `DEPLOYMENT_GRADE=dev`. The remote agent can be anything that speaks A2A 1.0: a serve app, an Everruns A2A endpoint, or another vendor’s agent. ## Run the example [A2A example](https://github.com/everruns/everruns/tree/main/examples/serve/a2a) puts both halves together. `researcher` is a serve app served over A2A, and `writer` is an `everruns` agent that delegates research to it and then drafts. It runs offline with scripted models, and on OpenAI when `OPENAI_API_KEY` is set: ```bash cargo run -p serve-example-a2a --bin researcher # serves on :3000 cargo run -p serve-example-a2a --bin writer -- "tide pools" # in another shell ``` The writer’s `spawn_agent` call goes over real A2A even offline, so the example is also a quick way to check an A2A setup end to end. ## See also * [Serve](https://everruns.com/docs/framework/serve/#a2a), the serve routes and configuration. * [A2A](https://everruns.com/docs/features/a2a/), the Everruns A2A endpoint and the hosted delegation capability. * [Serve AG-UI](https://everruns.com/docs/framework/ag-ui/), the protocol for agent-to-UI rather than agent-to-agent. --- # Configure and author capabilities > Use one open AgentBuilder capability entrypoint for typed built-ins, dynamic references, and code-defined packages. Source: Every agent capability enters through `AgentBuilder::capability`. The method accepts the public, non-sealed `IntoCapability` contract, so Framework built-ins and third-party packages compose without adding a method or enum variant to `AgentBuilder`. Cargo features determine which environment-backed implementations a Framework binary contains. See [Capability integrations](https://everruns.com/docs/framework/capability-integrations/) for filesystem, Bashkit, web-fetch, Lua, and MCP boundaries; this page covers agent-level configuration and authoring after an implementation is available. ## Configure capabilities Use typed values when the Framework exposes a stable configuration, a `capability::Definition` for application code, and `CapabilityRef` when the ID and JSON arrive dynamically: ```rust use everruns::{ Agent, CapabilityRef, CompactionConfig, OpenAI, ToolSearch, }; use serde_json::json; let weather_definition = build_weather_capability(); let agent = Agent::builder() .instructions("Use configured capabilities when relevant.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability(CompactionConfig::new().budget_percent(0.85)) .capability(ToolSearch::automatic()) .capability(weather_definition) .capability( CapabilityRef::new("vendor.custom") .config(json!({ "mode": "database-driven" })), ) .build()?; ``` `ToolSearch::automatic` uses hosted deferred loading on supported models and the existing provider-neutral client-side implementation everywhere else. Its optional threshold and never-defer allowlist map to the built-in’s real configuration; there are no provider fields on the Framework value. `CapabilityRef` is the explicit escape hatch for database, plugin, or catalog configuration. Its ID stays open. An unknown ID is retained as a reference but contributes nothing until the selected host or plugin provides that implementation. This is not a function tool: ordinary functions remain on `AgentBuilder::tool` and `#[everruns::tool]`. The same rule applies to Everruns Platform capability IDs. The default Framework registry does not advertise or execute hosted knowledge, delegation, task, hook, or management capabilities. See [portable and hosted capabilities](https://everruns.com/docs/framework/capability-boundaries/). JSON capability config is not a credential store. Framework debug output redacts it, but a host may persist or inspect it; pass a provider-owned secret handle rather than API keys or tokens. Conversion is infallible. `AgentBuilder::build` validates ID syntax and the JSON object boundary, runs known built-in and declarative/plugin validators, and rejects duplicate IDs after built-in alias resolution. A code implementation cannot shadow a built-in or be paired with a second reference of the same ID; registrations never use last-write-wins behavior. Third-party typed values implement `IntoCapability` using only `everruns`: ```rust use everruns::{CapabilityRef, CapabilitySpec, IntoCapability}; struct VendorSearch { index: String, } impl IntoCapability for VendorSearch { fn into_capability(self) -> CapabilitySpec { CapabilityRef::new("vendor.search") .config(serde_json::json!({ "index": self.index })) .into() } } ``` No `everruns-core`, registry, store, or host dependency is needed. ## Choose the standard policy bundle The Framework’s default `builtins` feature links `everruns-core` (`builtins` feature), the backend-neutral implementation bundle for compaction, tool search, budgeting, loop/progress safeguards, prompt caching, tool-call repair, output handling, and guardrails. Linking the package has no registration side effect: each host constructs its registry explicitly, so a custom registry cannot be changed by dependency order. Applications that want only the open Framework contracts can disable default features and add the integrations they need. The policy bundle owns no network client, process runner, interpreter, database, or hosted service. Output persistence and distillation declare `session_file_system` as a host-provided dependency; enable them only in a composition that supplies that capability. The optional `ui-capabilities` feature also owns the namespaced `everruns_core::builtins::{openui,a2ui}` component catalogs and prompt generators; applications do not need separate UI-protocol crates. ## Choose an authoring level Use the smallest extension contract that fits the behavior you own. | Contract | `#[everruns::tool]` | `everruns::capability` | | ------------------------------------- | ----------------------------- | ----------------------------- | | Best for | One application function | A reusable capability package | | Typed input and result | Yes | Yes | | Generated input schema | Yes | Yes | | Inspectable output schema | No | Yes | | Multiple tools | Register functions separately | One stable capability id | | Capability instructions and metadata | No | Yes | | Session/workspace identity and locale | No | Curated `Context` accessors | | Progress events | No | `Context::progress` | | [Narration](#tool-narration) override | `narrate = …` | `Handler::narrate` | | Child-work cancellation | Turn future only | `Context::cancellation` | | Backend/store/tenancy access | No | No | ## Ordinary tools Annotate a typed async function. Its doc comment becomes the description and its arguments become JSON Schema. ```rust /// Convert Celsius to Fahrenheit. #[everruns::tool] async fn fahrenheit(celsius: f64) -> f64 { celsius * 1.8 + 32.0 } let agent = everruns::Agent::builder() .instructions("Use the conversion tool.") .provider(everruns::OpenAI::from_env()?) .model("gpt-6.2-sol") .tool(fahrenheit()) .build()?; ``` Prefer this until you need a capability-level contract. ## Advanced capabilities An advanced capability is an immutable `capability::Definition`. It owns a stable id, catalog text, optional instructions and JSON metadata, and one or more typed handlers. `AgentBuilder::capability` installs its implementation on the private in-process runtime and activates that stable id once. ```rust use everruns::{Agent, OpenAI, capability}; #[derive(capability::Deserialize, capability::JsonSchema)] #[serde(crate = "everruns::capability::serde")] #[schemars(crate = "everruns::capability::schemars")] struct LookupInput { id: String, } #[derive(capability::Serialize, capability::JsonSchema)] #[serde(crate = "everruns::capability::serde")] #[schemars(crate = "everruns::capability::schemars")] struct Record { id: String, score: f64, labels: Vec, } struct Lookup; #[capability::async_trait] impl capability::Handler for Lookup { type Input = LookupInput; type Output = Record; type Error = capability::Error; fn name(&self) -> &str { "lookup_record" } fn description(&self) -> &str { "Look up one record by exact id." } fn hints(&self) -> capability::Hints { capability::Hints::default() .readonly(true) .idempotent(true) } async fn execute( &self, input: Self::Input, context: capability::Context, ) -> Result { context.progress("Looking up the record").await; if input.id != "rec_42" { return Err(capability::Error::user( "record_not_found", "No record has that id", ).details(capability::serde_json::json!({ "id": input.id }))); } Ok(Record { id: input.id, score: 0.98, labels: vec!["verified".into()], }) } } let records = capability::Definition::new( "records", "Records", "Application-owned record lookup.", ) .instructions("Use exact record ids and do not infer missing records.") .metadata(capability::serde_json::json!({ "owner": "risk" })) .tool(Lookup); let agent = Agent::builder() .instructions("Answer with verified record data.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability(records) .build()?; ``` Both input and output types must satisfy the compile-time protocol bounds. The generated schemas are available through `Definition::tools()[..].spec()` for tests, documentation, or a host catalog. Results are serialized directly to JSON, so structs, arrays, numbers, booleans, and null do not pass through a string conversion. ## Errors Return `capability::Error::user(code, message)` for an expected domain failure. Add bounded JSON details when they help the model recover. The code, message, and details travel through the model-visible tool-error channel. Return `capability::Error::internal(code, message)` for diagnostic details that are unsafe to show to the model, such as network internals or implementation bugs. The engine logs the diagnostic and gives the model a generic error; internal details do not cross the model boundary. Never include credential values or other secrets in any error because host logs may retain internal diagnostics. Custom application error enums can implement `Into` and be used as `Handler::Error`. ## Context, progress, and cancellation `capability::Context` exposes only stable lifecycle data: * opaque session and workspace ids; * the resolved locale, when present; * best-effort correlated `tool.progress` events; * a call-scoped cancellation signal. Observe progress through `Session::events()` and `SessionEventKind::ToolProgress`. Everruns does not currently expose a custom capability result-streaming protocol; return one typed result when execution finishes. Normal awaited work needs no cancellation branch. Cancelling a turn drops the handler future. Clone `context.cancellation()` only into child tasks, processes, or watchers that might otherwise survive after `execute` is dropped. The signal fires on cancellation and on every other call completion path. ## Tool narration Clients show one human-readable line per tool call phase (started, waiting, completed, failed), carried as `narration` on tool events and read back with `SessionEvent::narration()`. Every tool gets one without extra code: the engine renders a generic, localized line from the tool’s display name and, when it can pick a safe argument, a short detail, such as `Running Lookup Record`. Override it when a domain phrase reads better. Every authoring level has the same optional hook, and returning `None` keeps the generic line: * `capability::Handler::narrate(&self, &ToolNarration) -> Option` * `FunctionTool::narrate(|call: &ToolNarration| -> Option)` * `#[everruns::tool(narrate = |args: &Args, call: &ToolNarration| …)]`, which receives typed arguments; arguments that do not parse keep the generic line. `ToolNarration` gives the raw call `arguments()`, the `phase()`, the session `locale()`, and, on completion, the model-visible `result()`. The `everruns::narration` helpers phrase it the way built-in tools do: `safe_arg_str` reads a string argument but never one whose key looks like a secret, `truncate` and `narration_detail` bound a value to one short line (the latter also strips URL credentials and query strings), and `narrate_labeled_action` renders started, completed, and failed verbs with an optional detail and a Ukrainian locale variant. ```rust use everruns::{capability, narration}; struct Lookup; #[capability::async_trait] impl capability::Handler for Lookup { // Input, Output, Error, name, description, and execute as above. fn narrate(&self, call: &capability::ToolNarration<'_>) -> Option { // No id yet: keep the generic line. narration::safe_arg_str(call.arguments(), &["id"])?; Some(narration::narrate_labeled_action( call.arguments(), call.phase(), call.locale(), ("Looking up record", "Looked up record", "Could not look up record"), ("Шукаю запис", "Знайшов запис", "Не вдалося знайти запис"), &["id"], )) } } /// Convert Celsius to Fahrenheit. #[everruns::tool(narrate = |args: &FahrenheitArgs, _call: &everruns::ToolNarration<'_>| { Some(format!("Converting {} °C", narration::truncate(&args.celsius.to_string(), 16))) })] async fn fahrenheit(celsius: f64) -> f64 { celsius * 1.8 + 32.0 } ``` Narration arguments are untrusted model output and narration is shown to people and stored with session events. Render only fields you chose, keep them bounded, and never include credentials or other secrets. Hosts that implement the low-level `everruns_core::tools::Tool` trait directly override its `narrate` method with the same contract. ## Security boundary The advanced SPI intentionally does not export provider credentials, stores, tenant or organization objects, registries, payment authority, filesystem backends, or other host services. Pass application-owned clients or state into your handler struct when constructing the definition. A handler is trusted application code and retains whatever process authority those values provide; `Hints` describe behavior but do not enforce authorization, egress, or approval policy. Apply authorization, egress policy, timeouts, and input bounds at those application boundaries, and never place secrets in capability or tool metadata. For a complete provider-backed program, run the [`advanced_capability` example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/advanced_capability.rs). Some built-ins expect the application to supply behavior rather than configuration. `ask_user` is one: it needs someone to answer, so `AgentBuilder::ask_user` takes a responder instead of a config value. See [Answer agent questions](https://everruns.com/docs/framework/ask-user/). --- # Serve AG-UI > Stream a Framework session to CopilotKit or any AG-UI 1.0 client from your own HTTP server, with ask_user questions and tool approvals as interrupts. Source: [AG-UI](https://docs.ag-ui.com) is the event protocol between an agent and the application that renders it. CopilotKit and `@ag-ui/client` post a `RunAgentInput` and read back a stream of events, usually as server-sent events. The `ag-ui` feature lets any Rust host answer those requests from a Framework session, without the Everruns server: ```bash cargo add everruns --features ag-ui ``` `Session::ag_ui(input)` sends the input’s last user message and returns the run as a stream of AG-UI 1.0 events: `RUN_STARTED`, the projected turn, then one `RUN_FINISHED` or `RUN_ERROR`. The projection is the one the Everruns server uses, with the trusted policy: assistant text, reasoning, and token usage are visible, and a failure carries the runtime’s message. ## Mount it on axum The `ag-ui-axum` feature adds `AgUiHandler`, a ready-made route: ```bash cargo add everruns --features ag-ui-axum ``` ```rust use axum::Router; use everruns::ag_ui::{AgUiHandler, AgUiOptions, AgUiThreads, InterruptGate, StaticToken}; use everruns::{Agent, Engine, OpenAI}; #[tokio::main] async fn main() -> Result<(), Box> { let gate = InterruptGate::new(); let agent = Agent::builder() .instructions("You are a helpful assistant.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .ask_user(gate.clone()) .approver(gate.clone()) .build()?; let handler = AgUiHandler::new( AgUiThreads::new(Engine::new(), agent), StaticToken::bearer(std::env::var("AG_UI_TOKEN")?), ) .options(AgUiOptions::new().gate(gate)); let router = Router::new().route("/ag-ui", handler.route()); let listener = tokio::net::TcpListener::bind("127.0.0.1:3000").await?; axum::serve(listener, router).await?; Ok(()) } ``` Each request is authorized, its `threadId` resolved to a session with `AgUiThreads` (the first run of a thread creates one from your agent, later runs reopen it), and the run streamed as server-sent events: one unnamed `data: ` event per AG-UI event and a `: keepalive` comment every 15 seconds. Everything that can fail does so before the stream opens: | Status | When | | ------ | ---------------------------------------------------------------------------------------------- | | `401` | The authorizer refuses the request. The body is not read. | | `400` | The body is not a `RunAgentInput`, the `threadId` is invalid, or the run cannot use the input. | | `500` | The thread store or the session failed. The detail is logged, not returned. | Errors are `application/problem+json`. Once the stream is open, a failure is the run’s own `RUN_ERROR`. Request bodies are capped by axum’s default 2 MB limit. ### Authorization `AgUiHandler::new` takes an authorizer, so an open endpoint is a choice you write down: * `StaticToken::bearer(token)` accepts `Authorization: Bearer `, and `StaticToken::header(name, token)` a custom header. Tokens are compared in constant time; a refusal carries `WWW-Authenticate: Bearer` for the bearer form. * `Unauthenticated` accepts everything: for local development, or behind a proxy that already authenticates. * Any `Fn(&HeaderMap) -> Result`, or your own `AgUiAuthorizer` implementation when the check needs to await. The authorizer returns an `AgUiCaller`. `AgUiCaller::scoped(user_id)` gives each caller its own threads, so one user cannot continue another’s conversation by sending its `threadId`: ```rust use axum::http::HeaderMap; use everruns::ag_ui::{AgUiCaller, Unauthorized}; // Your proxy has verified the user and set this header. let by_user = |headers: &HeaderMap| { headers .get("x-user-id") .and_then(|value| value.to_str().ok()) .map(AgUiCaller::scoped) .ok_or_else(Unauthorized::new) }; let handler = AgUiHandler::new(threads, by_user); ``` ### Your own route On another HTTP server, or to resolve sessions yourself, call `Session::ag_ui_with` (or `AgUiThreads::run`) and frame the stream. With `ag-ui-axum`, `sse_response(run)` does the framing; without it, each event is `serde_json::to_string(&event)` in a `data:` line. The crate’s `ag_ui_axum` example is the handler on the offline simulated model: ```bash cargo run -p everruns --features ag-ui-axum --example ag_ui_axum ``` Point any AG-UI client at the route: ```ts import { HttpAgent } from "@ag-ui/client"; const agent = new HttpAgent({ url: "http://127.0.0.1:3000/ag-ui", headers: { Authorization: `Bearer ${token}` }, }); ``` ## Threads across restarts `AgUiThreads::new` keeps its thread map in memory. To keep threads when the process restarts, keep both halves on disk: the sessions, with an agent on the `local` backend, and the map, with `SqliteThreadStore` (both need the `local` feature): ```rust use everruns::ag_ui::{AgUiThreads, SqliteThreadStore}; use everruns::{Agent, Engine, LocalConfig, OpenAI}; let config = LocalConfig::new("/var/lib/my-app"); let agent = Agent::builder() .instructions("You are a helpful assistant.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .local(config.clone()) .build()?; let threads = AgUiThreads::with_store(Engine::new(), agent, SqliteThreadStore::local(&config)?); ``` After a restart the next run of a thread reopens its session, history included. Implement `ThreadStore` to keep the map in your own database instead. With durable session hosting (`SqliteHosting`), use `AgUiThreads::hosted` and a registered build instead of an Agent value. Each new thread is created through the authority, at an identity derived from the thread, so a retried first run reuses the same session. Threads whose sessions were moved from the `local` backend with `Engine::cut_over_legacy_session` keep answering from those same sessions: ```rust let threads = AgUiThreads::hosted(engine, build, SqliteThreadStore::local(&config)?); ``` A thread id is chosen by the client, and anyone who knows one continues that conversation. When one server serves several users, scope threads to the authenticated caller: `AgUiCaller::scoped` does it in `AgUiHandler`, and `run_in` on your own route: ```rust let run = app.threads.run_in(&user_id, input, options).await?; ``` Thread ids must be 1 to 128 characters of `[A-Za-z0-9-_.]`; anything else is `AgUiError::InvalidInput`. ## History the client already has A client such as CopilotKit keeps the conversation and sends all of it with every run. When a thread is new to the server, for example after a restart without a durable backend, `AgUiThreads` records the input’s earlier user and assistant messages as the new session’s history before the run’s user message, so the agent picks up where the conversation was. Once the session has history, the earlier messages are not read again: the session’s own conversation wins. Only user and assistant text is seeded (an assistant’s tool calls as text); `tool`, `system`, `developer`, activity and reasoning messages are not. The most recent 256 messages and 512 KiB of text are kept. On a session you resolve yourself, turn it on with `AgUiOptions::seed_history(true)` for a thread’s first run. The client writes these messages, assistant turns included, so treat them as no more trusted than the user message. ## Questions and approvals become interrupts In the Framework, `ask_user` questions and tool approvals are answered by responders on the agent while the turn waits. `InterruptGate` is a responder for both that answers through AG-UI instead: register it with `.ask_user(...)` and `.approver(...)`, and pass it with `AgUiOptions::gate`. A question or approval then ends the run with the 1.0 interrupt outcome while the turn stays parked in your process: | Parked on | `reason` | Answer (`resume[].payload`) | | ----------------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------- | | `ask_user` | `everruns.ask_user` | `{ "answers": [{ "id", "selected", "other_text" }], "status"? }`, described by the interrupt’s `responseSchema` | | `ask_user` with a `secret` question | `everruns.secret_required` | none: only abandoning is accepted | | a tool that needs approval | `tool_approval` | `{ "decision": "allow" \| "allow_always" \| "reject" \| "reject_always" }` | The interrupt id is the tool call id. The next run sends `resume` entries and no new message; the entries answer the parked requests and the run streams the rest of the same turn. Abandoning an entry (`status: "cancelled"`) declines the question or rejects the call. An interrupt left without an entry is not abandoned: nothing is resolved and the run ends with the interrupts again, as it does when a new message arrives while one is open. An entry that cannot be applied returns `AgUiError::InvalidInput` before the stream opens, and nothing is resolved. Parked requests live in memory. If the process exits, the waiting turn is left unfinished in the session’s log; after a restart, `Session::resume_interrupted_turn` runs its waiting calls again, so they park on the gate anew and a client’s resume run can answer them (see [session history](https://everruns.com/docs/framework/sessions/#history-and-resume)). ## Policy `AgUiOptions::policy` takes a `ProjectionPolicy` to hide reasoning or usage, show fixed text while tools run, or map failures to your own error codes: ```rust use everruns::ag_ui::{AgUiOptions, ProjectionPolicy}; let options = AgUiOptions::new().policy(ProjectionPolicy { reasoning_visible: false, usage_visible: false, tool_activity_text: Some("Working on it...".into()), ..ProjectionPolicy::default() }); ``` ## What a run reads from the input The session owns the conversation, so only the last message is sent, and it must be a user message (its text parts), unless the run carries frontend tool results (below). Earlier messages are read only to seed a new thread (above); `state` and `forwardedProps` are not read. `RUN_STARTED` carries `protocolVersion: "1.0"` only when the request declared a version, so pre-1.0 clients see the stream they expect. ## Frontend tools The input’s `tools` are tools your page runs, such as a confirmation dialog or a navigation action. Each run makes them the session’s client-side tools, so send them on every run; the latest set wins. When the model calls one, the turn parks and the run streams the call as `TOOL_CALL_START`, `TOOL_CALL_ARGS` and `TOOL_CALL_END` with its name and arguments, then finishes in success with `outcome.pendingToolCallIds`. Run the calls in the page, then post the next run with their results as trailing `tool` messages (`toolCallId` set, `content` the result, `error` for a failure). The same turn continues and the run streams the rest of it. A result for a call that is not parked is ignored. Until every parked call has a result, nothing is recorded and the run reports the calls again, as it does when a new user message arrives instead. Definitions are bounded: at most 64 tools, names matching `^[A-Za-z0-9_-]{1,64}$`, descriptions up to 4096 characters, parameter schemas up to 16 KiB, results up to 256 KiB. Names starting with `mcp_` are refused. Parked calls live in memory: if the process exits, they stay unanswered. `system` and `developer` messages and `context` entries are ignored by default, and a system message after the user message is refused. If your server authenticates whoever posts the input, you can let each run carry instructions: ```rust use everruns::ag_ui::AgUiOptions; let options = AgUiOptions::new().input_instructions(true); ``` Each run’s system and developer messages, in order, then its context entries, are then appended to the agent’s instructions for that run. They may appear anywhere in `messages`, including after the user message, which stays the run’s input. The next run replaces them with its own, and a run with none clears them; they never enter the conversation history. Leave this off for callers you do not trust: these messages carry the authority of the system prompt. A [serve](https://everruns.com/docs/framework/serve/#ag-ui-and-copilotkit) app gets this route built in with its `ag-ui` feature, at `/v1/channels/{agent}/ag-ui`. For a hosted agent with no server code, Everruns serves the same protocol at `/v1/channels/{channel_id}/ag-ui`. --- # Author an agent blueprint > Contribute a code-defined specialist agent from a capability, with a typed configuration contract the spawn path enforces Source: An **agent blueprint** is a code-defined specialist agent: a baked-in prompt, a set of private tools, a model-selection strategy, an iteration bound, and a narrow configuration surface. A host agent delegates to it through the ordinary [sub-agents](https://everruns.com/docs/capabilities/sub-agents/) tool without gaining access to its internals. Reach for a blueprint when work needs different tools, instructions, or model economics than the parent agent — repository scouting, catalog benchmarking, any job where the parent should get the answer without carrying the tools that produced it. Blueprints are not persisted user-created agents; they ship with a capability and are available wherever that capability is enabled. ## Contribute the blueprint A capability contributes blueprints by implementing `agent_blueprints()`. The returned `AgentBlueprint` carries everything the child runtime needs: ```rust fn agent_blueprints(&self) -> Vec { vec![AgentBlueprint { id: "repo_scout", name: "Repo Scout", description: "Search repositories for code, files, and issues. \ Read-only agent for codebase exploration.", model: BlueprintModel::Fixed("claude-haiku-5-5".to_string()), system_prompt: REPO_SCOUT_PROMPT, tools: vec![Box::new(SearchCodeTool), Box::new(ReadFileTool)], max_turns: Some(15), config_schema: Some(json_schema_for::()), }] } ``` The `description` is what a parent agent reads when deciding whether to delegate, so write it as a routing decision: what the blueprint is for, and when to pick it. `model` chooses one of three strategies. `Fixed` pins a model the host cannot override, which suits specialist work with a known cost/quality target. `Default` names a model the validated config may override. `Inherit` takes the parent’s model, for work that genuinely needs the parent’s characteristics. Tools listed here are **private**. They are instantiated only for the blueprint’s own child session and never appear in the host agent’s tool list. ## Define configuration as a type Derive the config schema from a Rust struct rather than writing JSON by hand. The struct is the single source of truth: field set, bounds, defaults, and descriptions all reach the spawning agent from one place. ```rust use everruns_contracts::capability::json_schema_for; use everruns_contracts::capability::schemars::JsonSchema; use serde::{Deserialize, Serialize}; /// The same ceiling the search tools apply to their own arguments. const MAX_REPOS: u32 = 50; /// Configuration for the repository scout. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields, default)] #[schemars(crate = "everruns_contracts::capability::schemars")] pub struct RepoScoutConfig { /// Maximum number of repositories to scan. #[schemars(range(min = 1, max = MAX_REPOS))] pub max_repos: u32, } impl Default for RepoScoutConfig { fn default() -> Self { Self { max_repos: 10 } } } ``` Three habits keep the contract honest: * **Express each bound once.** Write it as a constant shared by the schema attribute and the code that enforces it at runtime, so a schema edit cannot drift from the clamp it describes. * **Write doc comments for the caller.** They become the schema descriptions the spawning agent reads. Implementation notes belong in ordinary `//` comments. * **Close the object** with `deny_unknown_fields` unless the blueprint genuinely accepts open configuration. Requiring configuration is a matter of leaving a field without a default: fields serde cannot default become `required` in the derived schema, and the spawn path rejects a call that omits them. ## What the spawn path enforces Configuration is validated against the derived schema **before** a child session exists, so a declared bound constrains the host rather than merely advising the model. A spawn is rejected when: * a value violates the schema — out of range, wrong type, missing a required property; * the config carries a key the schema does not define (with `deny_unknown_fields`); * the blueprint declares no schema at all but config was supplied. The error returned to the calling agent names the violations and includes the schema, so a model can usually correct its own call and retry. Validation governs what a **host** may configure. It is not a substitute for a tool checking its own arguments: keep the runtime clamps in the blueprint’s tools, so a misbehaving child agent stays inside the same envelope. Configuration can only select behavior the blueprint intentionally exposes. It cannot replace the system prompt, inject tools, bypass model policy, or expand capability permissions. ## Delegation and lifetime A spawned blueprint session is a real durable session. It takes part in the same message, event, task, workspace, cancellation, and recovery infrastructure as any other sub-agent; the difference is only how its runtime is assembled. The session persists the blueprint identity and its validated config, so a worker can reconstruct the same runtime after a retry or handoff. The blueprint itself stays a stateless template. Follow-ups address the durable session, not the blueprint. ## Worked examples Two blueprints ship in-tree and are worth reading as references: * [`crates/integrations/src/github/lib.rs`](https://github.com/everruns/everruns/blob/main/crates/integrations/src/github/lib.rs) — the minimal case: one config field, a pattern constant shared with the runtime check that enforces it. * [`crates/integrations/src/openrouter/model_scout.rs`](https://github.com/everruns/everruns/blob/main/crates/integrations/src/openrouter/model_scout.rs) — the fuller case: numeric bounds tied to runtime constants, a nested config type, and a spend budget. ## Related * [Sub-agents](https://everruns.com/docs/capabilities/sub-agents/) — the delegation tool blueprints are invoked through * [GitHub Scout](https://everruns.com/docs/capabilities/github-scout/) — a shipped blueprint from the caller’s side --- # Agents and Tools > Describe an agent with instructions, a model, tools, files, integrations, and an optional workspace, and add typed Rust functions as tools. Source: An `Agent` is an immutable, validated application description. Pass it to an application-owned engine to create independent sessions. ```rust use everruns::{Agent, McpServer, OpenAI}; let agent = Agent::builder() .name("researcher") .instructions("Research carefully and cite the evidence you used.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .file("brief.md", "Investigate the supplied question.") .readonly_file("policy.md", "Never expose secrets.") .mcp_server(McpServer::http("catalog", "https://example.com/mcp")) .build()?; ``` Builder validation catches blank instructions, a missing model, duplicate providers, tools, or capabilities, invalid tool schemas, invalid capability IDs/configuration, implementation collisions, and invalid MCP configuration before a session starts. Configure typed built-ins, code-defined packages, and dynamic references through the single `capability(...)` entrypoint; see [Configure and author capabilities](https://everruns.com/docs/framework/advanced-capabilities/). ## Harness foundations Choose `Harness::base()` for the system essentials only, `conversation()` for dialogue, `worker()` for files, skills, long context, budgeting and task coordination without a shell, or `bashkit_worker()` to add the Bashkit shell. These constructors share the hosted platform’s [preset definitions](https://everruns.com/docs/features/harnesses/). ```rust let engine = Engine::new(); let session = engine .create(agent) .harness(Harness::conversation()) .start() .await?; let response = session.send_and_wait("Tell me a joke.").await?; ``` Enable optional host integrations for the tools your application needs. Worker [delegation requires host backends](https://everruns.com/docs/built-ins/harnesses/worker/); the default in-memory host does not supply them. `Harness::generic()` is deprecated and preserves its legacy capabilities. With `builtins` enabled, sessions without a bound harness include managed compaction and history retrieval. Explicit custom harnesses keep their own capability selection. Compaction observation is available through the [Framework event API](https://everruns.com/docs/framework/evaluation-and-traces/#compaction-observation); organization monitoring consent and reporting remain host responsibilities. Presets preserve the framework’s read-only workspace default. For an agent that writes files, explicitly select `Agent::builder().workspace_policy(WorkspacePolicy::read_write())`. ## Files and workspaces * `file(path, content)` seeds an editable file. * `readonly_file(path, content)` seeds a file the agent may read but not change. * `workspace(root)` exposes one trusted real-disk root as `/workspace`. Choose workspace roots from trusted application configuration. Model output and untrusted request fields must not select executable paths or host directories. The underlying filesystem boundary rejects traversal and symlink escape. Use a [`WorkspacePolicy`](https://everruns.com/docs/framework/workspaces-and-environments/#workspace-security) to configure portable read, write, hidden-path, and recursive-delete restrictions. ## MCP and plugins `McpServer::http` adds a remote Streamable HTTP server. Headers may be supplied by the host and are redacted from `Debug`. Local-process MCP is separately feature-gated with `mcp-stdio`; its command, arguments, and environment are trusted host configuration. `AgentBuilder::plugin(path)` loads a local plugin directory and returns a typed error if it cannot be compiled. Non-fatal compiler warnings remain visible in the application-facing session context. ## Inspect effective context Inspect the next model call before or after a turn: ```rust let engine = Engine::new(); let session = engine.create(agent); let context = session.inspect().await?; println!("messages: {}", context.messages.len()); println!("tools: {}", context.tools.len()); ``` Inspection uses the same assembly path as execution, including MCP discovery, plugin prompt contributions, message filters, and model selection. ## Tools The default-enabled `everruns::tool` macro turns an async Rust function into a typed agent tool. Parameter types produce JSON Schema and call arguments are deserialized before the function runs. ```rust use everruns::{Agent, OpenAI}; #[everruns::tool] /// Add two integers. async fn add(left: i64, right: i64) -> Result { Ok(left + right) } let agent = Agent::builder() .instructions("Use the add tool for arithmetic.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .tool(add()) .build()?; ``` Use `#[everruns::tool(name = "…", description = "…")]` to override metadata, or `#[tool(rename = "…")]` on a parameter to change its model-facing name. Functions must be async, non-generic, and have plain named parameters. The published `everruns-macros` package is an implementation crate. Applications should use the re-exported `everruns::tool` macro and should not depend on `everruns-macros` directly. ### Dynamic handlers `FunctionTool::new` is available when a tool schema is determined at runtime: ```rust use everruns::FunctionTool; use serde_json::json; let echo = FunctionTool::new( "echo", "Return the supplied text.", json!({ "type": "object", "properties": { "text": { "type": "string" } }, "required": ["text"] }), |args: serde_json::Value| async move { Ok::<_, String>(args["text"].clone()) }, ); ``` Prefer the macro for normal typed application tools. Use the dynamic form for schemas obtained from configuration or another protocol. ### Call context and progress Make the first parameter a `ToolCallContext` (by value or reference) to learn which session, turn, and tool call a handler is serving, and to report progress. Progress arrives on `Session::events()` as `SessionEventKind::ToolProgress`. The context is not a model argument, so it never appears in the schema. The dynamic equivalent is `FunctionTool::with_context`. ```rust use everruns::ToolCallContext; /// Export the report. #[everruns::tool] async fn export(ctx: &ToolCallContext, pages: u32) -> String { ctx.progress(format!("rendering {pages} pages")).await; format!("exported for session {}", ctx.session_id()) } ``` ### Approval Mark a tool with `needs_approval` to ask a host approver before it runs. The bare flag asks on every call. `needs_approval = ` asks only when the rule returns `true`. The rule receives the generated arguments struct, named after the function in PascalCase with an `Args` suffix (`run_sql` becomes `RunSqlArgs`). The approver gets the session id and the tool call, including its id and arguments, so a host serving many sessions can route the request. A rejected call never runs, and the model receives a tool error saying it was rejected. An agent with a gated tool but no `.approver(..)` fails to build. ```rust use everruns::approval::{ApprovalDecision, ToolApprover, async_trait}; use everruns::{Agent, Model, SessionId, ToolCall, ToolDefinition}; /// Run one SQL statement. #[everruns::tool(needs_approval = |args: &RunSqlArgs| args.sql.contains("DROP"))] async fn run_sql(sql: String) -> String { sql } struct Console; #[async_trait] impl ToolApprover for Console { async fn approve(&self, _: SessionId, call: &ToolCall, _: &ToolDefinition) -> ApprovalDecision { println!("approve {}? {}", call.name, call.arguments); ApprovalDecision::Reject } } let agent = Agent::builder() .instructions("Answer with SQL.") .model(Model::simulated("Done.")) .tool(run_sql()) .approver(Console) .build()?; ``` `FunctionTool::needs_approval(|args: &Value| ..)` and `FunctionTool::always_needs_approval()` are the dynamic forms. ## Agents that hand work to other agents Add the agents this one can hand work to with `.member(agent)`. Each member keeps its own instructions, model and capabilities. The lead gets the `spawn_agent` tool and sees each member by its name and description; work it hands over runs as the member’s own session in the same Engine, in the foreground or as a background task. ```rust use everruns::hosting::{AgentBuild, SqliteHosting}; use everruns::{Agent, Engine, Model, SessionId}; let reviewer = Agent::builder() .name("reviewer") .description("Reviews a change and lists what to fix.") .instructions("Review the change you are given. Be specific.") .model(Model::simulated("Looks good.")) .build()?; let lead = Agent::builder() .name("lead") .instructions("Hand each change to reviewer before you answer.") .model(Model::simulated("Done.")) .member(reviewer) .build()?; let build = AgentBuild::new("lead", "v1"); let engine = Engine::builder() .hosting(SqliteHosting::open("./.everruns/authority.db")?) .register(build.clone(), lead) .build(); let session = engine.create_registered(SessionId::new(), &build).await?; session.send_and_wait("Review the login change.").await?; ``` Members need a host that keeps child sessions: an Engine with `SqliteHosting`, or a local profile. Registering the lead is enough; a member’s sessions resume after a restart under a build derived from the lead’s. Members nest one level. Inviting a member into the lead’s own session is Platform-only. ## File-based agents Load and export the shared portable package format, including initial files and complete skill directories. See [Define agents as files](https://everruns.com/docs/how-to/define-agents-as-files/) for validation, diffs, model bindings and runnable examples. --- # Framework Architecture > Understand Agent, Engine, Session, pluggable session hosting, and the shared execution kernel. Source: Everruns uses one agent runtime across native processes, platform workers and Wasm isolates. Applications define ordinary `Agent` values and use `everruns::Engine` and `Session`. A session hosting profile owns durable admission, canonical state, ownership epochs and activation. It does not reimplement agent planning, tool policy or recovery. ``` flowchart TD App[Framework app or serve] --> Engine[Engine and Session] Engine --> Memory[Memory] Engine --> SQLite[Native SQLite] Engine --> External[External host] Memory --> Driver[Shared SessionActorDriver] SQLite --> Driver External --> Driver Platform[Platform workers] --> Driver Driver --> Runtime[Input / Reason / Act runtime] ``` ## Public Framework objects | Object | Responsibility | | ------------- | --------------------------------------------------------------------------------------------------------------------- | | `Agent` | Immutable behavior: instructions, model and provider, tools, capabilities, files, and lifecycle hooks | | `Engine` | Retained behavior registry, session admission, lifecycle, queries and activation through the selected hosting profile | | `Session` | First-class, engine-bound conversation used for turns, steering, events, cancellation, inspection, and history | | `Environment` | Session resources, including one exact backend-owned workspace head and typed extensions | New Framework code creates and resumes sessions through an Engine: ```rust use everruns::{Agent, Engine, OpenAI}; let agent = Agent::builder() .instructions("Answer concisely.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let engine = Engine::new(); let session = engine.create(agent); let session_id = session.session_id(); let turn = session.send_and_wait("Begin.").await?; assert!(turn.success); drop(session); let resumed = engine.resume(session_id).await?; assert_eq!(resumed.session_id(), session_id); ``` ## Shared execution, different authorities Framework profiles use the same core `SessionActorDriver`, turn loop and event-log recovery. Memory coordinates one process. SQLite commits creation, commands, receipts and wakeups in one database and can run an independent native supervisor. External hosts implement the [public hosting SPI](https://everruns.com/docs/framework/hosting/). The source-only Durable Object reference demonstrates object-local Wasm and a separately authenticated native Engine executor without a Framework vendor dependency. Admission commits before a send is acknowledged; a turn identity is assigned later. Retain command identities across uncertain replies. Dropping a Session, wait handle or SSE connection does not cancel durable accepted work. Reopen its admission handle to observe assignment, pending client tools, suspension or a finalized outcome. Awaited Framework completion hooks settle before that outcome becomes final. The Platform server uses its existing transactional PostgreSQL actor authority. Workers lease runnable sessions and drive the same runtime. The Platform API’s internal `TurnBackend` remains a control-plane integration detail. The generic `everruns-durable` task engine supplies Platform background jobs, worker management and schedules. Agent turns use the shared actor driver. ## Choose a recovery boundary * **Volatile Framework:** `Engine::new()` is offline and database-free. The creating Engine can resume a dropped Session, but process exit loses it. * **Native durable Engine:** select `hosting::SqliteHosting`, register the exact `AgentBuild` and Environment factory, and start its supervisor for independent recovery. SQLite durability covers the configured disk’s failure domain. * **External durable Engine:** implement `SessionHosting`, retain exact builds, and configure independent activation plus explicit host services. The [Durable Object reference](https://everruns.com/docs/framework/durable-objects/) is one source-only implementation, outside the product publish set. * **Legacy local storage:** explicit `LocalConfig` and custom host backends retain the older facade actor lane and its weaker recovery boundary. Persistent stores alone do not provide coherent session hosting or independent activation. * **Distributed durable Platform:** server and workers keep each session’s event log in PostgreSQL and lease sessions to workers, so a turn recovers across process or worker loss. Applications call it through the remote API or SDKs rather than configuring the facade Engine. See [Persistence](https://everruns.com/docs/framework/sessions/#persistence) and [Session History and Resume](https://everruns.com/docs/framework/sessions/#history-and-resume) for the exact application lifecycle. ## Extension boundaries Normal applications depend on `everruns`. Engine is concrete. Hosting plugins implement `SessionHosting` using `hosting::spi` and supply a coherent authority plus epoch-scoped storage ports. The Framework owns lifecycle execution and outcome projection; plugins never implement another agent loop. Providers implement `ChatDriver`. Advanced core hosts can still compose the experimental `Execution` and internal turn-backend contracts. Framework applications should use the session hosting boundary, which also covers creation, admission, callbacks, reads and wakeups. Workspaces, compute, secrets, channels and schedules require explicit services with their own lifetime and isolation guarantees. A durable journal does not preserve those resources automatically. --- # Answer an agent's questions > Implement the AskUser trait so an embedding application can answer an agent's structured questions from its own interface. Source: An agent with the [Ask User](https://everruns.com/docs/capabilities/ask-user/) capability can ask the person it is working with a small batch of structured questions and wait for the answer. In a hosted product the browser renders that card. In an embedding application there is no browser, so the application answers — which is what the `AskUser` trait is for. ```rust use everruns::ask_user::{Answer, AnsweredBy, AskUser, Outcome, Question, Status, async_trait}; use everruns::{Agent, Model}; struct HouseRules; #[async_trait] impl AskUser for HouseRules { async fn ask(&self, questions: &[Question]) -> Outcome { let answers = questions .iter() .map(|question| Answer { id: question.id.clone().unwrap_or_default(), selected: question .options .iter() .find(|option| option.is_default) .or_else(|| question.options.first()) .map(|option| option.label.clone()) .into_iter() .collect(), other_text: None, secret_ref: None, }) .collect(); Outcome { status: Status::Answered, answered_by: AnsweredBy::Unattended, answers, } } } let agent = Agent::builder() .instructions("Confirm deployment choices before acting.") .model(Model::simulated("Done.")) .ask_user(HouseRules) .build()?; ``` `AgentBuilder::ask_user` registers the responder and enables the capability in one call. The responder runs **inside** the tool call, so the turn never parks waiting for an external result — the agent asks, your code answers, and the turn continues. ## Route by session A host that serves several sessions overrides `ask_in` instead of relying on the default, which ignores the context and calls `ask`. `AskContext` carries the session id, the `ask_user` tool call id, and the turn id when known. ```rust use everruns::ask_user::{AskContext, AskUser, DefaultsResponder, Outcome, Question, async_trait}; struct Router; #[async_trait] impl AskUser for Router { // Only reached when a caller has no context to give. async fn ask(&self, questions: &[Question]) -> Outcome { DefaultsResponder.ask(questions).await } async fn ask_in(&self, context: &AskContext, questions: &[Question]) -> Outcome { // Send `questions` to the client that owns `context.session_id()`. } } ``` ## Without a responder `.capability("ask_user")` on its own uses `DefaultsResponder`: it applies the options the model marked as recommended, falls back to the first option, and reports `AnsweredBy::Unattended`. Headless runs resolve immediately rather than waiting out the timeout for somebody who is not there. A text or secret question has no unattended value. `DefaultsResponder` declines a batch containing either kind rather than returning an empty answer. ## Report who answered `answered_by` is part of the contract, not decoration: | Value | Meaning | | ------------ | --------------------------------------------- | | `User` | A person actually chose this | | `Timeout` | The deadline passed and a default was applied | | `Unattended` | Nobody could be asked; a default was applied | Report `User` only when a person really answered. An agent that reads a fallback as a considered choice acts with more confidence than the answer earns, and that is the failure this field exists to prevent. ## A worked responder [Weekend Concierge host](https://github.com/everruns/everruns/tree/main/examples/weekend-concierge-host) implements `TerminalResponder` over stdin: numbered options, comma-separated toggles for a multi-select, a free-text path when the question allows one, and terminal echo turned off for a credential. ```plaintext [Energy] How much energy does the group have on Friday? *1. Up for anything — Games, noise, moving around. 2. Low-key — Sitting, talking, snacks. 3. Something else > ``` Two details in it are worth copying into any responder: **An empty answer takes the declared default** rather than returning nothing. A question the model asked and nobody addressed is something it cannot distinguish from a deliberate skip. **A secret answer carries a reference, never a value.** `Answer` has no `value` field at all, so there is no path from a collected credential into the transcript. The host keeps the value; the agent gets `session:MY_TOKEN` and tools resolve it by name. **A text answer carries its string in `other_text`.** It has no selected option: ```rust Answer { id, selected: Vec::new(), other_text: Some("feature/open-question".to_string()), secret_ref: None, } ``` ```rust Answer { id, selected: Vec::new(), other_text: None, secret_ref: Some(everruns::ask_user::session_secret_ref("MY_TOKEN")), } ``` For the smallest possible version, [Ask User example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/ask_user.rs) runs a responder and the unattended path side by side and prints what each decided. ## Questions are not permission `ask_user` auto-resolves, so it is for decisions and preferences only. A destructive, irreversible, or outward-facing action needs `request_approval`, whose wait does not auto-resolve. See [the boundary](https://everruns.com/docs/capabilities/ask-user/#not-a-consent-gate). ## See Also * [Ask User capability](https://everruns.com/docs/capabilities/ask-user/), the contract and its limits * [Configure and author capabilities](https://everruns.com/docs/framework/advanced-capabilities/) * [Lifecycle hooks](https://everruns.com/docs/framework/lifecycle-hooks/), for intercepting tool calls rather than answering them --- # Attach to a Running Session > Let a second local process list a host's sessions, watch their output, send input and cancel turns, with same-user access, a rotating token and revocation. Source: A terminal host often wants `myagent attach` in a second terminal: see which sessions are live, watch one’s output, type into it, stop a turn that went the wrong way. The Engine’s sessions live in the host process, so the host serves a small local control plane and the second process talks to it. The `everruns::control` module (feature `control`, Unix only) provides both halves. Stability: experimental. ```toml everruns = { version = "*", features = ["control", "openai"] } ``` ## Serve sessions from the host Put the sessions a second process may reach in a `ControlRegistry`, then start a `ControlServer` on a private directory. Nothing outside the registry is reachable. ```rust use everruns::control::{ControlRegistry, ControlServer, default_dir}; use everruns::{Agent, Engine, OpenAI}; let agent = Agent::builder() .instructions("Keep answers short.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let session = Engine::new().create(agent); let registry = ControlRegistry::new(); registry.register("main", &session); let server = ControlServer::start(default_dir("myagent"), registry.clone())?; println!("attach with: myagent attach --dir {}", server.attach_info().dir.display()); // The host's own prompts go through the registry, so an attached // `cancel` can reach the turn they start. let reply = registry.send("main", "Hello").await?.wait().await?; println!("{}", reply.response); ``` `default_dir(app)` is `$XDG_RUNTIME_DIR/` when set, otherwise `/-`. Keep the server value alive for as long as the host serves; dropping it stops serving and removes the socket and token file. A turn is cancellable from outside only if the registry knows its `TurnHandle`. Send through `registry.send`, or report a send made elsewhere with `registry.track(&session.id(), &sent)`. ## Drive it from a second process `ControlClient` is the attached side. Each call opens one connection. ```rust use everruns::control::{ControlClient, default_dir}; let client = ControlClient::new(default_dir("myagent")); for session in client.list().await? { println!("{} {} {:?}", session.label, session.id, session.active_turns); } // Start a turn (or steer the running one) and wait for its reply. let (_, turn) = client.send_and_wait("main", "Summarize the plan").await?; println!("{}", turn.response); // Stop whatever is running. client.cancel("main").await?; // Watch output; `Some(0)` replays the conversation first, `None` starts live. let mut stream = client.attach("main", Some(0)).await?; while let Some(event) = stream.next().await? { if let Some(text) = event.render() { print!("{text}"); } } ``` Sessions are addressed by id or by a unique label. `attach` streams user messages from every sender, assistant text, tool names and turn boundaries. Tool arguments and results are not forwarded. Each event carries the durable `sequence` when it has one, so a client that disconnected or saw a `lagged` event re-attaches with `Some(last_sequence)` and misses nothing durable. The [`attached-cli` example](https://github.com/everruns/everruns/tree/main/examples/attached-cli) is a complete host and attached CLI in about 200 lines. Copy it to start. ## Who may attach Access is local and has two independent gates: * **Same OS user.** The control directory is created with mode 0700 and the socket with 0600. The server reads each connection’s kernel peer credentials and refuses a different uid before reading a byte. The client checks the directory and token file belong to it and are private, and that the process behind the socket runs as the same user, before it sends the token or any input. A directory that already exists with group or other permissions, or another owner, is refused by both ends. * **Bearer token.** 32 random bytes, hex encoded, in the 0600 `token` file next to the socket. Every connection’s first line carries it, and the server compares it in constant time. There is no way to grant another user access. A host that needs remote or multi-user access should serve sessions over an authenticated protocol such as [AG-UI](https://everruns.com/docs/framework/ag-ui/) or [serve](https://everruns.com/docs/framework/serve/) instead. ## Token lifecycle | Event | Effect | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ControlServer::start` | Issues a token and writes the token file. | | Every `token_ttl` (default 15 minutes) | Issues a new token. The old one stops opening connections at once; the client re-reads the file once when a token is refused, so rotation is invisible to it. Open attach streams continue. | | `rotate_token()` | Same as a scheduled rotation, on demand. Also re-opens a revoked server. | | `revoke()` | Deletes the token and the file. Every open attach stream and waiting send ends with `revoked`, and new connections are refused until `rotate_token()`. | | Dropping the server | Stops serving, ends open streams with `shutdown`, and removes the socket and token file. | ```rust use std::time::Duration; use everruns::control::{ControlRegistry, ControlServer}; let server = ControlServer::builder("/run/user/1000/myagent") .token_ttl(Some(Duration::from_secs(60))) .max_connections(8) .start(registry)?; // Someone copied the token somewhere it should not be: server.revoke(); server.rotate_token()?; ``` ## Cancel semantics `cancel` targets every unfinished turn the registry tracks for the session: the running turn and any message admitted behind it that has not started. It returns their identities, the same values `list` reports in `active_turns`. An identity is fixed when the message is sent: the turn id when `send` returned one, otherwise the accepted message’s id (the default Engine admits durably first, so expect message ids there). It does not change once the turn is assigned. Each target behaves like `TurnHandle::cancel`: * The turn stops at its next await point; running tool work is dropped and the session records `turn.cancelled`. * Messages that steered into that turn end with it, and messages still queued behind it are dropped rather than run afterwards. * The runtime records a short user message noting the cancellation (“User requested to cancel the work.”), so the next turn knows the work was stopped. Attached streams show it like any other user message. * The session stays usable; the next `send` starts a new turn. * A session with nothing tracked still running answers `no_active_turn`, including when the turn finished while the cancel was on its way. Disconnecting never cancels anything. Dropping an attach stream, a client that exits while waiting on `send_and_wait`, a revocation or a server shutdown all leave the session’s turns running. A client that leaves frees its connection slot at once, so abandoned waits cannot starve `cancel`. ## Protocol One newline-delimited JSON request per connection, answered by one or more JSON lines: `{"token": "...", "command": {"op": "list"}}`, with `send`, `cancel` and `attach` taking a `session` field. Errors are `{"type": "error", "code": "...", "message": "..."}`, with codes `unauthorized`, `bad_request`, `not_found`, `ambiguous`, `no_active_turn`, `session_closed`, `busy` and `failed`. Requests are capped at 256 KiB and must arrive within 10 seconds; connections beyond `max_connections` get `busy`. The client caps response lines at the same size and expects the first line within 30 seconds; later lines (attach events, a waited-for turn) have no deadline. Only one server may use a directory at a time: it holds an exclusive lock on `/lock` while it runs. The serde types in `everruns::control` are the wire format, so a client in another language can speak it. --- # Session work and wakes > Run immediate and scheduled background work with explicit delivery and restart semantics. Source: `everruns::work` lets an application request and handle work owned by a session without importing runtime registries, platform stores, or task-kind constants. The application chooses its own work kinds and JSON payloads. ```rust use std::time::Duration; use everruns::work::{TaskOutcome, TaskRequest, WakePolicy, WorkQueue}; use serde_json::json; let queue = WorkQueue::in_memory(); let work = queue.for_session("session_123"); work.submit( TaskRequest::new("thumbnail", json!({ "image": "cover.png" })) .idempotency_key("thumbnail:cover.png") .wake_policy(WakePolicy::OnCompletion), ).await?; for delivery in queue.claim_due(Duration::from_secs(30), 16).await? { // Route on delivery.task.kind and check cancellation before side effects. queue.finish( &delivery, TaskOutcome::success(json!({ "path": "cover-thumb.png" })), ).await?; } ``` The runnable version is [`session_work.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/session_work.rs). ## Contract * **Ownership:** every task and wake has one opaque `session_id`. A `SessionWork` handle fixes that owner for all requests and reads. * **Persistence and restart:** `WorkQueue::in_memory()` is process-local and database-free. State survives replacing the queue only while the same `Arc` is retained. It does not survive a process restart. A host that needs durable recovery supplies a `WorkBackend`. * **Scheduling:** `Immediate` work is claimable now; `At(SystemTime)` work is not claimable early. Both are one-shot requests. The host owns polling; recurring calendars and schedule runners stay host concerns. There is no hidden scheduler or database in the default build. * **Delivery:** task and wake claims are leased and at least once. If a process stops before settlement or acknowledgment, the item is claimable after the lease expires. Each retry has a new token and attempt; stale attempts cannot settle newer work. * **Idempotency:** task keys and direct-wake keys each have a session-scoped namespace. Repeating the same request returns the original task or wake. Reusing a key for different input fails with `IdempotencyConflict`. Workers should also deduplicate external side effects on the stable task id or submission key. * **Cancellation:** pending work cancels immediately. Running work records cancellation intent; the worker checks the latest task snapshot, stops cooperatively, then reports `TaskOutcome::Canceled`. A task may still succeed if it passes its safe cancellation point first. * **Wakes:** applications can request an immediate wake directly. A task can also create one atomic completion wake with `WakePolicy::OnCompletion`. Wakes use the same lease/retry/acknowledgment model as tasks. Durable platform scheduling, retention, distributed polling, and multi-host coordination belong in the host’s `WorkBackend`; they are not enabled by the offline Framework default. The in-memory backend retains accepted payloads until it is dropped and does not enforce admission quotas, so hosted providers must apply tenant authorization, payload limits, quotas, and retention at their own boundary. --- # Batch Model Requests > Submit independent model completions through native provider batch APIs, persist a credential-free ticket, and collect results in another process. Source: Use `everruns::batch::Batch` for offline classification, extraction, or evaluation where latency can be hours. A batch sends independent model requests to a provider’s batch API. It does not start an agent, stream tokens, run tools, or execute side effects. This is an opt-in **alpha** API; ordinary completions and sessions keep their existing behavior. ## Supported providers and hosts | Surface | Support | | --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Anthropic direct Messages API | Native Rust, `anthropic` Cargo feature; Message Batches submit, poll, collect, cancel | | Anthropic-compatible proxy | Explicit versioned `base_url`; must implement the Message Batches API, not just Messages | | OpenAI direct Responses API | Native Rust, `openai` feature; Files upload and Batches submit, poll, collect, cancel | | OpenAI direct Chat Completions API | Same lifecycle, using `OpenAI::from_env()?.api(OpenAIApi::ChatCompletions)`; tickets are separate from Responses tickets | | Gemini Developer API | Native Rust, `gemini` feature; inline batch submit, poll, collect, cancel; Files and Vertex AI modes are unsupported | | Mistral La Plateforme | Native Rust, `mistral` feature; inline request submission, polling, inline/file result collection and cancellation | | Groq Chat Completions | Native Rust, `groq` feature; Files/Batches submit, poll, collect and cancel; audio batches are outside this completion API | | Fireworks | Native Rust, `fireworks` feature; explicit account and artifact policy; dataset upload, native batch job submission, poll and collect. Cancellation returns `UnsupportedOperation` | | Custom endpoints | Must implement the complete corresponding native batch API; a chat-compatible endpoint alone is insufficient. Azure batch configurations are not qualified here | | OpenRouter, Bedrock, ChatGPT and remaining built-in drivers | Unsupported; returns `BatchError::Unsupported`, with no synchronous fallback | | Legacy endpoint-bound `into_boxed_driver()` / `DriverRegistry::create_chat_driver()` adapters | No batch hook: retain the configured `Provider` for ticket endpoint binding instead. Registry request-option/credential wrappers also fail closed | | Custom providers | Implement the optional `batch::BatchDriver` SPI and expose it through `ChatDriver::batch_driver` | | Native applications with memory, files, SQLite or another application database | Supported direct API; application owns ticket persistence and scheduling | | Engine memory/SQLite hosting, `serve`, Platform workers and eval service | No batch scheduling or ticket persistence integration; an embedding native application can separately use the direct API | | Wasm, Cloudflare Durable Objects and celld | No built-in batch driver in this version | The batch API is currently on `main` and is not included in the published `0.49.0` crate. Until a crate release includes it, use `main` with the provider features you need. Commit `Cargo.lock` to retain the resolved Git revision for reproducible builds: ```toml [dependencies] everruns = { git = "https://github.com/everruns/everruns", branch = "main", features = ["anthropic"] } serde_json = "1" tokio = { version = "1", features = ["macros", "rt"] } ``` For Core hosts using the stored-message conversion example, source `everruns-core` from the same branch and enable `host`, so both crates share the message contract: ```toml [dependencies.everruns-core] git = "https://github.com/everruns/everruns" branch = "main" default-features = false features = ["host"] ``` ## Submit now, collect later The batch owns one configured provider. Supply typed request values directly, or describe each request with a completion builder. Both paths share validation, submission and the same resumable tickets. ### Pass message lists directly `submit_requests` accepts a `Vec`. Each value contains its ID, existing messages, and `LlmCallConfig`, so code that already has a conversation can submit it without rebuilding it through `.system()` and `.user()` calls. The client stays available for polling, collection and cancellation. ```rust use everruns::{ batch::{Batch, BatchRequest}, llm::{Message, MessageRole}, providers::anthropic::Anthropic, LlmCallConfig, }; let mut config = LlmCallConfig::new("claude-sonnet-5"); config.max_tokens = Some(64); let requests = vec![ BatchRequest { id: "review_001".into(), messages: vec![ Message::text(MessageRole::System, "Classify as positive, negative, or neutral."), Message::text(MessageRole::User, "Delivery arrived early."), ], config: config.clone(), }, BatchRequest { id: "review_002".into(), messages: vec![ Message::text(MessageRole::System, "Classify as positive, negative, or neutral."), Message::text(MessageRole::User, "The replacement does not turn on."), ], config, }, ]; let client = Batch::new(Anthropic::from_env()?, "tenant-42/anthropic-account-7"); let ticket = client.submit_requests(requests).await?; let saved_ticket = serde_json::to_vec(&ticket)?; ``` `everruns::llm::Message` is the provider-message contract used by core. Stored `everruns_core::RuntimeMessage` records are a different type: core hosts reuse the existing [message conversion helpers](https://github.com/everruns/everruns/blob/main/crates/core/src/llm_conversions.rs). For a text-only core conversation: ```rust use everruns::{batch::BatchRequest, LlmCallConfig}; use everruns_core::{RuntimeMessage, llm_conversions::llm_message_from_message}; let conversation = vec![ RuntimeMessage::system("Classify the review."), RuntimeMessage::user("Delivery arrived early."), ]; let request = BatchRequest { id: "review_001".into(), messages: conversation.iter().map(llm_message_from_message).collect(), config: LlmCallConfig::new("claude-sonnet-5"), }; // Pass vec![request] to client.submit_requests(...). ``` For stored images or files, core hosts use the attachment-aware conversion helper with their resolved attachments. This direct API does not load session history or resolve stored file references itself. All request IDs must be unique, and each request needs messages. The entire list is validated before submission. Mixing an explicit list with buffered `.request(...)` builder entries is rejected to avoid silently discarding work. ### Completion builder Build each completion with a **bare model ID**, then give it a unique request ID. A completion carrying its own provider is rejected so a batch cannot silently route it to a different account. ```rust use everruns::{Model, batch::Batch, providers::anthropic::Anthropic}; let model = Model::from("claude-sonnet-5"); let batch = Batch::new(Anthropic::from_env()?, "tenant-42/anthropic-account-7") .request("review_001", model.completion() .system("Classify as positive, negative, or neutral. Return one word.") .user("Delivery arrived early.").max_tokens(64))? .request("review_002", model.completion() .system("Classify as positive, negative, or neutral. Return one word.") .user("The replacement still does not turn on.").max_tokens(64))?; let ticket = batch.submit().await?; // Save immediately in access-controlled durable application storage. let saved_ticket = serde_json::to_vec(&ticket)?; ``` ### Resume and collect In another process, reconstruct the provider from current credentials and use exactly the same application scope and endpoint. Credential rotation is allowed within that provider account. The ticket contains a provider job ID, request IDs, scope, format version and endpoint fingerprint. It contains no credentials, prompts, result URLs, or results. ```rust use everruns::{batch::{Batch, BatchStatus, BatchTicket}, providers::anthropic::Anthropic}; let ticket: BatchTicket = serde_json::from_slice(saved_ticket)?; let client = Batch::new(Anthropic::from_env()?, "tenant-42/anthropic-account-7"); if client.poll(&ticket).await? == BatchStatus::Ended { for item in client.collect(&ticket).await? { match item.result { Ok(response) => println!("{}: {}", item.id, response.text), Err(error) => eprintln!("{}: {error:?}", item.id), } } } // Otherwise save the ticket and schedule another poll, usually ≥60 seconds later. ``` Collection checks that the job ended and maps results by request ID, returning them in submission order. Unknown, duplicate or missing IDs fail the whole collection. A request can succeed while another expires, is cancelled, or fails. Successful responses retain text, reasoning, function tool calls, finish reasons and token usage. Tool calls are data; nothing executes them. Validate model output before consuming it. Anthropic’s existing driver does not enforce `Completion::response_format`; batch submission rejects that option too. ## Choose a provider through the same API `Provider::supports_batch()` checks the driver’s local capability. It does not probe model eligibility, account permissions, quotas or a custom server’s API. Unsupported drivers retain the default `ChatDriver::batch_driver() == None`. Custom drivers implement the same optional SPI; Framework does not dispatch on provider names. With the `openai` feature: ```rust use everruns::{Model, batch::Batch, providers::openai::OpenAI}; let provider: everruns::Provider = OpenAI::from_env()?.into(); assert!(provider.supports_batch()); let client = Batch::new(provider, "tenant-42/openai-account-7"); let ticket = client.request("review_001", Model::from("gpt-6.2-sol") .completion().user("Classify: delivery arrived early.").max_tokens(64))? .submit().await?; ``` For Gemini, enable `gemini`, use `Gemini::from_env()` from `everruns::providers::gemini`, select a Gemini model and its own tenant/account scope. `submit_requests(Vec)`, tickets and lifecycle operations stay the same. Requests in each OpenAI, Gemini, Mistral, Groq or Fireworks job must use one model. To select OpenAI Chat Completions explicitly, configure the connection with its typed API selector: ```rust use everruns::{OpenAI, OpenAIApi, Provider}; let provider: Provider = OpenAI::from_env()? .api(OpenAIApi::ChatCompletions) .into(); assert!(provider.supports_batch()); ``` ### Mistral, Groq and Fireworks Mistral and Groq use the same lifecycle with their own features and provider values: `Mistral::from_env()` (`MISTRAL_API_KEY`) and `Groq::from_env()` (`GROQ_API_KEY`). Groq is also a first-class interactive Chat Completions driver, with model discovery and the shared message/tool codec. Only batch-compatible models are eligible; driver support does not imply account or model access. Fireworks separates its inference and management paths and requires an account. Configure exact trusted storage origins before submitting; obtain your account’s artifact origin from its storage configuration or a trusted existing dataset. For example, an account whose signed downloads use `storage.googleapis.com`: ```rust use everruns::{batch::Batch, providers::fireworks::{Fireworks, FireworksBatchConfig}}; let config = FireworksBatchConfig::new("my-fireworks-account")? .artifact_origin("https://storage.googleapis.com")?; let client = Batch::new(Fireworks::from_env()?.batch(config), "tenant/fireworks-account"); ``` Enable `fireworks`. The key comes from `FIREWORKS_API_KEY`; account and artifact policy stay explicit. Add each exact storage origin your account uses. Downloads carry only the signed URL, never provider authentication or configured headers; redirects and untrusted origins fail closed. A custom endpoint must serve both management `/v1` and inference `/inference/v1` paths on the configured origin. Changing the account requires a separate client even if its tenant scope was accidentally reused. Fireworks’ published API offers job deletion rather than a cancellation contract. `cancel` returns `BatchError::UnsupportedOperation`; it never deletes jobs or datasets. Manage deletion explicitly in the provider console after preserving needed results. Failed/cancelled/expired Fireworks jobs return `JobFailed`; this version does not collect partial datasets from those job states. ### Provider limits and terminal results | Driver | Submission boundary | Collection and cancellation | | ----------------------------------- | ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Anthropic | 100,000 requests, 256 MiB | Native result for every request; completed work remains collectable after cancellation | | OpenAI Responses / Chat Completions | 50,000 requests, 200 MB including this implementation’s small multipart envelope; one model | Merges output and error JSONL by `custom_id`. Cancelled/expired jobs may omit unprocessed requests: ticket IDs and validated provider counts identify these, preserving completed results | | Mistral | Inline submission only, at most 10,000 requests and 20 MB encoded body; one model, 24-hour job timeout | Native inline results or output/error files, correlated by `custom_id`; cancelled/expired omissions require verified counts and original ticket IDs | | Groq | 50,000 requests, 200 MB including the multipart envelope; one model, 24-hour processing window | Native output/error files and verified count mapping preserve completed requests on cancellation or expiry | | Fireworks | At most 50,000 requests and 100 MB including the multipart envelope; one model; provider default job window | Downloads up to 16 signed artifacts with an aggregate configured response-byte budget. Missing/duplicate/unknown request IDs fail closed | | Gemini Developer API | Inline only, 20 MB encoded body, at most 100,000 requests; one model | Maps `metadata.key`, including inline per-item errors. A failed/cancelled job without inline outputs returns `BatchError::JobFailed`; missing outputs are never invented | Account/model quotas can impose lower limits. Split jobs explicitly. Gemini’s Files mode and Vertex storage outputs are outside this implementation. OpenAI global validation failures also return `JobFailed`, separate from a list of per-item failures. Missing successful/error files or inconsistent counts fail collection rather than being relabelled as cancellation. OpenAI and Groq submission upload a `purpose=batch` file before creating a job. These are separate effects. An uncertain upload or job creation returns `SubmissionUncertain`; a definite rejection can still leave an uploaded file. The library does not automatically delete input/output files, cancel shared jobs, or follow provider-supplied URLs. Reconcile files and jobs in your own account, apply your retention policy and collect results before provider retention expires. Fireworks dataset creation, upload and job creation are separate external effects too. No stage is automatically replayed or deleted, including staging artifacts left after rejection. Persist intent before submitting and reconcile ambiguous responses through provider job/dataset listings before trying again. Mistral inline submission has one create operation; an ambiguous response still needs reconciliation. Provider retention and account quotas apply to all artifacts. See the current [Mistral batch lifecycle](https://docs.mistral.ai/studio/batch-processing), [Groq batch lifecycle](https://console.groq.com/docs/batch), [Fireworks batch lifecycle](https://docs.fireworks.ai/guides/batch-inference), [OpenAI batch lifecycle](https://developers.openai.com/api/docs/guides/batch) and [Gemini batch lifecycle](https://ai.google.dev/gemini-api/docs/batch-api). OpenAI batch function schemas explicitly use non-strict arguments, preserving optional properties without retaining original schemas in tickets. Structured text output remains available through OpenAI’s response format. Gemini batch rejects response formats, metadata, speed and verbosity options this encoder does not support. Stateful continuations, tool search, provider execution options and per-item credential/header overrides are rejected before submission. Function tool calls are returned as data, with no provider-hosted tool promotion. ## Runnable examples Run these examples from a repository checkout containing this change. Both examples call the real provider and incur its normal batch charges. Set the provider credential locally; do not put it in a ticket or command argument. **Classification across processes:** submit three reviews, then inspect or cancel the job and collect validated JSON classifications. Every command uses the same state directory and `BATCH_SCOPE`. `submit` requires a new directory, writes `intent.json` before the provider call, and saves `ticket.json` after acceptance. `collect` writes `results.json` without overwriting an earlier collection. ```bash export BATCH_SCOPE=tenant-42/anthropic-account-7 cargo run -p everruns --features anthropic --example batch_classification -- submit /tmp/reviews-001 cargo run -p everruns --features anthropic --example batch_classification -- poll /tmp/reviews-001 # Run when the poll says Ended: cargo run -p everruns --features anthropic --example batch_classification -- collect /tmp/reviews-001 # Or request cancellation, then poll and collect partial results: cargo run -p everruns --features anthropic --example batch_classification -- cancel /tmp/reviews-001 ``` Select a provider with `BATCH_PROVIDER` and enable its feature. Keep that selection, `BATCH_SCOPE` and state directory identical on later invocations: ```bash # Set OPENAI_API_KEY locally first; each command can incur provider charges. export BATCH_PROVIDER=openai export BATCH_SCOPE=tenant-42/openai-account-7 cargo run -p everruns --features openai --example batch_classification -- submit /tmp/openai-reviews-001 cargo run -p everruns --features openai --example batch_classification -- poll /tmp/openai-reviews-001 cargo run -p everruns --features openai --example batch_classification -- collect /tmp/openai-reviews-001 # Gemini uses GEMINI_API_KEY (or GOOGLE_API_KEY). BATCH_PROVIDER=gemini cargo run -p everruns --features gemini --example batch_eval # Mistral and Groq use MISTRAL_API_KEY and GROQ_API_KEY respectively. BATCH_PROVIDER=mistral cargo run -p everruns --features mistral --example batch_eval BATCH_PROVIDER=groq cargo run -p everruns --features groq --example batch_eval # Fireworks also needs explicit account and trusted storage origin. export FIREWORKS_ACCOUNT_ID=my-fireworks-account export FIREWORKS_ARTIFACT_ORIGIN=https://storage.googleapis.com BATCH_PROVIDER=fireworks cargo run -p everruns --features fireworks --example batch_eval ``` The [classification example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_classification.rs) defaults to `claude-sonnet-5`, `gpt-6.2-sol` or `gemini-4-argon` for the selected provider; set `BATCH_MODEL` on submission to choose another eligible model. Protect the state directory: intents and results contain user content even though tickets do not contain prompts. **Batch eval:** a single process submits an explicit list of message-based requests for three capital-city cases, polls every 60 seconds, and scores exact-match accuracy, correlating each answer by ID. This example keeps its ticket only in memory; restarting submits a new job. ```bash cargo run -p everruns --features anthropic --example batch_eval ``` See the [eval source](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_eval.rs) and the [example catalog](https://everruns.com/docs/framework/examples/). ## Retries, recovery and cancellation There are no automatic retries, resubmissions, background pollers, or cross-client batch pools. `BatchError::retryable()` identifies transient failures for retrying **that operation**. A failed poll or collection can use the same ticket again. For individual failed or expired items, select the work deliberately and create a new batch. Retain your source inputs separately; the ticket cannot reconstruct a prompt. **Uncertain submission is a separate outcome.** If a submit times out, loses its response, receives a server error, or returns an invalid acceptance response, it returns `SubmissionUncertain`. The provider may already be processing it. Do not automatically call `submit` again. Reconcile the intent and request IDs with the provider console/API first. If the process crashes between acceptance and saving the ticket, treat the surviving intent the same way. This API does not guarantee idempotent creation or exactly-once external effects; it cannot atomically commit a provider job and your database record. After reconciliation identifies the accepted job, `client.recover(job_id, original_request_ids).await?` checks access and returns a ticket without creating work. Supply IDs from the durable intent, then persist this ticket. The classification example exposes the same recovery path: ```bash # Only after identifying this intent's accepted job in the provider console/API: cargo run -p everruns --features anthropic --example batch_classification -- recover /tmp/reviews-001 msgbatch_YOUR_RECONCILED_JOB ``` `client.cancel(&ticket).await?` requests cancellation of the **whole job**. Cancellation is asynchronous: keep the ticket, poll until `Ended`, and collect available successes together with cancelled items, or handle the provider job-level failure described above. Completed or already running requests may still be billed. Dropping a client, ticket, or pending future does not cancel an external job. Retrying cancellation with the same ticket is safe; already ended jobs return `Ended`. ## Limits and isolation * Anthropic accepts at most 100,000 requests and 256 MiB per job. This driver rejects oversized submissions; split them explicitly. IDs are 1–64 ASCII letters, digits, underscores or hyphens. `max_tokens` must be positive. * Jobs can take up to 24 hours. Results are retained by Anthropic for 29 days from batch creation; collect them into your own storage before retention expires. Polling is not a streaming or interactive response path. * The client defaults to a 60-second timeout per HTTP request and a 256 MiB cap per response. Configure `Batch::timeout` and `max_response_bytes` for your workload. Streaming first-token/response limits, request-body capture, per-item headers and provider execution options are rejected by this batch driver. The timeout controls HTTP work, not the offline job’s lifetime. * Every client uses only its own provider credentials and endpoint. It rejects mismatched ticket scope, provider, endpoint, or format before network access. Choose a distinct stable scope for every tenant **and provider account**; the library cannot infer the account behind an API key. A scope is an application guard, not provider authorization. Never accept tenant identity or a ticket from an unauthenticated caller as proof of access. * Tickets are serialized routing data, not signed capabilities. Authorize their storage and use through your application. Do not pool work across tenants. Provider error bodies and endpoint URLs are excluded from batch errors. Result URLs are ignored except for Fireworks signed artifacts: those are fetched only from explicitly trusted origins with no provider credentials. Job operations stay on the configured provider origin. * Usage reports token counts, not a locally estimated bill. Consult the provider’s [current batch pricing and retention rules](https://platform.claude.com/docs/en/build-with-claude/batch-processing). For direct interactive calls, see [Direct Model Calls](https://everruns.com/docs/framework/direct-model-calls/). For sessions and their hosting guarantees, see [Hosting](https://everruns.com/docs/framework/hosting/). --- # Portable and hosted capabilities > Understand which capabilities run in the Framework and which require the Everruns Platform host. Source: The Framework advertises only capabilities that can run through its portable, in-process host contract. A capability reference remains an open value, so an application may retain configuration for an ID supplied by another host or plugin, but an unknown or hosted-only ID contributes no prompt, tools, or behavior in the default Framework runtime. ## Portable Framework capabilities Portable built-ins use services available from the Framework runtime or from an explicit application integration. Examples include files, session storage, current time, compaction, tool search, skills, and application-authored capabilities. Register application behavior with `#[everruns::tool]` or `everruns::capability` rather than depending on product internals. ## Hosted Platform capabilities Knowledge Bases and Knowledge Indexes, Memories, subagents and agent handoff, background/session tasks and schedules, user hooks, model scouting, OpenRouter workspace management, citations, and platform-management tools need hosted persistence or orchestration. Their implementations and narrow service contracts live in `everruns-capabilities`; the server and worker product presets register them explicitly. This boundary prevents the public Framework from promising tools whose stores, tenant scope, workers, or authorization services are absent. It does not change persisted capability IDs or JSON configuration. A specialized low-level host can depend on `everruns-capabilities`, install the required services, and select the hosted registry deliberately. Moving an agent to the Platform keeps its capability IDs and configuration; see [Moving to Platform or Cloud](https://everruns.com/docs/framework/moving-to-platform/). For application-owned behavior, continue with [advanced capabilities](https://everruns.com/docs/framework/advanced-capabilities/). For low-level host composition, see [custom backends](https://everruns.com/docs/framework/custom-backends/). --- # Capability integrations > Select filesystem, shell, web, Lua, and MCP implementation boundaries without pulling them into the Everruns kernel. Source: The Framework separates capability contracts from environment-backed implementations. `everruns-core` defines capability, tool, filesystem, egress, and MCP invocation contracts; focused crates own code that touches an interpreter, network transport, local process, or session filesystem. This keeps a custom host’s dependency and trust boundaries visible in `Cargo.toml`. It also prevents a core registry from silently granting an execution or network surface. ## Framework features | `everruns` feature | Default | Implementation | Effect boundary | | ------------------ | ------: | ---------------------------------------------- | ---------------------------------------------------------------------------------------- | | `filesystem` | Yes | `everruns-integrations` (`filesystem` feature) | Host-provided, session-scoped filesystem only | | `bashkit` | No | `everruns-integrations` (`bashkit` feature) | Sandboxed shell; HTTP remains capability-config and egress-policy gated | | `web-fetch` | No | `everruns-integrations` (`web-fetch` feature) | FetchKit requests through the host egress contract | | `lua` | No | `everruns-integrations` (`lua` feature) | Vendored Lua 5.4 sandbox; also requires `FEATURE_LUA=prod` at runtime | | `mcp` | No | `everruns-core` (`mcp` feature) | Remote HTTP MCP through the host egress contract | | `mcp-stdio` | No | `everruns-core` (`mcp` feature) | Adds local-process MCP servers and implies `mcp` | | `host-shell` | No | `everruns-core` (`host` feature) | `bash` tool over real host processes, bounded by a kernel policy; implies `host-compute` | \| `images` | No | `everruns-integrations` (`image-generation` feature) | [Image generation](https://everruns.com/docs/capabilities/image-generation/) (stills, GIFs, SVGs) through the host egress contract; keys from `OPENAI_API_KEY` and friends | | `duckduckgo` | No | `everruns-integrations` (`duckduckgo` feature) | DuckDuckGo web search through the host egress contract | | `a2a` | No | `everruns-capabilities` | Outbound A2A delegation to remote agents; implies `local` | | `otel` | No | `everruns-core` (`host` feature) | OpenTelemetry exporter; see [Observability](https://everruns.com/docs/observability/) | | `braintrust` | No | `everruns-core` (`host` feature) | Braintrust exporter; see [Observability](https://everruns.com/docs/observability/) | The default is offline: the filesystem capability can only use the session-filesystem implementation supplied by the host. Shell, web, Lua, MCP, and local-process transports require explicit features. ```toml [dependencies] everruns = { version = "0.34", features = ["bashkit", "web-fetch"] } ``` Enabling an implementation does not activate it on every agent. Add the matching capability reference to the agent, and retain the documented role, network-access, and runtime feature gates. In particular, Bashkit, web fetch, and Lua remain high-risk capabilities in the hosted product. ## Provider integrations Integration packages can expose typed values through `IntoCapability`. Brave Search supports the ordinary Framework builder: ```rust use everruns::{Agent, OpenAI}; use everruns_integrations::brave_search::BraveSearch; let agent = Agent::builder() .instructions("Search and cite primary sources.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability(BraveSearch::from_env()?) .build()?; ``` Depend on `everruns-integrations` with `default-features = false` and the `brave-search` feature. Add `brave-search-integration` to opt into Platform connector registration; leave it off for a Framework-only dependency. The Framework adapter reads `BRAVE_SEARCH_API_KEY` at construction; `BraveSearch::new` accepts an explicit application-owned key. Keys are retained privately by the client, never placed in capability JSON. Hosted execution continues to resolve connections and session secrets at tool execution time. Both paths use the same search schema and operation. Framework calls use the application’s direct HTTP client. ## Advanced host composition Advanced embedders select integrations on `everruns` and build the runtime registry through `everruns::batteries::runtime_capability_registry()`: ```toml [dependencies] everruns-core = "0.34" everruns = { version = "0.34", default-features = false, features = ["filesystem", "web-fetch"] } ``` ```rust let registry = everruns::batteries::runtime_capability_registry(); let egress = everruns::batteries::runtime_egress_service(); assert!(registry.has("session_file_system")); assert!(registry.has("web_fetch")); assert!(!registry.has("bashkit_shell")); ``` If the host starts from a caller-owned registry, preserve it and apply the same feature-selected integrations with `everruns::batteries::compose_runtime_capability_registry(registry)`. Hosted server and worker composition uses `everruns_capabilities::capabilities::hosted_capability_registry_for_grade` with the platform’s `environment-capabilities` feature. That preset preserves the hosted catalog while keeping the implementations outside core. Depend directly on a focused crate when you need its public implementation types. The former core paths move as follows: | Former public path | New public path | | ------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | | `everruns_core::FileSystemCapability` and filesystem tools | `everruns_integrations_filesystem::*` | | `everruns_core::BashkitShellCapability`, `BashTool`, and adapter | `everruns_integrations_bashkit::*` | | `everruns_core::WebFetchCapability` and `WebFetchTool` | `everruns_integrations_web_fetch::*` | | `everruns_core::LuaCapability` and `LuaCodeModeCapability` | `everruns_integrations_lua::*` | | `everruns_core::McpCapability` and MCP capability-ID helpers | `everruns_core::mcp::*` | | `everruns_core::DirectEgressService` | `everruns_core::host::DirectEgressService` with `direct-egress` | | `everruns_core::SystemEmailConfig` and Resend types | `everruns_capabilities::*` | | `everruns_core::ModelScoutCapability` and `OpenRouterWorkspaceCapability` | `everruns_integrations_openrouter::*` | | `everruns_core::OpenRouterServerToolsCapability` | `everruns_integrations_openrouter::OpenRouterServerToolsCapability` | | `everruns_core::{HumanIntentCapability, InfinityContextCapability, SkillsCapability, AttachSkillCapability, ToolApprovalCapability}` | `everruns_core::builtins::*` | | `everruns_core::{OpenUiCapability, A2UiCapability}` | `everruns_core::builtins::*` with `ui-capabilities` | | `everruns_core::skill::ProcessCommandExecutor` | `everruns_core::host::ProcessCommandExecutor` with the host `process` feature | Continue with [Configure and author capabilities](https://everruns.com/docs/framework/advanced-capabilities/) for agent-level activation or [Custom backends](https://everruns.com/docs/framework/custom-backends/) for host-level storage and orchestration. ## GitHub The `github` feature adds `everruns::github::GitHub`: native repository and PR tools with application-owned credentials, read-only by default. See [GitHub tools](https://everruns.com/docs/framework/github/) for token providers, write options, and the runnable PR review example. For application-bound Slack, Cursor, Browserless, E2B, SQL and knowledge services, see [Application services](https://everruns.com/docs/framework/service-integrations/). These typed bindings supply the native tools with their real credential/storage dependencies. --- # Code Mode with MCP > Let an agent call a large MCP server's tools from Bashkit scripts, so their schemas stay out of the prompt and many calls take one step. Source: Attach an MCP server with hundreds or thousands of tools natively and every model request carries every tool schema. Code mode moves those tools into the sandboxed [Bashkit shell](https://everruns.com/docs/capabilities/bashkit-shell/) instead. The model sees one `bash` tool, finds operations with `tools search`, and writes scripts that call them, join the results with `jq`, and print only the answer. The intermediate payloads never enter the conversation. ![Code mode over a 1,000-tool MCP server, terminal demo](https://raw.githubusercontent.com/everruns/everruns/main/examples/code-mode-mcp-agent/demo/demo.gif) The demo is the [Code Mode MCP Agent](https://github.com/everruns/everruns/tree/main/examples/code-mode-mcp-agent) example running `gpt-6.1-sol` against a synthetic MCP server with 1,000 tools (355 KiB of schemas). Each `❯ bash` block is a script the model wrote; each `↳ tools acme ...` line under it is an MCP tool that script called, with its input. The model’s tool list stayed at four tools and about 4 KiB, its largest prompt was about 3,500 tokens, and the answer matched totals computed from the server’s data. ## Declare it Code mode is three lines on the agent builder: the shell, the [Shell Code Mode](https://everruns.com/docs/capabilities/shell-code-mode/) capability in `cli` mode, and the MCP server. Enable the `bashkit` and `mcp-stdio` (or `mcp`) features. ```toml [dependencies] everruns = { version = "*", features = ["bashkit", "mcp-stdio", "openai"] } serde_json = "1" ``` ```rust use everruns::{Agent, BashkitShell, CapabilityRef, Engine, McpServer, OpenAI}; use serde_json::json; let agent = Agent::builder() .name("acme-analyst") .instructions( "The company APIs are available only in the shell, through the `tools` \ command: find operations with `tools search`, read one with `--help`, \ then do the work in one script and print only what you need.", ) .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability(BashkitShell::new()) .capability( CapabilityRef::new("shell_code_mode") .config(json!({"mode": "cli", "rate_unlabeled_tools": false})), ) .mcp_server(McpServer::stdio("acme", "acme-mcp-server")) .build()?; let session = Engine::new().create(agent); let turn = session .run("Which EU customers have overdue invoices? Give the top three by amount.") .await?; println!("{}", turn.response); ``` `mode: "cli"` removes every eligible tool from the model’s tool list and makes it a `tools ` command in the shell. `mode: "hybrid"` keeps the native route as well. `rate_unlabeled_tools: false` skips the decision service for tools that declare no read-only or destructive hint; leave it on when your server does not label its tools. Approvals still apply: a call that needs approval stops the script and asks. See [Shell Code Mode](https://everruns.com/docs/capabilities/shell-code-mode/) for every option, and [Large MCP catalogs](https://everruns.com/docs/features/large-mcp-catalogs/) for the limits on catalog size and discovery. ## What the model writes In a run of the example, after `tools search invoices customers`, the two operations’ `--help` and one script that sampled the data, the model wrote this script. It is copied unedited from that run; the two MCP calls below it ran inside the shell: ```bash set -euo pipefail fetch_all() { local op="$1" base="$2" cursor='' page args all='[]' while :; do args=$(jq -cn --argjson base "$base" --arg cursor "$cursor" '$base + {limit:500} + (if $cursor == "" then {} else {cursor:$cursor} end)') page=$(tools acme "$op" "$args") all=$(jq -cn --argjson all "$all" --argjson page "$page" '$all + $page.items') cursor=$(jq -r '.next_cursor // empty' <<< "$page") [[ -n "$cursor" ]] || break done printf '%s' "$all" } customers=$(fetch_all crm-list-customers '{"region":"EU"}') invoices=$(fetch_all billing-list-invoices '{"status":"overdue"}') jq -n --argjson customers "$customers" --argjson invoices "$invoices" '[ $customers[] as $c | [$invoices[] | select(.customer_id == $c.id)] as $overdue | select(($overdue|length)>0) | {customer:$c.name,total_overdue_usd:($overdue|map(.amount_usd)|add)} ] | sort_by(-.total_overdue_usd) | .[:3]' ``` ```text ↳ tools acme crm-list-customers {"limit":500,"region":"EU"} ↳ tools acme billing-list-invoices {"limit":500,"status":"overdue"} ``` A `tools` call prints the tool’s JSON reply as is, so a script pipes it straight into `jq`. When an MCP server answers in text, the text is printed unchanged. ## Watch the calls Each call a script makes is a `tool.nested_call` event on the session’s event stream, with the tool name, the command as the script spelled it, a shortened input, and a status. The model’s own `bash` calls stay ordinary `tool.started` and `tool.completed` events, so a host can tell the two apart: ```rust let mut events = session.events(); // ... run the turn ... while let Some(event) = events.try_recv()? { if event.event_type() == "tool.nested_call" { let data = &event.canonical_json()["data"]; println!("{} {}", data["command"], data["input_preview"]); } } ``` ## Run the example From a checkout of the repository: ```bash export OPENAI_API_KEY="your-key" # or OPENROUTER_API_KEY cargo run -p everruns-code-mode-mcp-agent ``` Add `-- --interactive` to type your own question. The MCP server is the same binary, started again with `--mcp-server`, so nothing outside the repository is needed. After the turn the example checks that no MCP tool reached the model’s tool list, that the model called none directly, that the calls ran inside scripts, and that the answer is right. A failed check exits nonzero. --- # Coordinator agents > One session hands focused work to threads, hears back from them, and resolves them. Source: A coordinator is an agent that does not do the focused work itself. The person talks to one session. That session starts a **thread** for each piece of work, relays follow-ups to it, and reports back when it finishes. Each thread keeps a checklist and ends its work with a summary. Every report wakes the coordinator with an automatic update, so nobody has to poll. This is the same model as Platform Chat’s threads, running inside your own process. ```rust use everruns::coordination::{self, Coordination, ThreadStatus}; use everruns::{Agent, Engine, LocalConfig, Model, WorkspacePolicy}; let agent = Agent::builder() .name("launch-coordinator") .instructions("Start one thread per piece of work. Tell me when each is ready.") .model(Model::simulated("On it.")) .capability(Coordination::new().max_active_threads(4)) .workspace_policy(WorkspacePolicy::read_write()) .local(LocalConfig::new("./launch/.everruns").workspace("./launch")) .build()?; let engine = Engine::new(); let session = engine.create(agent); session.run("Write the announcement and draft a pricing FAQ.").await?; for thread in coordination::threads(&session).await? { if thread.status == ThreadStatus::ReadyForReview { println!("{}: {}", thread.title, thread.summary.unwrap_or_default()); } } ``` Coordination needs the `local` feature and an agent built with [`.local(...)`](https://everruns.com/docs/framework/sessions/): threads, their assignments and their checklists live in the local profile’s SQLite store. ## What the coordinator does The capability gives the coordinator five tools: | Tool | Use | | ---------------- | --------------------------------------------------------------------------------------------- | | `start_thread` | Start a thread with a title and a brief that says what done looks like. | | `message_thread` | Relay a follow-up. After a thread finished, this opens a new unit of work on the same thread. | | `list_threads` | See every thread and where it stands. | | `get_thread` | Read one thread’s assignments and recent messages. | | `resolve_thread` | Close a thread once the person accepted its work, or reopen it. | Threads never talk to each other. Every route goes through the coordinator, so routing has one owner and one record. ## What a thread does A thread is an ordinary session of the same engine, running the coordinator’s agent. While it works an assignment it gets its own tools in place of the coordinator’s: `update_checklist`, `complete_assignment`, `ask_decision`, `report_to_coordinator` and `redirect_to_coordinator`. A thread turn that ends without `complete_assignment` flags the assignment as needing attention, so the coordinator always hears back. You can open a thread like any other session with [`Engine::resume`](https://everruns.com/docs/framework/sessions/) and its id from `coordination::threads`. ## Automatic updates When a thread finishes, asks a question or reports, the coordinator gets a user-role message marked as an automatic update. If the coordinator is idle, the update starts a turn. If it is mid-turn, the update steers that turn. Your application sees the coordinator’s answer in its history and events like any other reply. ## Status `coordination::threads` returns one entry per thread, from its latest unit of work: | Status | When | | ---------------- | ------------------------------------------------------------- | | `NeedsYou` | The thread asked a question, or its work failed. | | `Working` | The thread is working. | | `ReadyForReview` | The thread finished; the coordinator has not resolved it yet. | | `Open` | The work was cancelled and the thread was left open. | | `Resolved` | The coordinator resolved the thread. | ## Limits * Threads run the coordinator’s own agent. A local profile has no agent catalog, so another worker agent is refused. * A turn your application starts on a thread directly is not settled against its assignment. Send follow-ups through the coordinator. * The default caps are 8 threads working at once and 100 in total; set them with `max_active_threads` and `max_total_threads`. ## Example [`examples/coordinator-agent`](https://github.com/everruns/everruns/tree/main/examples/coordinator-agent) runs a launch coordinator with two threads that write their drafts to a shared workspace. `--offline` runs it with a deterministic simulated model; without it, the example uses OpenAI. ## Named members with separate behavior `AgentBuilder::member` composes a named agent with its own instructions, model and capabilities. Foreground delegation waits for its result; background delegation uses the Engine’s task services. Members remain separate sessions under the same host. The [named members example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/named_members.rs) uses a real lead model and a lighter reviewer model: ```bash cargo run -p everruns --features openai,local --example named_members ``` --- # Custom Backends > Decide when a Framework application should cross into low-level execution-host composition. Source: Most applications should use `everruns::Agent`, `Model`, and `Session`. That surface deliberately hides stored harness records, platform registries, backend stores, worker phases, and durable scheduling topology. For durable execution under Framework, implement [`SessionHosting` through the public hosting SPI](https://everruns.com/docs/framework/hosting/). Engine owns the runtime and recovery; the host supplies transactional session authority, epochs and independent activation. This path needs only an `everruns` dependency. To keep a host’s own capability implementations, provider resolution, retry budgets, MCP credentials or observers while Engine runs the sessions, use [host services](https://everruns.com/docs/framework/host-services/) instead of composing a runtime. Cross into separate low-level composition only when your application is itself an execution host, for example, a server, evaluation harness, research runtime, or specialized embedder that must replace storage or orchestration components. ## Host-level choices The low-level crates expose focused contracts for: * core agent, event, capability, and provider values; * the shared Input/Reason/Act kernel and sans-I/O turn planner; * runtime host phases, canonical event history, and in-memory reference stores; * local SQLite-backed task and schedule state; * portable session and agent definitions, plus explicitly selected durable deployment components. A separately composed low-level host depends on `everruns`, `everruns-core` with the `host` feature, and `everruns-contracts` for neutral service contracts. The low-level boundary is `everruns_core::host`; concrete integrations are selected on the facade. It is healthy for such a host to use low-level extension traits; the goal is not to re-export every backend through one facade. ## Two engine boundaries `everruns::Engine` is a concrete application object that owns the behavior registry, sessions, lifecycle, queries and activation through its hosting profile. It is the normal Framework entrypoint, not an extension trait. Applications do not implement it. `everruns_core::engine` is the shared execution kernel, enabled by the `engine` feature. Advanced hosts compose its `Execution` contract and serializable `TurnExecution` state machine, `InputAtom`/`ReasonAtom`/`ActAtom`, and phase values. The implementation lives in `everruns_core::host`, and the Platform’s workers drive the same turn loop under a session lease. Both use narrow contracts from `everruns-core`. The engine feature has no dependency on host effects, platform, server, worker, or durable crates. Do not copy state advancement or the phase loop into a custom backend; implement the execution boundary and keep deployment-specific service selection in the host. Where a turn runs is a separate interface: `everruns_core::host::TurnBackend` starts, cancels, and observes a session’s turns. `InProcessBackend` runs a turn in process. The legacy local/custom-backend facade lane uses `ActorRunner` under the session’s lease, and the Platform’s server marks the session runnable for a worker that leases it and runs the whole turn. Coherent Framework hosting instead enters Engine’s shared session actor path. `TurnBackend` is not the durable Framework extension point. A low-level host that records input through its own event store starts turns from it with `TurnInput::StoredMessage` and continues a parked turn with `TurnInput::RecordedToolResults`, which every backend serves. The trait is public and unsealed but experimental. The facade’s [backend conformance suite](https://github.com/everruns/everruns/tree/main/crates/everruns/tests/backend_conformance) defines the behavior every backend must match. See [Framework Architecture](https://everruns.com/docs/framework/architecture/) for the complete layer map and the distinction between immediate and durable execution. Conversation persistence is the one backend with a single write path. Replace it by implementing the canonical `EventLog`/`EventReader` SPI and passing it to `HostBackends::with_event_log`; the required snapshot, continuation, and polling behavior is specified in [Implementing a custom event log](https://everruns.com/docs/framework/events-and-cancellation/#implementing-a-custom-event-log). ## Security boundary Backend replacement does not relax tenant, credential, filesystem, or tool execution boundaries. Preserve event ordering, credential redaction, workspace containment, and cancellation behavior when adapting the host. A custom backend must fail explicitly when it cannot satisfy a required contract. --- # Data services > Use persistent session SQL, named memory directories, and application-owned citable knowledge retrieval in Framework agents. Source: Data services attach to real Engine tool contexts. They require no Server database. Choose storage directories outside the agent workspace so filesystem tools cannot rewrite service state. ## Persistent SQL Enable `sql-database` and add `SqlDatabase::open(directory)`: ```rust use everruns::{Agent, Model, database::SqlDatabase}; let agent = Agent::builder() .instructions("Use SQL to retain structured notes.") .model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?)) .sql_database(SqlDatabase::open(".private/session-databases")?) .build()?; ``` This enables `sql_execute`, `sql_query`, and `sql_schema`. Databases are isolated by Engine session identity; retaining a SQL directory alone does not retain Engine sessions. Reopen your Engine’s durable hosting profile and reconstruct its agent registrations to resume the same sessions. The local backend uses the Server’s SQLite authorizer, statement execution, validation, row/byte limits and timeouts. `ATTACH`, extension loading, unsafe pragmas and virtual tables are denied. File locking coordinates quota checks across independent store instances/processes. Database names cannot select host paths. This is an agent-tool boundary; it does not isolate arbitrary host shell processes from storage directories. ## Named memory ```rust use everruns::{Agent, Model, MemoryDirectory}; let agent = Agent::builder() .instructions("Read policy from /memory/policy; save notes in /memory/notes.") .model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?)) .capability("session_file_system") .memory(MemoryDirectory::directory("policy", "./policy").read_only()) .memory(MemoryDirectory::directory("notes", "./notes")) .build()?; ``` Names are distinct path components below `/memory/`. The application decides which sessions share a directory. Read-only mounts reject creation, editing, deletion and directory creation through every filesystem tool, while keeping host file permissions unchanged. Directory bindings are trusted host configuration. They provide no containment for tools that execute arbitrary native host processes. ## Knowledge bases and indexes Enable `knowledge`. `Knowledge::documents` supplies a small deterministic keyword retriever; `Knowledge::directory` snapshots Markdown and text files with bounded enumeration and byte limits. Symlinks inside the snapshot are rejected. ```rust use everruns::{Agent, Model, knowledge::{Knowledge, Document}}; let source = Knowledge::documents("support", vec![Document::new( "Refund policy", "Refund requests are accepted within 30 days.", "https://example.com/policy/refunds", )]); let agent = Agent::builder() .instructions("Search policy before answering and cite the source URI.") .model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?)) .knowledge(source) .build()?; ``` `search_knowledge` returns ranked passages with immutable file/version identities, source URIs and optional locations. `browse_knowledge` pages through bound originals and metadata; read and grep them under `/knowledge//`. Only the application-bound source is accessible. For semantic/vector search, implement `Retriever::search` and return `Passage` values through `Knowledge::new`. Attach originals with `.catalog(documents)` when the provider should also support browsing and mounted reads. The retriever owns embeddings, refresh and provider billing. Result counts, snippets and citation metadata are bounded, and provider errors are sanitized. Enable `.automatic(true)` to retrieve source evidence before model execution; it is retained by input ID for replay in protected host service storage. The [complete data example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_data.rs) runs a real agent that retrieves policy, persists a SQL decision and writes named memory: ```bash cargo run -p everruns --features openai,sql-database,knowledge --example service_data ``` --- # Deploying a Framework App > Ship a Framework app as a binary or container: persistent storage for its data directory, provider keys from the environment, and when to use serve instead. Source: A Framework app is your own Rust binary. The `everruns` crate needs no database, message broker, or Everruns server, so deploying it is deploying that binary: build it and give it provider credentials. If sessions must outlive the process, select durable session hosting and preserve its storage and registered Agent builds. A persistent directory alone is not an independent recovery mechanism. ## Choose where state lives | Engine setup | State | Deploy with | | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- | | `Engine::new()` | In memory, lost when the process exits | Any binary or container. Nothing to persist | | Engine with `hosting::SqliteHosting` | Transactional session authority, canonical events, admission receipts and wakeup obligations | Persistent SQLite storage, retained Agent builds and Environment factories, and a running native supervisor | | An agent built with legacy `.local(LocalConfig::new(dir))` | Local events, session catalog, and SQLite task and schedule state under `dir`; agent files under `dir/workspace` unless `.workspace(path)` sets another root | A persistent volume and application-driven reattachment and recovery | `SqliteHosting` and `LocalConfig` need the `local` feature. New durable services should select `SqliteHosting` on `Engine::builder()`, register immutable `AgentBuild` identities and start `SqliteHosting::supervise` at startup, independently of incoming requests. Workspace files, compute and secrets need explicit services with their own persistence guarantees. See [session hosting](https://everruns.com/docs/framework/hosting/) and [Sessions](https://everruns.com/docs/framework/sessions/#persistence) for the recovery boundaries. ### Existing LocalConfig deployments `AgentBuilder::local` is deprecated. The following guidance and example apply to that legacy lane; use [legacy session cutover](https://everruns.com/docs/framework/sessions/#moving-legacy-sessions-to-sqlitehosting) to move existing sessions to coherent Engine hosting. For a legacy deployment that keeps local state: * **One process per data directory.** The local profile is built for one embedded process at a time. Do not point two replicas or two containers at the same directory. Scale by giving each instance its own directory and routing each session to the instance that owns it. If two processes do share one, a session’s turns still run in only one of them at a time: each turn holds the session’s lease in the local database. * **Trusted paths only.** Choose the data and workspace directories from deployment configuration, never from model output or request input. * **Persistent storage.** Mount the data directory from a volume. New state files are created owner-only on Unix; your backups and copies need the same protection, because the event log holds conversation content. * **Resume after a restart.** Rebuild each agent from the same trusted configuration, then call `engine.attach(session_id, agent)` and `engine.resume(session_id)`. See [Resume after a process restart](https://everruns.com/docs/framework/sessions/#resume-after-a-process-restart). ## Advanced local hosts Hosts built with `everruns::local::LocalRuntimeBuilder` reconcile orphaned background executions after constructing their runtime and start periodic maintenance. Recovery requires executor evidence that ownership is lost. A missing owner process leaves the execution outcome unverified; interrupted commands are never replayed. When composing `LocalBackends` directly, retain its recovery-wrapped task registry, bind `task_recovery` to the runtime’s task wake observer, then call `recover_all` and `start_maintenance`. Recovery wakes are delivered at least once across crashes; a durable observer must deduplicate by session, task ID and terminal attempt. ## Pass secrets through the environment Each provider type reads its key with `from_env`: `OpenAI::from_env()` reads `OPENAI_API_KEY` and `OPENAI_BASE_URL`, `Anthropic::from_env()` reads `ANTHROPIC_API_KEY`, and so on. The full per-driver table is under [Credentials](https://everruns.com/docs/framework/models-and-providers/#credentials). Set those variables from your platform’s secret store (a container secret, a Kubernetes `Secret`, a systemd credential), not in the image. `from_env` returns an error when a required variable is missing, so a misconfigured deployment fails at startup instead of on the first turn. When your application already resolves the key itself, pass it with `OpenAI::new(key)`. On AWS, `Bedrock::default_chain()` uses the instance or task role, so no key is stored at all. A legacy `LocalConfig` service entry point: ```rust use std::time::Duration; use everruns::{Agent, Engine, LocalConfig, OpenAI}; // Deployment configuration, not request input. let data_dir = std::env::var("APP_DATA_DIR").unwrap_or_else(|_| "/data".into()); let agent = Agent::builder() .instructions("Answer support questions.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .local(LocalConfig::new(data_dir)) .build()?; let engine = Engine::new(); let session = engine.create(agent); // ... accept requests and drive sessions ... tokio::signal::ctrl_c().await?; engine.shutdown(Duration::from_secs(10)).await; ``` `APP_DATA_DIR` is this example’s own setting; the Framework reads no variable for the data directory. `Engine::shutdown` drains and flushes event listeners such as the OpenTelemetry and Braintrust exporters, so call it before the process exits. See [Observability](https://everruns.com/docs/framework/observability/). ## Build a container A multi-stage build keeps the toolchain out of the runtime image. This follows the Dockerfile in [AgentCore app](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore): ```dockerfile FROM rust:1-bookworm AS build WORKDIR /src COPY . . RUN cargo build --release --locked --bin my-agent FROM gcr.io/distroless/cc-debian12:nonroot COPY --from=build /src/target/release/my-agent /my-agent VOLUME /data ENTRYPOINT ["/my-agent"] ``` The `everruns` crate’s HTTP clients use rustls, so the image needs no OpenSSL. It still needs CA certificates to reach provider APIs; the distroless `cc` image includes them. The `nonroot` user must be able to write the mounted data directory. Run it with the key from a secret and the data directory on a volume: ```bash docker run --rm \ -e OPENAI_API_KEY \ -e APP_DATA_DIR=/data \ -v my-agent-data:/data \ my-agent ``` A volatile Engine needs no state volume. A container using `SqliteHosting`, `LocalConfig` or persistent workspace services needs storage that survives replacement. ## When to use serve instead [serve](https://everruns.com/docs/framework/serve/) is an experimental application framework on top of the Framework. Use it instead of a hand-written binary when you want: * the Everruns `/v1` session API, so the SDKs, CLI, and UI chat view can drive your agents, plus AG-UI and A2A endpoints; * a manifest of agents, tools, secrets, and models that a host can provision from; * a `start` command that requires every declared secret, and that fails a model with no real route instead of falling back to the simulator. Hosts configure `OPENAI_API_KEY` for `openai/…` models, optionally with `OPENAI_BASE_URL` for a compatible Responses API endpoint, or use `OPENROUTER_API_KEY` for provider-qualified model IDs. A serve binary listens on `0.0.0.0`, on `PORT` (default 3000). Its state goes to the SQLite file named by `DATABASE_URL`, else to `SERVE_DATA_DIR`, else to `.serve/` in the working directory, or, with `start --store s3://bucket/prefix`, to a bucket that a daemon on another machine can take over (see [Keep the data in a bucket](https://everruns.com/docs/framework/serve/#keep-the-data-in-a-bucket)). Its wire API is unauthenticated by default. Configure [serve authentication](https://everruns.com/docs/framework/serve/#authentication) or place it behind an authenticating host before exposing it. See [Serve on AgentCore](https://everruns.com/docs/framework/serve-agentcore/) for Amazon Bedrock AgentCore; the [celld adapter](https://everruns.com/docs/framework/serve-celld/) is a separate source-only reference, not a published deployment integration. Keep a plain Framework binary when the agent is embedded in an application that already owns its API, authentication, and storage. To move to a managed, multi-tenant runtime instead, see [Moving to Platform or Cloud](https://everruns.com/docs/framework/moving-to-platform/). ## Connect custom code to a control plane Use an experimental [custom agent worker](https://everruns.com/docs/how-to/custom-agent-workers/) when your binary should own private tools and model access while the self-hosted Platform owns user sessions, event history and tracing. The [data-agent walkthrough](https://everruns.com/docs/framework/examples/custom-data-agent/) includes container packaging and a real-model end-to-end smoke test. --- # Direct Model Calls and Decisions > Call a model once through the Framework's provider edge, or ask Decisions for a number instead of prose, without building an agent. Source: Some work is one prompt and one answer: classify a string, draft a summary, extract a field. That needs the provider edge — drivers, endpoints, credentials, retries, error classification — but none of the agent loop around it. `Model::complete` is the whole API for that case: ```rust use everruns::{Model, OpenAI}; let model = Model::new("gpt-6.2-sol", OpenAI::from_env()?); let answer = model.complete("Name the three primary colors.").await?; println!("{answer}"); ``` The model is the same value an [agent](https://everruns.com/docs/framework/agents/) takes, reached through the same [`Provider`](https://everruns.com/docs/framework/models-and-providers/), which can also be asked [which models it offers](https://everruns.com/docs/framework/models-and-providers/#model-catalogs). Nothing is persisted: a direct completion owns no session, no history, and no workspace. Reach for an agent as soon as the work needs tools, multiple turns, durability, or events. When the answer should be a number rather than prose, such as a probability, a choice among options, or a graded score, use [Decisions](#decisions) instead. ## Testing without a provider `Model::simulated` returns canned responses from an in-process simulator (`everruns-llmsim`). It is a **test double**, not a local model: it runs no inference and is not a way to use Everruns without a model provider. It exists so tests and examples can assert on agent behavior without a network call or an API key. Real work always goes through a provider — see [Supported providers](https://everruns.com/docs/framework/models-and-providers/#supported-providers). ```rust use everruns::Model; let answer = Model::simulated("4").complete("What is 2 + 2?").await?; assert_eq!(answer, "4"); ``` See [Testing and simulation](https://everruns.com/docs/framework/testing-and-simulation/) for scripted multi-response simulators. ## System messages, context, and controls `Model::completion` describes the call before sending it. Messages append in call order; each control maps to one provider request field and stays unset unless assigned, so the provider keeps its own defaults. ```rust use everruns::{Model, ReasoningEffort}; let response = model .completion() .system("Answer with a single word.") .user("What is the capital of France?") .max_tokens(16) .reasoning_effort(ReasoningEffort::Low) .send() .await?; println!("{}", response.text); println!("{:?} tokens", response.metadata.total_tokens); ``` `send` returns the full `LlmResponse` — text, reasoning artifacts, tool calls, and call metadata. `text()` returns only the answer text. Replay prior turns with `.assistant(...)`: the completion carries no history of its own, so context is whatever the call passes. When the model is a bare provider-visible id, attach the provider on the completion instead of the model: ```rust use everruns::{Model, OpenAI}; let answer = Model::from("gpt-6.2-sol") .completion() .provider(OpenAI::from_env()?) .user("Summarize this in one line: ...") .text() .await?; ``` ## Streaming `stream()` returns the provider’s events as they arrive, ending with a `Done` event carrying the call’s metadata: ```rust use everruns::{LlmStreamEvent, Model}; use futures::StreamExt; let mut stream = model.completion().user("Write a haiku.").stream().await?; while let Some(event) = stream.next().await { if let LlmStreamEvent::TextDelta(delta) = event? { print!("{delta}"); } } ``` What the stream guarantees: * Every `TextDelta` carries text. Assistant text is their concatenation. * `ReasoningDelta` streams reasoning live; the `ReasoningItem` that follows repeats the whole block, plus any replay state. Show the deltas and store the item, or use the item alone; adding both duplicates the reasoning. * The stream ends with exactly one `Done` or `Error`. `Error` keeps the provider’s message, error code, and HTTP status, including errors a gateway reports inside a `200` stream. * `Done` carries `finish_reason` as the provider sent it, or `None` when it sent none. It is never filled in as `stop`, so a caller can treat a missing reason as a failure. * Drivers retry `429` and transient `5xx` responses before the first event: up to 2 retries and 30 seconds by default. That time counts against any timeout you put around the call. To own retries yourself, build the driver with `.with_retry_config(everruns::llm::LlmRetryConfig::no_retry())`. The full contract is on `LlmStreamEvent` in the API reference. ## Errors `CompletionError` separates configuration mistakes from provider failures: * `MissingProvider` — the model names an id but nothing says how to reach it. * `NoMessages` — the completion was sent empty. * `Call(..)` — the provider call failed, carrying the `AgentLoopError` and its full `LlmError` decision. The first two are caught before any request leaves the process. ## Going lower `Completion` is a thin value-first layer over `Provider`, which is public. Applications that already hold a `Provider` — or implement their own [`ChatDriver`](https://everruns.com/docs/framework/models-and-providers/#custom-providers) — can call it directly with `everruns::llm`’s `Message` and `MessageRole`, plus the crate-root `LlmCallConfig` and `LlmResponse` re-exports: ```rust use everruns::llm::{Message, MessageRole}; use everruns::{LlmCallConfig, Provider}; let response = provider .chat_completion( vec![Message::text(MessageRole::User, "What is 2 + 2?")], &LlmCallConfig::new("gpt-6.2-sol"), ) .await?; ``` That surface is the driver boundary itself: every field of `LlmCallConfig`, including tool definitions, is available, and nothing is defaulted for you. When all you have is a base URL, `DriverId::for_base_url` names the driver whose vendor serves it (OpenAI, Azure OpenAI, OpenRouter, Anthropic, Gemini, Fireworks). It returns `None` for a gateway or self-hosted server; those speak the OpenAI-compatible wire, so use `DriverId::OpenAICompletions`. The runnable version of this section is [`direct_llm.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_llm.rs), which runs offline without an API key. ## Decisions Some questions have typed answers. *Is this claim supported by the source? How severe is this complaint? Which queue does this ticket belong in?* A chat model answers those in prose, so the call site ends up with a prompt asking for JSON, a parser, and a fallback for when the parse fails. A decision service answers them as numbers instead, and the decision stays in your code. This is the counterpart to a direct model call: the same shape, a different contract. | | `Model` | `Decisions` | | ------------------- | ----------------- | ---------------------------- | | you send | messages | state plus typed questions | | you get back | text | calibrated numbers | | decides the outcome | the model’s words | your threshold, in your code | | streams | yes | no — one round trip | ### Quick start ```bash cargo add everruns --features typesafe cargo add tokio --features macros,rt-multi-thread export TYPESAFE_API_KEY=... # a key from typesafe.ai ``` ```rust use everruns::{Decisions, TypeSafeAI}; #[tokio::main] async fn main() -> Result<(), Box> { let decisions = Decisions::new("jev-latest", TypeSafeAI::from_env()?); let text = "CONGRATULATIONS! You've WON $1,000,000. Click here to claim your prize now!"; let spam = decisions.probability("Is this message spam?", text).await?; println!("spam: {spam:.2}"); if spam > 0.9 { println!("quarantined"); } Ok(()) } ``` ```text spam: 0.98 quarantined ``` One number, and your own `> 0.9` decides — the model reports how likely, not what to do. The same call answers `0.03` for “Standup moved to 10am.” and `0.74` for a bare “Claim your prize now!”; the middling ones are what a threshold is for. `--features typesafe` adds `TypeSafeAI` and the `Jev` capability. Without it `everruns` names no vendor: `Decisions::new` takes any `DecisionsService`. ### OpenAI’s Decisions API [OpenAI’s Decisions API](https://developers.openai.com/api/docs/guides/decisions) (public beta, model `gpt-6-luna`) answers the same questions. `OpenAIDecisions` in `everruns-integrations` is a `DecisionsService`, so it drops into the same `Decisions`: ```bash cargo add everruns cargo add everruns-integrations --features openai-decisions export OPENAI_API_KEY=... ``` ```rust use everruns::Decisions; use everruns_integrations::openai_decisions::OpenAIDecisions; let service = OpenAIDecisions::new(std::env::var("OPENAI_API_KEY")?); let decisions = Decisions::new("gpt-6-luna", service); let urgent = decisions .probability("Does this convey urgency?", "My card was charged twice, fix it today.") .await?; println!("urgent: {urgent:.2}"); ``` A runnable version that routes a support ticket with all three primitives is [`openai_decisions_triage`](https://github.com/everruns/everruns/blob/main/crates/integrations/examples/openai_decisions_triage.rs): ```bash OPENAI_API_KEY=... cargo run -p everruns-integrations --features openai-decisions --example openai_decisions_triage ``` All three primitives are native and calibrated, and one request sends every question in one call. Score questions take at most ten levels. On Everruns Platform, set `UTILITY_DECISION_DRIVER=openai` with `UTILITY_OPENAI_API_KEY` to answer guardrail `jev` checks with it (see [environment variables](https://everruns.com/docs/sre/environment-variables/)). ### Three primitives A decision asks one or more questions about the same state. Each is one of three shapes: ```rust use everruns::Decisions; let answers = decisions .about("I've been on hold for two hours and my card was charged twice.") .noul("urgent", "Does this convey urgency?") .score( "severity", "How severe is the problem the writer describes?", [ "A minor annoyance", "A real problem with their account", "Serious harm requiring immediate action", ], ) .choice( "queue", "Which team should handle this message?", ["billing", "technical", "sales"], ) .send() .await?; let urgent: f64 = answers.probability("urgent")?; let queue: &str = answers.selected("queue")?; ``` * **`noul`** — whether something holds, as the probability of yes. A value near 0.5 means yes and no are near-equally likely, not “medium”. ([Noul](https://docs.typesafe.ai/primitives/noul)) * **`choice`** — exactly one option from your set, with the distribution behind it. Needs at least two options. ([Choice](https://docs.typesafe.ai/primitives/choice)) * **`score`** — a position along levels you define, lowest first. Needs at least two levels. ([Score](https://docs.typesafe.ai/primitives/score)) The three are System One’s own, so TypeSafe’s [Primitives](https://docs.typesafe.ai/primitives) documents what each answer means and how to choose between them, and [State](https://docs.typesafe.ai/concepts/state) covers what to put in the `about(...)` value. Everruns names them the same way rather than inventing synonyms. Questions in one call are answered **in parallel inside a single request**, so asking five costs one round trip, not five. TypeSafe calls leaning on that [speculative fan-out](https://docs.typesafe.ai/patterns/fan-out): ask the questions you *might* need, and let your code decide which ones mattered. ### Ids are yours; instructions are the model’s The id labels the answer for your code and is never sent to the model. A question whose meaning lives in its id asks nothing: ```rust // Wrong: the model never sees "is_the_joke_funny". .noul("is_the_joke_funny", "?") // Right: the instructions carry the question. .noul("funny", "Would a general audience laugh at this joke?") ``` The same applies to score levels. Describe concrete situations — “A minor annoyance” reads on its own where “2 out of 5” does not. ### Read the tail, not the average For “is there any serious hit here” rules, read the probability mass at or above a level rather than the weighted score. Something probably fine but possibly awful must not average into fine: ```rust // Not: answers.score("severity")? > 1.5 let serious = answers.tail("severity", 2)?; if serious > 0.3 { println!("escalated"); } ``` ### Errors `DecisionsError` separates configuration mistakes from service failures, the same split [`CompletionError`](#errors) makes: * `MissingService` — the decisions was built without a service to reach. * `NoQuestions` — the decision was sent with nothing to ask. * `Unconfigured` — the service exists but the deployment never configured its credential, so it would answer nothing. * `NoSuchAnswer(id)` — you read an id that was not asked, or read an answer as the wrong shape (a `score` as a probability). * `Call(..)` — the service call failed, carrying the `AgentLoopError`. The first three are caught before any request leaves the process. `Unconfigured` is worth handling separately: a guardrail treats it as fail-open, but a direct caller usually wants to know the number never arrived rather than read a confident-looking default. ### Going lower `Decision` is a thin value-first layer over `DecisionsService`, which is public. Applications that already hold a service — or implement their own, over a different vendor or a local model — can call it directly with `everruns`’s `DecisionRequest`, `DecisionQuestion`, and `DecisionAnswer` re-exports: ```rust use everruns::{DecisionQuestion, DecisionRequest, DecisionsService}; let outcome = service .evaluate( DecisionRequest::new("Claim your prize now!") .ask("spam", DecisionQuestion::noul("Is this message spam?")), ) .await?; ``` That surface is the contract itself: every question type and the full `DecisionOutcome`, including usage, with nothing defaulted for you. Implementing `DecisionsService` is also how a different decision service (another vendor, or a fine-tuned local model) plugs into the same `Decisions`, guardrails included. ### Choosing a model The model is named up front, the way `Model::new` names one: the service is transport, and the model is the thing that answers. There is no default to inherit without noticing, because a threshold calibrated against one version is not evidence about the next. Ids are the provider’s own, so they are spelled the way the vendor spells them. `jev-latest` is TypeSafe’s alias for the current Jev, so it tracks whatever the current version is; an exact id like `jev-1.14.0` pins one, so a vendor update cannot move your thresholds under you. Bare `jev` is not an id the API knows — nothing here rewrites what you pass. Ask for the alias and read back what answered, which is the id to pin once a threshold is calibrated: ```rust let version = answers.model(); // "jev-1.14.0" for a "jev-latest" request ``` A single call can name a different model with the same method on the request builder, and it wins for that call: ```rust let answers = decisions .about("...") .noul("urgent", "Does this convey urgency?") .model("jev-1.14.0") .send() .await?; ``` A deployment that must pin a model does so by never exposing the knob in the config an agent writes — not by the type being unable to carry one, because there will be other decision services and other models. A deployment running the Everruns platform configures a separate `UTILITY_TYPESAFE_API_KEY` for its [guardrails](https://everruns.com/docs/capabilities/guardrails/) — a different account from the one an embedding application holds. ### Giving an agent the decisions Everything above is agentless: your code asks, your code decides. The other half is letting an *agent* classify as part of its own work — checking a claim against a source before citing it, rating a draft before sending it. `Jev` is the same decisions as a capability, so an agent gets it as a tool: ```rust use everruns::{Agent, Engine, Jev, Model, OpenAI}; let agent = Agent::builder() .name("reviewer") .instructions( "You review copy. When asked how something reads, measure it with \ jev_decision and report the numbers rather than judging by eye.", ) .model(Model::new("gpt-6.2-sol", OpenAI::from_env()?)) .capability(Jev::from_env()?) .build()?; let session = Engine::new().create(agent); let turn = session .run("Rate this subject line for pushiness: 'Act now before it is too late'") .await?; ``` The agent calls `jev_decision`, writing its own questions about whatever it is looking at, and gets the same calibrated numbers back. It is the identical tool the hosted [TypeSafe integration](https://everruns.com/docs/integrations/typesafe/) gives platform agents — same name, same schema — so behavior matches whether you embed the Framework or run on Everruns. Which one to reach for: | | you decide | the agent decides | | ------------ | -------------------------------------- | --------------------------------- | | **who asks** | your code writes the questions | the model writes the questions | | **use** | `Decisions` | the `Jev` capability | | **good for** | a policy check, a routing rule, a gate | verification inside a longer task | ### What stays with an agent A decision owns no session, no history, and no workspace, and runs no tools. Reach for an [agent](https://everruns.com/docs/framework/agents/) as soon as the work needs any of those. Typed output guarantees the interface, not the truth: validate thresholds against your own data and consequences. ### Testing without a credential `Decisions::simulated` returns a fixed number from an in-process stub. It is a **test double**, not a local decisions: it runs no inference, reads nothing from the state you pass it, and is not a way to classify without a provider. It exists so tests and examples can assert on the code around a decision without a network call or an API key. Real work always goes through a decision service: `TypeSafeAI` above, or your own `DecisionsService`. ```rust use everruns::Decisions; let p = Decisions::simulated(0.93) .probability("Does this convey urgency?", "Two hours on hold.") .await?; assert!(p > 0.9); ``` Because the answer is fixed, a simulated decision proves your threshold logic runs — never that a real decision service would return that number. The runnable version of this section is [`direct_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_decisions.rs). It uses the stub by default so it runs with no key; pass `--live` (with `--features typesafe` and `TYPESAFE_API_KEY` set) to send the same questions to a real decision service. [`agent_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/agent_decisions.rs) does the same for the agent path. ### Shared provider accounts for decisions `Decisions::from_registry` accepts a `ProviderRegistry` and an explicit `ModelSpec`. Select `ModelSpec::on("typesafe", "jev-1.14.0")` for direct TypeSafe, or `ModelSpec::on("openrouter", "jev-1.14.0")` for OpenRouter. The OpenRouter driver maps that explicit alias to `typesafe/jev-1.14`; other IDs remain opaque. Providers hold authentication once, so their chat and decision drivers use the same account. Returned model IDs report the snapshot that actually answered. `Jev::with_provider(provider, model_spec)` supplies that same account to a Framework capability. Use its exact provider key in the model spec. Existing `Jev::new` and `Jev::with_client` conveniences remain available for applications holding a direct TypeSafe key. ## Offline batches For independent requests that can wait hours, use [Batch Model Requests](https://everruns.com/docs/framework/batch-model-requests/) to submit through Anthropic Message Batches and persist a ticket for later collection. ## Model refusals The Responses driver reports explicit model refusals as `LlmErrorKind::Refusal` for streamed calls, native JSON completions, and batch results. These are terminal decisions and are not automatically retried. The diagnostic retains the provider’s reason, bounded to 16 KiB; treat it as provider content when displaying or logging it. Engine sessions apply their configured error disclosure policy and expose the stable `model_refusal` category instead of treating refusal text as a successful answer. --- # Durable Object hosting reference > Run ordinary Everruns Rust agents with durable admission, leases, event logs and alarm-driven recovery owned by a Durable Object. Source: The source-only `everruns-durable-object` reference demonstrates the [pluggable session hosting SPI](https://everruns.com/docs/framework/hosting/) beneath `everruns::Engine`. It is independently resolved and is not a published Framework dependency or a first-class Cloudflare service integration. Application code defines ordinary Framework Agents and uses the same Session API on native execution and Wasm inside an object. The reference and deployment APIs remain experimental; their guarantees cover the failure domains exercised by the qualification tests. ![Durable Object authority: a Framework Engine admits input to a Durable Object, whose alarm activates a native or Wasm Engine executor. The executor writes fenced journal and state back to the object, and runs the shared Everruns runtime with providers and tools.](https://everruns.com/docs/_astro/durable-objects.X7FOrkCF.svg) Create an Engine with `ObjectHosting::new(client)` and register trusted Agent builds under immutable `AgentBuild` identities. `create_registered` asynchronously commits a session binding at a caller-retained session ID. `Session::send_with_id` commits input, its deduplicated receipt and an independent wakeup obligation. Retain those identities across uncertain transport responses. Admission precedes turn assignment. Reopen an admission by its message identity to observe pending assignment, a durable wait, suspension or finalized outcome. Dropping a handle or SSE connection does not cancel accepted work. Approval and tool answers name their exact logical turn; cancellation names a retained input. Acceptance of cancellation is separate from reaching a cancelled terminal state. Framework start/completion hooks run on the executor under the same ownership epoch. Invocation records conservatively handle uncertain external effects; explicitly idempotent hooks can opt into recovery. A turn result becomes final only after its awaited Framework completion hooks settle. ## Native and isolate placements The [native example](https://github.com/everruns/everruns/tree/main/crates/durable-object/examples) registers an ordinary Agent on Engine and supplies that Engine to an authenticated `Executor`. Activation validates the authoritative binding and retained registry. The [Wasm application](https://github.com/everruns/everruns/tree/main/examples/durable-object-agent) uses the same [Agent definition](https://github.com/everruns/everruns/blob/main/crates/durable-object/examples/shared/agent.rs), which imports only `everruns`. Its Workers `/v1` adapter calls Engine for creation, admission and queries, and uses bounded canonical pages for SSE replay. JavaScript handles transport, storage and alarms; Rust owns agent and tool policy. [serve on celld](https://everruns.com/docs/framework/serve-celld/) selects this hosting profile and exposes its same Engine for native activation. The celld isolate bootstrap imports the shared Wasm application. There is no application runtime factory or celld step machine to implement. ## Hosting boundaries Keep API, authority and executor credentials separate. Tenant scope comes from trusted host configuration. Privileged authority credentials stay on trusted hosts; browsers use the user API. Production native authority/executor endpoints require HTTPS. Retain exact builds and Environment references while sessions need them; unavailable behavior suspends rather than silently selecting the latest. Every canonical/session-state mutation checks an unexpired epoch at the authority. An old owner cannot commit after replacement. External effects still need their own idempotency or reconciliation: a started tool with at-most-once recovery is reported interrupted when its effect is uncertain, and is not blindly repeated. This does not promise exactly-once external effects. Workspace and compute services are explicit retained Environment factories. A journal does not preserve ephemeral container files, secrets, background tasks, channels or schedules. Unsupported requirements must be refused before admission. The adapter’s default secret backend refuses plaintext storage. See the [adapter setup and tests](https://github.com/everruns/everruns/tree/main/crates/durable-object) for workerd, Cloudflare and replicated celld qualification. A test qualifies its specific revision, placement and failure domain; local fleet results do not establish production bucket or network-partition guarantees. --- # Evaluation and traces > Project portable session traces and run shared rule, judge and decision scorers in offline cases or bounded online listeners. Source: Portable trace projection uses the same pure event fold as the Server’s trace index: ```rust use everruns::trace::Trace; let trace = Trace::project(&session.events_after(0).await?)?; println!("{}", serde_json::to_string_pretty(&trace)?); ``` Supply a complete, sequence-ordered prefix from one session. Mixed sessions, missing stored sequences and duplicate/out-of-order events are rejected. The result contains turns and model/tool/approval/delegation steps with bounded previews. Full model prompts and tool bodies are not retained in the projection. Treat trace previews as application data when exporting them. The indexed Server API and trace viewer are host features. ## Offline evaluation Enable `evaluation`. `Evaluator` validates and runs the same deterministic scoring rules the Server uses. Supported rules include contains/not-contains, regex, tool use and argument matching, call/turn/token bounds, JSON schema and citation faithfulness. File checks use explicitly collected session workspace evidence. ```rust use everruns::evaluation::{Evaluator, Scorer}; let evaluator = Evaluator::new(vec![ Scorer::Contains { text: "42".into(), weight: 1.0 }, Scorer::TurnsWithin { max: 1, weight: 1.0 }, ])?; ``` `run(engine, agent, cases, concurrency)` creates a fresh real session per case, runs each input in order, captures evidence and returns results in case order. Concurrency is explicitly bounded to 1–32. Each case has a per-input deadline, and timed-out turns are cooperatively cancelled. Results retain session IDs, exact build registrations, evidence, ordered scores and failures for debugging. For a durable Engine, use `run_registered(engine, build, cases, concurrency)` with an application-retained `AgentBuild`; re-register it after restart. Use `.judge(Model)` for rubric/citation judges and `.decisions(Decisions)` for typed yes/no grading. Judge prompt construction and response parsing are shared with Server. Providers and credentials are explicitly selected by the application. Hosted model IDs require a custom `Grader`; they are never silently translated into another account’s model. Missing evidence/models and provider failures are errors. `Grader` also supports application-specific external evidence. The [complete evaluation example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_evaluation.rs) runs a real arithmetic agent, prints scores and projects its trace: ```bash cargo run -p everruns --features openai,evaluation --example service_evaluation ``` ## Online scoring Install `OnlineScoring::new(evaluator, sink)` with `Engine::builder().listener(...)`. Implement `ScoreSink` to retain/export grades. It uses the same evaluator as offline runs and executes on the Engine’s bounded observer queue, outside turn execution. The listener bounds retained turns and evidence size; overflow, unsuccessful turns and grading failures are reported to the sink. Monitor Engine observer drop statistics. Online scoring is observational and must not be used as a durable audit ledger; replay canonical events for complete historical grading. `Engine::shutdown(deadline)` flushes listeners within a caller-selected deadline. Server evaluation definitions, history/reporting storage, durable scoring jobs and dashboards remain host concerns. ## Compaction observation The Framework exposes the same pure operational assessment used by hosted compaction monitoring, without an extra feature or capability: ```rust use everruns::{SessionEvent, CompactionVerdict, observe_compaction}; fn inspect(event: &SessionEvent) { if let Some(observation) = observe_compaction(event) { if observation.verdict == CompactionVerdict::Failed { eprintln!("{}", observation.reason); } } } ``` Call it from an application-owned Engine listener and retain findings in your own sink. It does not make model calls or change the session. A pass reports execution metadata, not independent durability or semantic validation. Old events without the runtime’s history-evidence permission are ineligible for content assessment. Hosts own consent, tenant boundaries, bounded evidence reads, judge selection and spending admission. Organization opt-in, durable jobs, reports and review UI are Platform features; built-in and custom harnesses share their runtime instrumentation. --- # Event triggers > Route authenticated events through shared filters, changes-only checks, durable replay protection and recoverable Engine admission. Source: Enable `triggers` to use the Server’s filtering, change detection, template rendering and GitHub normalization without its management APIs. An application owns HTTP authentication, subscriptions and scheduling; `Dispatcher` owns the event-to-Engine pipeline. ```rust use everruns::triggers::{Dispatcher, SqliteStore, Trigger, Filter, Routing}; let dispatcher = Dispatcher::new(engine, SqliteStore::open("private-triggers.sqlite")?, Trigger { key: "deployment-watcher/v1".into(), build, message: "Deployment {{payload.version}} completed for {{payload.service}}".into(), filter: Filter::default(), routing: Routing::PerSubject, })?; ``` Register the exact `AgentBuild` with the Engine before dispatch. Change the trigger key when changing its configuration; accidental reuse is rejected. Routing can create one session per event, per subject, or one shared session. Missing subjects under `PerSubject` use an event-specific session. Filters combine dotted paths and accepted values. Numeric/boolean scalars match their string representations using Server semantics. `ChangesOnly` remembers the last value per subject: the first event establishes the baseline, repeats are filtered, and a change passes. Events missing the selected value do not erase it. The latest 500 subjects are retained. ## Recovery and replay protection Before calling the Engine, a transaction freezes the rendered input, session and input IDs in a durable outbox. Concurrent retries and recovery use `Session::send_with_id` to admit the same input once. A delivery reports `Dispatched` after admission, not after the agent finishes; use its session/input IDs to inspect or wait for the admission. Call `recover()` at application startup and from your scheduling loop. It retries unadmitted deliveries. Engine hosting and the outbox must both survive process loss. At most 200 unadmitted deliveries are accepted per trigger; recent payload history is bounded. Permanent receipt tombstones retain replay protection even after delivered/filtered payload history is pruned. Retire the database explicitly when that retention is no longer required. `StateStore` lets applications replace SQLite. Its compare-and-swap must atomically update state and insert any new receipt, and receipt tombstones must outlive history pruning. Store state outside the agent workspace, with application-level access restrictions. ## Sources `Event` accepts already authenticated application events. Never construct it directly from a public webhook without authentication. `github(secret, GitHubRequest, own_bot)` verifies `X-Hub-Signature-256` over the exact body before parsing, suppresses the selected bot’s own events, and shares GitHub/CI context normalization with Server. Restrict subscribed event types and repositories in the ingress/filter configuration. GitHub signatures cover the body, without a timestamp or delivery header. The adapter derives replay identity from the verified body, so changing the unsigned delivery header cannot bypass deduplication. Identical signed bodies are treated as the same event. The header remains informational `github.delivery_id` metadata; event/repository routing must also validate the expected payload shape. `mcp(secret, McpRequest)` uses the shared Standard Webhooks verifier, enforces the timestamp window and expected live subscription, then builds MCP template context. The application owns subscribe/renew/unsubscribe calls and passes only the active subscription identity. Oversized bodies and invalid signatures are rejected before dispatch. The [complete trigger example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_triggers.rs) uses a real model and persistent local storage. Run it twice with the same event ID to see durable duplicate suppression: ```bash cargo run -p everruns --features openai,local,triggers --example service_triggers -- deploy-123 ``` --- # Events and Cancellation > Subscribe to live Framework session events, replay durable events, cancel a turn, and work with the canonical event protocol. Source: Subscribe before sending a message to observe its live event projection: ```rust use everruns::{Agent, Engine, OpenAI}; let agent = Agent::builder() .instructions("Be concise.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let session = Engine::new().create(agent); let mut events = session.events(); let pending = session.send("Start.").await?; while let Some(event) = events.recv().await? { println!("{}", event.event_type()); if event.kind.is_terminal() { break; } } let turn = pending.wait().await?; assert!(turn.success); ``` Known events have typed `SessionEventKind` values. Unknown canonical event types are preserved as `Other` with their payload, so the projection does not silently drop information. The feed is live and non-blocking: `send` starts execution in the session’s background actor, a slow or dropped consumer does not stop the turn, and the feed is not a durable replay API. Use [lifecycle hooks](https://everruns.com/docs/framework/lifecycle-hooks/) instead when application work must be awaited at an execution boundary or its failure must affect the run. See [Canonical events](#canonical-events) below for canonical envelopes, live-delta memory bounds, explicit lag handling, ordering, and the durability boundary. Use [Session history and resume](https://everruns.com/docs/framework/sessions/#history-and-resume) to rebuild a bounded persisted transcript after live lag or a process restart. ## Replay durable events Durable events carry an increasing per-session `sequence()` starting at 1. `session.events_after(n)` reads a stable snapshot of durable events after sequence `n`. On a branch-aware authority such as `SqliteHosting`, it follows the active conversation branch, so sequences can have gaps. A backlog above 100,000 events returns `HistoryError::HistoryTooLarge` rather than an unbounded result. Replay works after `Engine::resume` in a new process when the hosting profile preserves the session and its events. `session.events_from(n)` returns an `EventStream` that replays that backlog and then continues live, deduplicating events already replayed. Live-stream lag is reported explicitly; reconnect to recover from the last sequence seen. Ephemeral deltas have no sequence and pass through live only: ```rust let last_seen = 2; let mut stream = session.events_from(last_seen).await?; while let Ok(Some(event)) = stream.try_recv() { println!("{:?} {}", event.sequence(), event.event_type()); } ``` ## Cancel a turn A message receipt exposes the specific accepting turn, so live applications can cancel without racing against whichever turn is active later: ```rust let pending = session.send("Start.").await?; pending.turn().cancel().await?; let cancelled = pending.wait().await?; assert!(!cancelled.success); ``` `run_with` retains cancellation-token convenience for request/response calls: ```rust use everruns::{CancellationToken, RunOptions}; let cancel = CancellationToken::new(); let options = RunOptions::new().cancel_token(cancel.clone()); cancel.cancel(); let turn = session.run_with("Stop before starting.", options).await?; assert!(!turn.success); ``` Cancellation is cooperative. Cancelling drops the in-flight turn future and tears down tool work through the same runtime path; it does not kill the host process or provide an independent transaction boundary. ## Cancel one tool call To stop one running tool call and let the turn go on, for example when a user stops a long shell command, cancel it by its tool call id: ```rust if session.tool_call_running("call_42") { session.cancel_tool_call("call_42"); } ``` The call’s future is dropped, its tool context cancellation fires, and it completes with status `cancelled` and an error the model reads, so the agent can react and continue. Other calls in the same batch finish as usual. Calls register in a process-wide registry; a host that wants its own puts a `ToolCallCancels` (`everruns_core::engine::tool_cancel`) in the tool context extensions. ## Observability To send every session’s events to OpenTelemetry or Braintrust, or to your own listener, register it on the Engine; see [Framework observability](https://everruns.com/docs/framework/observability/). ## Canonical events `Session::events()` installs an in-process subscriber without exposing runtime event buses or core event types. Subscribe before `Session::run()` so the stream sees the turn from its first event. ```rust use everruns::prelude::*; let agent = Agent::builder() .instructions("Answer concisely.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let engine = Engine::new(); let mut session = engine.create(agent); let mut events = session.events(); let observer = tokio::spawn(async move { let mut canonical_events = Vec::new(); while let Some(event) = events.recv().await? { // Recording for replay, so take the canonical envelope: it withholds // nothing. `as_json()` carries the reviewed projection instead. canonical_events.push(event.canonical_json().clone()); // Typed rendering for common terminal/service UI concerns. match event.kind { SessionEventKind::TextDelta { delta } => print!("{delta}"), SessionEventKind::ToolStarted { tool_name, .. } => { eprintln!("starting {tool_name}"); } SessionEventKind::ToolCompleted { tool_name, success, .. } => eprintln!("{tool_name}: {success}"), SessionEventKind::TurnFailed { error } => eprintln!("failed: {error}"), SessionEventKind::TurnCancelled => eprintln!("cancelled"), _ => {} } } Ok::<_, EventStreamError>(canonical_events) }); let turn = session.run("hello").await?; drop(session); // closes the subscriber once buffered events are drained let recorded = observer.await??; assert!(!recorded.is_empty()); ``` ### One protocol, two views `SessionEventKind` is a convenience projection for application renderers. It promotes assistant output lifecycle and deltas, model reasoning/generation, tool lifecycle/progress/output, and turn terminal states. It is non-exhaustive, so match it with a fallback arm. `SessionEvent` exposes two surfaces, and which one you want depends on whether you are rendering or recording. `SessionEvent::as_json()` and `SessionEvent::data()` return the **reviewed** form: the event envelope — id, type, timestamp, optional persisted sequence, correlation context and tags, with reviewed public provenance in metadata — and a `data` payload holding only the fields promoted onto `SessionEventKind`. Nothing else reaches it, so a field added inside the runtime cannot become part of the Framework’s public surface, or travel to wherever your application forwards these envelopes, without being promoted first. This is the form to log, forward, or expose to clients. `SessionEvent::canonical_json()` returns the **canonical** envelope with the complete payload: prompts, tool arguments, tool results, structured assistant messages, and the payloads of event types this version does not recognize. Nothing observable is lost — this is the form for recording, auditing, and replay. It follows the runtime’s internal shape rather than the Framework’s reviewed surface, so treat what you read from it as unstable, and do not forward it anywhere the conversation itself should not go. Live `output.message.delta` envelopes omit the redundant `data.accumulated` prefix on both surfaces; retaining every growing prefix in a slow subscriber’s buffer would use quadratic memory. Concatenate the typed `TextDelta::delta` values to reconstruct streamed text, or use the subsequent `output.message.completed` event for the complete message. When a provider stream breaks after some text, the runtime retries it under the same message id and sends `OutputStarted` for that id again first: drop the text collected for the message and start over. Model-generation accounting — model, provider, token counts, cost, and duration — is promoted onto `SessionEventKind::ModelGeneration`, so tracking spend never requires the unstable surface. This bridge does not define a second wire schema. The canonical event contract, compatibility rules, and lifecycle semantics remain documented in [Events](https://everruns.com/docs/explanation/events/). ### Durability, observation, and derived history These roles are deliberately separate: * `EventLog` is the host’s sole durable conversation write authority. It stores complete canonical event envelopes and provides bounded cursor replay. * `EventSink` is the host’s post-commit, nonblocking live-delivery boundary. `Session::events()` exposes that observation path as an ergonomic `EventStream` subscriber. Neither sink nor subscriber is durable or authoritative. * `EventHistory` is one read-only message projection rebuilt from `EventLog` replay. It is an index/view, never a second writable message store. Framework applications read that bounded projection through [`Session::history()`](https://everruns.com/docs/framework/sessions/#history-and-resume). It pages messages from a stable event-log snapshot; it does not maintain or write an independent transcript. Rebuild a transcript in persisted sequence order from `input.message`, `output.message.completed`, and relevant `tool.completed` events. An `output.message.replaced` event alone creates no history message; the subsequent completed message contains the safe replacement. If a crash leaves a replacement without completion, replay correctly omits that incomplete output. The Framework stream exposes canonical payloads, subject to the live-delta exception above, and introduces no independent writable message history. Canonical recordings can contain user messages, agent instructions, model inputs, tool arguments, and tool results. Treat them as application data with the same access controls and retention policy as the session itself; do not log them indiscriminately. Model and tool text is untrusted: terminal renderers should strip or escape control sequences, and web renderers should escape it as content rather than interpreting it as markup or commands. Provider credentials are not part of the event protocol. ### Implementing a custom event log An advanced host can store canonical events itself. `everruns-core` (`host` feature) exposes `EventReader` and `EventLog` as a public SPI: an external crate implements both against its own storage and supplies the result to composition through `HostBackends::with_event_log`. No in-crate access is required, cursors and pages are built with `EventCursor::continuation`, `EventCursor::after`, and `EventPage::new`, which validate the shared invariants. Three request shapes are distinguished by `EventReadRequest::cursor()`: * no cursor is an initial read that captures the session’s current high-watermark and reports it as `EventPage::snapshot_high_watermark()`; * a cursor whose `snapshot_high_watermark()` is `Some` is a continuation pinned to that snapshot, so appends committed later stay invisible and paging neither skips nor duplicates; * a cursor whose `snapshot_high_watermark()` is `None`, built by `EventCursor::after`, is a poll that captures a fresh snapshot and therefore does observe those later appends. ```rust use async_trait::async_trait; use everruns_core::events::{Event, EventRequest}; use everruns_contracts::typed_id::EventId; use everruns_core::host::{ EventCursor, EventDurability, EventLog, EventLogError, EventPage, EventReadRequest, EventReader, }; #[async_trait] impl EventReader for MyEventLog { async fn read_page(&self, request: EventReadRequest) -> Result { let session_id = request.session_id(); let current_high = self.high_watermark(session_id); let (after, snapshot) = match request.cursor() { None => (0, current_high), Some(cursor) => { if cursor.session_id() != session_id { return Err(EventLogError::CrossSessionCursor { detail: "cursor belongs to another session".into(), }); } match cursor.snapshot_high_watermark() { Some(snapshot) if snapshot > current_high => { return Err(EventLogError::ExpiredCursor { detail: "cursor snapshot is not available".into(), }); } // Pinned continuation, then the polling form. Some(snapshot) => (cursor.after_sequence(), snapshot), None => (cursor.after_sequence(), current_high), } } }; let limit = request.limit().get(); let mut events = self.events_in(session_id, after, snapshot, limit + 1); let has_more = events.len() > limit; if has_more { events.pop(); } let next_cursor = has_more .then(|| { let last = events.last().and_then(|event: &Event| event.sequence).unwrap_or(after); EventCursor::continuation(session_id, last, snapshot) }) .transpose()?; EventPage::new(events, next_cursor, snapshot) } } #[async_trait] impl EventLog for MyEventLog { async fn append(&self, request: EventRequest) -> Result { if request.is_ephemeral() { return Err(EventLogError::InvalidAppend { detail: "ephemeral events are sink-only".into(), }); } // The log owns identity: assign the event id and the next per-session // sequence, persist, and return the finalized canonical envelope. let sequence = self.next_sequence(request.session_id); let event = request.into_event(EventId::new(), sequence); self.persist(&event)?; Ok(event) } fn durability(&self) -> EventDurability { EventDurability::CrashDurable } } ``` The contract an implementation must uphold: * an accepted append owns id and sequence assignment and returns the finalized canonical `Event`, visible to the next read of that session; * durable sequences are unique and strictly increasing per session, and need not be contiguous, gaps are expected when a reader projects an append-only physical log into a filtered logical event sequence; * a continuation stays pinned to the first page’s high-watermark and cannot observe concurrent appends; a poll cursor can; * cursor/session mismatches and inconsistent positions return the typed `EventLogError` variants above rather than panicking; * the log is append-only. There is no truncate, rewind, or mutation contract, and `EventHistory` remains a read-only projection rather than a second writable message store. The [external event-log example](https://github.com/everruns/everruns/tree/main/crates/everruns/tests/fixtures/external-consumer/event-log) provides a complete implementation in a standalone application. ### Ordering and bounded delivery A subscriber receives events in channel arrival order and each session has its own stream. The canonical `sequence` field is a replay position, not a live delivery counter: durable events carry `Some(sequence)` and live-only ephemeral events such as streaming deltas carry no sequence. Persisted sequences increase monotonically per session and may have gaps. Ephemeral events do not consume replay positions. The live stream has a bounded buffer and never applies backpressure to the agent turn. A dropped or slow subscriber cannot stall model or tool execution. If a subscriber falls behind, `recv()` and `try_recv()` return `EventStreamError::Lagged { missed }`; loss is never hidden. The next receive can continue from the oldest retained event, but the renderer must treat its live projection as incomplete. Streaming deltas are provisional and sink-only. Completed assistant/tool events are authoritative, and an output-replacement event means accumulated text for that message must be discarded. Durable events reach the live sink only after their log append commits; ephemeral events go directly to the sink and never enter history. After live lag, a Framework application can rebuild its persisted transcript with bounded [`Session::history()` pages](https://everruns.com/docs/framework/sessions/#history-and-resume). That projection excludes ephemeral deltas by design. Applications that need raw durable envelopes rather than derived messages can provide and read an `EventLog` through the advanced `everruns-core` (`host` feature) SPI; neither recovery path relies on the in-process subscriber. ### Cancellation and failure Pass a `CancellationToken` through `RunOptions` to stop a turn. Cancellation produces both a `Turn` with `TurnStopReason::Cancelled` and a correlated `turn.cancelled` event carrying the same `turn_id`. Runtime failures remain available through `Session::run()`’s outcome/error semantics and the event stream. Subscribe before running and continue draining the stream after the run resolves to retain the terminal failure event and its full structured payload. The complete runnable example is [`canonical_events.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/canonical_events.rs): ```bash cargo run -p everruns --example canonical_events ``` --- # Runnable Examples > Complete Framework programs, maintained and compiled with the everruns crate. Source: The [`crates/everruns/examples` catalog](https://github.com/everruns/everruns/tree/main/crates/everruns/examples) contains the maintained public examples. Each imports the `everruns` facade. ## Complete agents The root-level [`examples`](https://github.com/everruns/everruns/tree/main/examples) catalog contains eight Framework walkthroughs. Each folder includes the program, instructions, fixtures where applicable, and recording scripts. Run them from a repository checkout: their dependencies point to the workspace crates. `cargo run` uses a real provider and can incur charges. CI tests offline tool behavior and recording logic; it does not establish the quality of a live model’s answer. | Example | Provider and model | What it does | | -------------------------------------------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------- | | [Support Agent](https://everruns.com/docs/framework/examples/support-agent/) | OpenAI `gpt-6.2-sol` | Chooses between MFA recovery, lockout, and browser troubleshooting from facts and policy. | | [Everruns Support Agent](https://everruns.com/docs/framework/examples/everruns-support-agent/) | Anthropic `claude-opus-5-5` | Searches and reads citable official documentation snapshots. | | [Coding Review Agent](https://everruns.com/docs/framework/examples/coding-review-agent/) | Anthropic `claude-sonnet-5` | Reads a refund contract and executes a fixed regression before reporting a defect. | | [Research Agent](https://everruns.com/docs/framework/examples/research-agent/) | OpenRouter `z-ai/glm-5.2` | Searches and fetches primary sources before writing a cited brief. | | [Incident Commander Agent](https://everruns.com/docs/framework/examples/incident-commander-agent/) | Meta Model API `muse-spark-1.3` | Investigates fixture telemetry and persists an evidence-backed incident update. | Start with Support for typed tools, Research for reusable capabilities, or Code Review for restricted execution. Each walkthrough shows the agent builder and session loop, explains expected behavior, and documents what remains a fixture. These are in-memory sessions. For durability itself, use the session-history and workspace examples below. Importable hosted Platform definitions live separately as TOML folders in [`examples/agents`](https://github.com/everruns/everruns/tree/main/examples/agents). See [file-based agents](https://everruns.com/docs/how-to/define-agents-as-files/) for loading, importing and serving them. ## Execution runtimes | Example | Provider and model | What it does | | -------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | [Bashkit Repo Agent](https://everruns.com/docs/framework/examples/bashkit-repo-agent/) | OpenAI `gpt-6.2-sol` | Cuts a release in a real repository with the sandboxed Bashkit shell as its only tool, then verifies the result on disk. | | [Code Mode MCP Agent](https://everruns.com/docs/framework/code-mode/) | OpenAI `gpt-6.2-sol` (or through OpenRouter) | Answers a question over a 1,000-tool MCP server by calling its tools from Bashkit scripts, with none of their schemas in the prompt. | | [Host Shell Agent](https://everruns.com/docs/framework/examples/host-shell-agent/) | OpenAI `gpt-6.2-sol` | Fixes a failing Rust test suite by compiling and running it through real processes inside a kernel-enforced boundary. | | [Foreman](https://everruns.com/docs/framework/examples/foreman-agent/) | TypeSafe `jev-latest` over an Everruns session, Codex, or yolop | Supervises a live coding session with nine decision questions per reading, and stops, verifies, or finishes it from a deterministic policy. | ## Terminal hosts | Example | Provider and model | What it does | | -------------------------------------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`attached-cli`](https://github.com/everruns/everruns/tree/main/examples/attached-cli) | OpenAI `gpt-6.2-sol` | Serves a running session to a second terminal, which lists, watches, sends to and cancels it through [attached control](https://everruns.com/docs/framework/attach-control/). | ## Talking to other companies’ agents | Example | Provider and model | What it does | | ------------------------------------------------------------------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`poppy-personal-agent`](https://github.com/everruns/everruns/tree/main/examples/poppy-personal-agent) | OpenAI `gpt-6.2-sol` | Acts as a person’s own agent over the [Personal Agent Protocol](https://everruns.com/docs/features/poppy/): finds a company by its domain, asks its agent about an order, and reports the answer. | ## Serving agents These run agents as HTTP services with [Serve](https://everruns.com/docs/framework/serve/). | Example | What it does | Command | | ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | | [`serve/hello`](https://github.com/everruns/everruns/tree/main/examples/serve/hello) | The smallest serve app: one agent, one tool | `cargo run -p serve-example-hello` | | [`serve/ag-ui`](https://github.com/everruns/everruns/tree/main/examples/serve/ag-ui) | An agent streamed to CopilotKit or any AG-UI client | `cargo run -p serve-example-ag-ui` | | [`serve/a2a`](https://github.com/everruns/everruns/tree/main/examples/serve/a2a) | Two agents over [A2A](https://everruns.com/docs/framework/a2a/): a researcher served by serve, and a writer that delegates to it. Offline by default | `cargo run -p serve-example-a2a --bin researcher`, then `--bin writer -- "tide pools"` | | [`serve/voice`](https://github.com/everruns/everruns/tree/main/examples/serve/voice) | A hotel front desk people [call from the browser](https://everruns.com/docs/framework/voice/), with a greeting and a speaking style | `OPENAI_API_KEY=… cargo run -p serve-example-voice` | ## Durable hosts [Durable multiplayer chat on Cloudflare](https://everruns.com/docs/framework/examples/durable-chat/) runs the portable kernel inside a SQLite-backed Durable Object, with shared rooms, forks, scheduled follow-ups, and a recoverable history-search child. This is an experimental advanced host example. ## Core crate catalog | Example | Demonstrates | Command | | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | [`capability_configuration.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/capability_configuration.rs) | Typed Compaction and ToolSearch, a code-defined Definition, and a dynamic third-party reference through one entrypoint | `cargo run -p everruns --example capability_configuration` | | [`model_catalog.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/model_catalog.rs) | Listing a provider’s models, reading their metadata, and running the selection | `cargo run -p everruns --example model_catalog` | | [`ask_user.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/ask_user.rs) | Structured questions answered by a host responder, and the unattended defaults path, fully offline | `cargo run -p everruns --example ask_user` | | [`ag_ui_axum.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/ag_ui_axum.rs) | An AG-UI 1.0 run served over axum | `cargo run -p everruns --features ag-ui-axum --example ag_ui_axum` | | [`image_generation.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/image_generation.rs) | An agent on GPT-6.2 Sol that draws an animated GIF with GPT Image 2.6 Flare and saves it next to you | `OPENAI_API_KEY=… cargo run -p everruns --features openai,images,local --example image_generation` | | [`workspace_policy.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/workspace_policy.rs) | Safe workspace scopes and trusted starter files, fully offline | `cargo run -p everruns --example workspace_policy` | | [`direct_llm.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_llm.rs) | One-shot, builder, and streamed model calls with no agent, fully offline | `cargo run -p everruns --example direct_llm` | | [`direct_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/direct_decisions.rs) | Typed questions and calibrated answers with no agent, fully offline | `cargo run -p everruns --example direct_decisions` | | [`agent_decisions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/agent_decisions.rs) | An agent that classifies with its own questions, offline by default | `cargo run -p everruns --example agent_decisions` | | [`live_session.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/live_session.rs) | Non-blocking send, automatic steering, and optional waiting, fully offline | `cargo run -p everruns --example live_session` | | [`hello.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/hello.rs) | Small live-provider agent | `cargo run -p everruns --features openai --example hello` | | [`production_agent.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/production_agent.rs) | Tools, files, and production-style setup | `cargo run -p everruns --features openai --example production_agent` | | [`github_monitor.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/github_monitor.rs) | Typed tools and an offline simulation mode | `cargo run -p everruns --features openai --example github_monitor -- --simulate` | | [`session_work.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/session_work.rs) | Offline session work, leased delivery, and completion wakes | `cargo run -p everruns --example session_work` | | [`session_history.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/session_history.rs) | Offline durable resume and bounded history pages | `cargo run -p everruns --features local --example session_history` | | [`engine_sessions.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/engine_sessions.rs) | Concrete Engine ownership, isolated sessions, and engine-scoped resume | `cargo run -p everruns --example engine_sessions` | | [`workspace_heads.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/workspace_heads.rs) | Isolated Git workspace heads, Environment binding, and durable reopening | `cargo run -p everruns --features local --example workspace_heads -- /path/to/repo /path/to/state` | | [`canonical_events.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/canonical_events.rs) | Offline bounded recording and typed rendering of live events | `cargo run -p everruns --example canonical_events` | | [`explicit_communication.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/explicit_communication.rs) | Offline agent with explicit communication: sent messages apart from working notes | `cargo run -p everruns --example explicit_communication` | | [`framework_observability.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/framework_observability.rs) | Engine-wide OpenTelemetry and optional Braintrust export with explicit shutdown flushing | `cargo run -p everruns --features otel,braintrust --example framework_observability` | | [`subagents.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/subagents.rs) | Public facade composition for delegated work | `cargo run -p everruns --features openai --example subagents` | | [`observe_and_cancel.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/observe_and_cancel.rs) | Live events and cancellation | `cargo run -p everruns --features openai --example observe_and_cancel` | | [`advanced_capability.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/advanced_capability.rs) | Code-defined capability through the unified `capability(...)` entrypoint | `cargo run -p everruns --features openai --example advanced_capability` | | [`lifecycle_hooks.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/lifecycle_hooks.rs) | Awaited agent, turn, tool, and completion handlers | `cargo run -p everruns --features openai --example lifecycle_hooks` | Live-provider modes use `gpt-6.2-sol` and require `OPENAI_API_KEY`. `capability_configuration`, `canonical_events`, `direct_llm`, `engine_sessions`, `explicit_communication`, `live_session`, `session_work`, `workspace_heads`, `workspace_policy`, and `session_history` are fully offline; the GitHub monitor also offers a simulated GitHub flow: ```bash cargo run -p everruns --features openai --example github_monitor -- --simulate ``` For copyable command details and behavior notes, use the [example catalog](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/README.md) next to the source. Examples that demonstrate low-level host internals remain advanced-host examples, not alternative Framework entrypoints. ## Provider-native batches The [batch guide](https://everruns.com/docs/framework/batch-model-requests/) shows offline work through Anthropic, OpenAI, Gemini, Mistral, Groq and Fireworks. Set `BATCH_PROVIDER` to the provider name, enable its Cargo feature and configure its API key. Both examples can incur charges. | Example | Behavior | Run | | ---------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | [`batch_classification.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_classification.rs) | Persist submission intent and ticket; resume classification, collect or cancel where supported | `cargo run -p everruns --features anthropic --example batch_classification -- submit /tmp/reviews-001` | | [`batch_eval.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/batch_eval.rs) | Submit typed message lists, poll and score three eval cases | `cargo run -p everruns --features anthropic --example batch_eval` | ## Custom platform workers The [Custom Data Agent Worker](https://everruns.com/docs/framework/examples/custom-data-agent/) packages a Framework agent with private SQLite tools behind the restricted Platform worker protocol. Its complete smoke runner uses a real model, the ordinary session APIs and SSE, and checks replica replacement and credential rotation. This advanced host example needs a self-hosted CP; its shared history lives there. The batch examples also accept `BATCH_PROVIDER=mistral`, `groq` or `fireworks` with the matching Cargo feature. Fireworks requires account and artifact-origin configuration; see [Batch Model Requests](https://everruns.com/docs/framework/batch-model-requests/). ## Native GitHub PR review [`github_review.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/github_review.rs) reads a real PR with the native GitHub capability and prints an evidence-based review. It requires `OPENAI_API_KEY` and `GITHUB_TOKEN` and does not write to GitHub. See [GitHub tools](https://everruns.com/docs/framework/github/). ```bash cargo run -p everruns --features github,openai --example github_review -- OWNER/REPO NUMBER ``` ## Application services | Example | Required features | What it runs | | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------------------------------------- | | [OAuth MCP](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/mcp_oauth.rs) | `openai,mcp-auth` | Browser login, callback validation, authenticated MCP tool use | | [Slack actions](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/slack_actions.rs) | `openai,slack-actions` | React and upload into one authorized thread | | [Data services](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_data.rs) | `openai,sql-database,knowledge` | Retrieve policy, retain a SQL decision and write named memory | | [Native services](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_credentials.rs) | `openai,cursor,browserless,e2b` | Use one selected vendor with explicit credentials and resource cleanup | | [Event triggers](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_triggers.rs) | `openai,local,triggers` | Recoverable event dispatch and duplicate suppression across restarts | | [Evaluation and traces](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_evaluation.rs) | `openai,evaluation` | Run real cases, print scores and project canonical trace events | Every example uses a real model and reports failed turns. See [Application services](https://everruns.com/docs/framework/service-integrations/) for configuration and host responsibilities. --- # Bashkit Repo Agent > Modify and verify a disposable repository through a sandboxed shell. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/bashkit-repo-agent). Cut a release in a disposable repository through the sandboxed [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/), then verify every claimed change directly from the host. This is a real `gpt-6.2-sol` agent, not a scripted turn. ![Bashkit Repo Agent terminal demo](https://raw.githubusercontent.com/everruns/everruns/main/examples/bashkit-repo-agent/demo/demo.gif) ## What you learn How to mount a narrow read-write workspace, give an agent one execution capability, accept an interactive task, and treat independent host assertions as the success condition. ## Scenario and expected outcome The bundled fixture is a two-crate Cargo workspace with three changelog fragments. The agent must cut release `0.2.0`: update both package versions and the path-dependency pin, create today’s changelog section, preserve the `0.1.0` history, and remove the folded fragments. After the turn, Rust code reopens the real working copy and fails the process if any invariant is false. A confident model answer cannot make a broken release pass. ## Run it Clone the repository and run from its root: ```bash export OPENAI_API_KEY="your-key" cargo run -p everruns-bashkit-repo-agent ``` Type the task interactively: ```bash cargo run -p everruns-bashkit-repo-agent -- --interactive ``` The default workspace is temporary and removed on exit. Pass a scratch directory to inspect the result afterwards: ```bash cargo run -p everruns-bashkit-repo-agent -- /tmp/release-run cargo run -p everruns-bashkit-repo-agent -- --interactive /tmp/release-run ``` The configured model is `gpt-6.2-sol`. Provider access and funded credits are required; missing credentials, unsuccessful turns, and failed disk assertions exit nonzero. Never pass a repository you care about: the example materializes its fixture into the target and the agent may change anything inside the mount. ## Build the agent Read-write access is explicit—the default workspace policy is read-only. ```rust pub fn build(provider: impl Into, workspace: &Path) -> Result { Agent::builder() .name("bashkit-repo-agent") .instructions(include_str!("resources/instructions.md")) .provider(provider) .model(MODEL) .max_iterations(12) .workspace(workspace) .workspace_policy(WorkspacePolicy::read_write()) .capability(BashkitShell::new()) .build() } ``` ## Run and verify The shared observer displays a bounded shell timeline and waits for a successful turn. The host then verifies the mounted files itself. ```rust // OPENAI_API_KEY, declared by the OpenAI driver itself. let agent = agent::build(everruns::OpenAI::from_env()?, &workspace)?; let engine = Engine::new(); let session = engine.create(agent); demo::run(&session, &request).await?; verify_release(&workspace, &release_date)?; ``` Use `session.send_and_wait(&request).await?` when a live tool timeline is not needed. ## How Bashkit behaves Bashkit interprets shell scripts in-process against `/workspace`. The model gets no host filesystem outside that mount, network, Git credentials, or subprocess execution. Commands, loops, output, and script size are bounded. Repository text is treated as untrusted data rather than agent instructions. ## Validate it ```bash cargo test -p everruns-bashkit-repo-agent bash examples/bashkit-repo-agent/demo/record.sh --check ``` These checks validate argument handling, fixture state, host assertions, and the live-demo contract without provider credentials. They do not grade model quality. ## Demo and recording The screencast types the release task into the interactive binary, displays real `bashkit_shell` calls, and ends with host-side checks over the mutated files. It does not replay prepared output. Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/bashkit-repo-agent/demo/transcript.txt) at your own pace. With VHS, ffmpeg, a VHS-compatible browser, and funded OpenAI credentials: ```bash bash examples/bashkit-repo-agent/demo/record.sh ``` The script uses `OPENAI_API_KEY` when exported, otherwise Doppler project `everruns-dev`, config `dev`. It updates the GIF and transcript only after the model turn and host verification succeed. ## Adapt it safely Replace the fixture, request acceptance criteria, and verifier together. Keep the mount disposable and narrow, start read-only unless mutation is required, and verify consequential claims outside the model/tool boundary. ## Boundaries This demonstrates one sandboxed repository mutation, not a general coding agent. It cannot fetch dependencies, run native programs, commit, or push. The Framework session is in-memory. --- # Coding Review Agent > Tool-based code inspection, a tightly scoped execution tool, and distinguishing test failure from tool failure. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/coding-review-agent). Review a refund implementation against a written contract, then execute a bundled regression test before reporting a defect. A proposed test is not treated as proof: the agent gets actual compiler and test output. ## What you learn Tool-based code inspection, a tightly scoped execution tool, and distinguishing test failure from tool failure. ## Scenario and expected outcome Two refunds of 1,000 cents are requested against a 1,000-cent payment. Each individual call caps its own amount, but the implementation retains no cumulative refunded balance. The regression fails with `left: 2000` and `right: 1000`. The review should connect that observed failure to the missing cumulative-refund state, describe the over-refund risk, and propose a minimal fix. It must not claim the code was changed or fixed. ## Run it Install Rust/Cargo (including rustc), clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient. ```bash git clone https://github.com/everruns/everruns.git cd everruns export ANTHROPIC_API_KEY="your-key" cargo run -p everruns-coding-review-agent ``` The configured model is `claude-sonnet-5`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question: ```bash cargo run -p everruns-coding-review-agent -- "Read the contract and test. Reproduce the cumulative refund bug and explain the smallest safe fix." ``` ## Build the agent This builder combines the agent instructions, tools, and model. Tools/capabilities supply evidence and actions; the model chooses how to use them. ```rust let agent = Agent::builder() .name("coding-review-agent") .instructions(include_str!("instructions.md")) .provider(everruns::Anthropic::from_env()?) .model(MODEL) .max_iterations(12) .tool(tools::inspect_change()) .tool(tools::run_regression()) .build()?; ``` ## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline. ```rust let engine = Engine::new(); let session = engine.create(agent); println!("MODEL: {MODEL}"); demo::run(&session, question).await?; ``` This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `inspect_change` can read only `sample_payment.rs`, `contract.md`, and `regression.rs`. `run_regression` invokes `rustc --test` on that fixed trusted fixture, runs the temporary binary, and returns its exit code and assertion output. Compilation and execution have timeouts; temporary files are removed automatically. ## Validate the behavior ```bash cargo test -p everruns-coding-review-agent python3 examples/coding-review-agent/src/render_demo.py --check ``` The offline test compiles and runs the bundled regression and asserts the observed failure. `cargo test -p everruns-coding-review-agent` passes because it verifies reproduction of the intentionally buggy fixture. A failing subprocess is expected evidence, not a failing example test. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording ![Coding Review Agent recorded run](https://raw.githubusercontent.com/everruns/everruns/main/examples/coding-review-agent/src/demo.gif) Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/coding-review-agent/src/demo.txt) for the full recorded output. The GIF shows excerpts from the run. With credentials exported and Python 3, VHS, ffmpeg, and a VHS-compatible browser installed: ```bash cd examples/coding-review-agent bash src/record.sh ``` The script captures a successful run, generates correctly wrapped pages and page durations, and renders `src/demo.gif`. It preserves the previous transcript when the provider run fails. To replay an existing transcript without another model call, run `(cd src && python3 render_demo.py && vhs demo.tape)`. `src/demo.txt` retains the displayed output; `.demo-pages/` is generated and ignored. Inspect results before sharing: public/demo data is safe here, but adapting tools may expose private data. ## Adapt it Replace the fixture with a trusted review checkout and a restricted test selection. Use a sandbox before accepting arbitrary repositories, generated code, or model-selected commands. Add a second verified run after applying a fix in a separate approved workflow. ## Boundaries This is one deliberately buggy, trusted fixture—not a general-purpose coding agent. The execution tool takes no shell command or user-supplied path, and cannot edit code. Rust including `rustc` must be installed. --- # Custom Data Agent Worker > Package a Framework agent with private database tools and connect it to an Everruns control plane while keeping the ordinary chat and tracing surfaces. Source: This example turns an ordinary Framework agent into an experimental customer-hosted worker. Its `sales_summary` function reads its own SQLite database. An OpenAI `gpt-6.2-sol` model chooses the tool arguments and composes the answer. Users talk to the Everruns control plane, not to the worker directly. Follow [Run a custom agent worker](https://everruns.com/docs/how-to/custom-agent-workers/) for complete provisioning, TLS, credential rotation and smoke-test instructions. The [source and README](https://github.com/everruns/everruns/tree/main/examples/custom-data-agent) contain the executable and container packaging. ## Agent code and invocation The guide includes the [complete private database tool and Agent builder](https://everruns.com/docs/how-to/custom-agent-workers/#define-the-custom-agent-and-tool), [database seed code](https://everruns.com/docs/how-to/custom-agent-workers/#4-start-the-data-agent), and [ordinary session API calls](https://everruns.com/docs/how-to/custom-agent-workers/#invoke-it-through-the-cp) to create a conversation and send a message. The Agent is listed in **Agents**; operator provisioning binds that identity to the worker. Starting this executable does not register an Agent automatically. For Slack, use the same Agent’s **Channels and triggers** tab and Direct communication. Read the [channel support and limitations](https://everruns.com/docs/how-to/custom-agent-workers/#connect-the-same-agent-to-slack) before deployment: basic CP reply routing does not install native Slack tools, approvals or delegated-task behavior into this implementation, and live custom-worker Slack qualification is still required. ## Follow the code | Source | What it owns | | -------------------- | ------------------------------------------------------------------------------------------------------------------- | | `src/main.rs` | Worker lease loop, CP journal reads, Framework agent/tool composition, canonical event forwarding and cancellation. | | `src/transport.rs` | Application-owned OpenAI transport with a fixed HTTPS origin, redirect refusal and deployment proxy support. | | `smoke.py` | End-to-end verification against a real CP, built-in worker and model, using either a process or container. | | `Dockerfile` | Source-built release image. | | `Dockerfile.runtime` | Package an already-built Linux binary against a compatible runtime for smoke/CI use. | The worker opens the database with SQLite’s read-only flag. Model input supplies only the region, bound as a query parameter. It cannot choose a file path or send arbitrary SQL. For a real data service, apply your application’s own authorization before querying and return only fields that session readers may see. ## State and events Each lease reads the canonical CP event log and selects the next unconsumed user input. The example reconstructs conversation context in a fresh in-memory Engine. The CP remains the durable record; a second private Engine checkpoint is not claimed. Completed final answers can finish after a crash without rerunning the model. Other interrupted reads can replay under the same CP turn, with a bounded attempt count. The adapter subscribes before sending to the Framework. It forwards generations, tool calls, streaming output and terminal events, replaces local session/input correlation with CP correlation and reconstructs streamed text prefixes omitted by the facade. A lost observer event is an error, not a successful acknowledgment. The server scopes trace ids to the deployment/session/input and sends events through the normal EventService, so existing SSE, work and trace consumers apply. Every exit path cancels local execution. Renewal happens during execution and cancellation is checked against the input it targets. The worker acknowledges only the completed input; input arriving later remains queued. ## Run and verify After provisioning the CP binding and private database: ```bash cargo run -p everruns-custom-data-agent ``` Credentials and endpoints come from the deployment environment, as documented in the how-to. The [full smoke test](https://everruns.com/docs/how-to/custom-agent-workers/#5-run-the-full-smoke-test) checks three real-model turns, live SSE, process/container replacement, changed private data, credential revocation and rotation. Offline tests cover the SQL boundary, streaming projection and fixed-origin transport. Neither fixture-only tests nor a successful compilation establish an end-to-end deployment. This is a deployment example for advanced hosts. It uses the internal protocol and canonical event contracts in addition to the public application facade. Pin the server and worker to the same revision. It is not native durable Engine hosting, a general-purpose SQL agent, a sandbox, or a Cloud image-hosting service. --- # Durable multiplayer chat on Cloudflare > Build a shared chat room with ordinary Everruns Agents and Engine sessions inside Cloudflare Durable Objects, with forks, reminders and history search. Source: This walkthrough recreates [Earendil’s multiplayer chat tutorial](https://earendil.com/posts/pi-durable-on-cloudflare/) with ordinary Everruns Framework Agents. Conversations and search children run inside native Cloudflare Durable Objects. SQLite journals and durable alarms keep their work independent of connected browsers. The [chat example](https://github.com/everruns/everruns/tree/main/examples/cloudflare-durable-chat) uses the shared Engine session-hosting profile and OpenRouter driver. This is experimental hosting; read [Durable Object hosting](https://everruns.com/docs/framework/durable-objects/) for its guarantees. ## See the demo [Open the Cloudflare demo](https://everruns-durable-chat.everruns.workers.dev/), create a conversation, and copy the room link into a second browser. [Durable multiplayer chat with Everruns on Cloudflare](https://www.youtube-nocookie.com/embed/eFmwod7GTrY?rel=0) [Watch on YouTube](https://www.youtube.com/watch?v=eFmwod7GTrY). ![Alice and Bob share a launch-planning room. The assistant recalls their PostgreSQL decision, both participants are present, and committed messages have Fork here controls.](https://everruns.com/docs/_astro/durable-chat-shared.kOHoVw7m_27JLBP.webp) Alice and Bob see the same committed conversation. **Copy room link** invites another browser; **Fork here** starts a new conversation from a committed message. ## Run the chat Clone the whole repository because the example uses local workspace crates. Install Rust 1.96 and Node.js 24, then: ```sh git clone https://github.com/everruns/everruns.git cd everruns/examples/cloudflare-durable-chat rustup target add wasm32-unknown-unknown cargo install wasm-bindgen-cli --version 0.2.129 --locked npm ci npm run build cp .dev.vars.example .dev.vars ``` Set `OPENROUTER_API_KEY` in `.dev.vars`. Start the local Worker: ```sh npm run dev ``` The default model is [Muse Spark 1.3 Contributor](https://openrouter.ai/meta/muse-spark-1.3-contributor) through OpenRouter. Provider charges apply; Contributor uses submitted data for training. `MODEL` and `OPENROUTER_BASE_URL` are deployment settings. Open the local URL, choose a name and create a conversation. Copy the room link into another browser. Both participants see committed messages. Record a decision in one conversation and ask about it in another to exercise the search child. Ask for a reminder in 30 seconds, close the browsers, and return after the deadline. ## Framework execution, application services ![Shared browsers use public room routes to a ChatRoom Durable Object. Stable admissions reach a ChatSession Durable Object for each conversation or search child, running Everruns Engine with its own journal and alarm. The shared driver calls OpenRouter through fetch.](https://everruns.com/docs/_astro/durable-chat.CnqZUOgV.svg) A room is addressed through `/rooms//...`. It stores the conversation list, command receipts and reminders. Each conversation and history-search child has its own Durable Object, SQLite journal and alarm. The app registers two ordinary Agents: `durable-chat` and `history-search`. Both use `Engine::builder().hosting(ObjectHosting::new(...))`. The child has only the search tool; the main agent has scheduling and history-search tools. Framework owns the model/tool loop, lifecycle, event history, interruption recovery, iteration limits and completion metadata. `ObjectHosting` connects Engine to the Durable Object’s storage and alarms. Engine runs as WebAssembly, with the Worker providing storage, network requests and wakeups. Each session keeps the model selected when it was created; changing `MODEL` affects new sessions. The shared OpenRouter driver handles Responses request encoding, streaming, provider options, retries, error mapping and usage metadata. The host adapter supplies `fetch`, bounded response bytes, cancellation, endpoint policy and the server-side credential. ## Accept commands without losing work Every create, fork, send and schedule cancellation includes a stable `request_id`. The room stores a command fingerprint and receipt. A matching retry returns the same receipt; reusing the ID for another command is rejected. Before accepting a mutation, the room arms its alarm. It then commits the receipt and outbox intent in one SQLite document update, without yielding between them. The room alarm delivers pending intents through Framework. Message delivery uses `Session::send_with_id` with an identity derived from the durable outbox entry. A crash after admission but before deleting the outbox entry resends the same identity, so Framework deduplicates it. The room receipt acknowledges a durable delivery obligation. Framework’s own admission receipt is the authority for session execution. A room outbox is application delivery state; it does not replace the actor journal. ## Fork a committed prefix A fork reads canonical events through Framework and selects committed user and assistant messages through the chosen boundary. It creates another registered session and uses the validated empty-history seed port to import that prefix. Pending tools, schedules and execution ownership are not inherited. A tool-call message cannot be used as a fork boundary. The room’s event projection serves rendering, search and fork selection. Framework constructs model context from the session’s canonical history. ## Schedule reminders without a browser The scheduling tool persists an instruction, deadline and optional recurrence. Its receipt is keyed by Framework’s tool execution identity, so retrying the same operation returns the existing schedule. The room alarm advances a due occurrence and records its delivery intent in the same write. Delivery enters the target session through normal Framework admission. An occurrence has a stable message identity. Repeated delivery after a crash does not create another admission. Missed recurring ticks collapse into one follow-up. Cancellation disables future occurrences; it does not retract an already admitted instruction. Schedules are an explicit application service here. A durable session journal alone does not provide a scheduler or room directory. ![A reminder is persisted in the room, browsers disconnect, the room alarm admits a stable reminder message, the session alarm runs Engine and commits a reply, and a reconnecting browser reads the result.](https://everruns.com/docs/_astro/durable-chat-reminders.Cwp8NirP_1Fw73Q.svg) ## Recover a history-search child `search_history` creates a child keyed by the parent tool execution identity. That identity distinguishes different assistant tool-call messages even when a provider reuses a call ID. The child is an ordinary Framework session with its own durable admission and alarm. It can query room history through `search_conversations`, returning conversation IDs and sequence numbers. ![A second conversation invokes search_history, receives a cited PostgreSQL decision and answers. The sidebar shows the original conversation, a fork and independent history-search children.](https://everruns.com/docs/_astro/durable-chat-history.DusPOBKs_Z2pftoi.webp) **Decision lookup** retrieves the choice recorded in **Launch plan**. The sidebar also shows **Fork of Launch plan** and the independent **History search** children. The tool result carries source conversation IDs and sequence numbers. A parent interrupted while waiting can repeat this idempotent tool. It finds the same child and reads its reconstructible admission outcome; it does not create another child. Framework handles the interrupted tool and actor recovery. The room only stores application ownership and delivery state. Search children and search-generated assistant answers are excluded from future search evidence. Retrieved transcripts are untrusted evidence, not instructions. ## Deploy and verify ```sh npm run build npm test npm run check:bundle npx wrangler deploy --dry-run npx wrangler login npm run deploy npx wrangler secret put OPENROUTER_API_KEY ``` The public Worker handles browser requests. Room and session Durable Objects communicate through private bindings, and the OpenRouter API key stays on the server. Public access and model costs This example has no sign-in, room access controls or per-user quotas. Anyone with its URL can open rooms and send messages using the configured model budget. Add access and usage controls before sharing your own deployment. The test suite covers shared conversations, forks, interrupted history searches and scheduled reminders across Worker restarts. It runs locally with workerd and simulated model responses. See [Durable Object hosting](https://everruns.com/docs/framework/durable-objects/) for hosting guarantees and upgrade compatibility, and [Cloudflare’s platform limits](https://developers.cloudflare.com/workers/platform/limits/) for deployment constraints. --- # Everruns Support Agent > Search and read citable documentation before answering Framework questions. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/everruns-support-agent). Answer Framework questions by searching and reading a small, inspectable corpus of official documentation. The agent retrieves evidence before answering instead of routing keywords to prewritten responses. ![Everruns Support Agent terminal demo](https://raw.githubusercontent.com/everruns/everruns/main/examples/everruns-support-agent/demo/demo.gif) ## What you learn A two-step search/read tool interface, citable evidence, and an explicit boundary around the available knowledge. ## Scenario and expected outcome The default question asks how to resume a durable session after restarting a process. The agent should search for persistence and session history, read the relevant pages, then explain the local catalog, persisted `SessionId`, and agent reattachment requirements with source URLs. It must not imply that `Engine::new()` survives a restart. When the corpus lacks an answer, it should say so instead of inventing one. ## Run it Install Rust/Cargo, clone the repository, and run from its root. This folder is self-contained **within the workspace**: its Cargo manifest references local Framework crates, so copying the folder alone is not sufficient. ```bash git clone https://github.com/everruns/everruns.git cd everruns export ANTHROPIC_API_KEY="your-key" cargo run -p everruns-framework-support-agent ``` The configured model is `claude-opus-5-5`. Provider access and funded credits are required. Keep keys in your environment, not in source control. Missing credentials, provider errors, and unsuccessful turns exit nonzero. Try another question: ```bash cargo run -p everruns-framework-support-agent -- "How do I register a custom provider?" cargo run -p everruns-framework-support-agent -- "How can a tool return a structured error?" ``` Or type a question interactively: ```bash cargo run -p everruns-framework-support-agent -- --interactive ``` ## Build the agent Tools retrieve evidence; Opus decides what to search, read, and explain. ```rust pub fn build(provider: impl Into) -> Result { Agent::builder() .name("everruns-support-agent") .instructions(include_str!("resources/instructions.md")) .provider(provider) .model(MODEL) .max_iterations(12) .tool(tools::search_docs()) .tool(tools::read_doc()) .build() } ``` ## Send, observe, and wait The shared demo helper subscribes before sending, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. Use `session.send_and_wait(&question).await?` when a live tool timeline is unnecessary. ```rust // ANTHROPIC_API_KEY, declared by the Anthropic driver itself. let agent = agent::build(everruns::Anthropic::from_env()?)?; let engine = Engine::new(); let session = engine.create(agent); println!("MODEL: {}", agent::MODEL); demo::run(&session, &question).await?; ``` This engine is in-memory. The agent can explain durable sessions from its corpus, but the example itself does not persist its session. ## How the tools work `search_docs` ranks matches against five bundled pages and returns page IDs, public URLs, and matching excerpts. `read_doc` accepts only those page IDs and returns the complete snapshot. It never accepts an arbitrary filesystem path. ## Validate the behavior ```bash cargo test -p everruns-framework-support-agent bash examples/everruns-support-agent/demo/record.sh --check ``` Tests cover content-based search, empty and unmatched queries, complete citable reads, arbitrary-path rejection, and interactive input validation. They do not grade the model’s answer; compare a live response with the expected outcome above. CI runs the offline checks without provider credentials. Live model behavior is evaluated separately. ## Demo and recording Watch the screencast or read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/everruns-support-agent/demo/transcript.txt) to see the tool calls and answer. With credentials exported and VHS, ffmpeg, and a VHS-compatible browser installed: ```bash bash examples/everruns-support-agent/demo/record.sh ``` Export `ANTHROPIC_API_KEY` before recording. The script updates `demo/demo.gif` and `demo/transcript.txt` after a successful run. ## Adapt it Replace the bundled pages with a versioned documentation index while retaining separate search and read operations. Attach source/version metadata, restrict reads to authorized documents, and treat retrieved text as untrusted evidence rather than instructions. ## Boundaries The corpus is a five-page snapshot from 2026-09-08, not a live search of the docs site. Provenance is recorded in the [corpus README](https://github.com/everruns/everruns/blob/main/examples/everruns-support-agent/src/resources/docs/README.md), and the snapshot can lag current APIs. --- # Foreman > A decision service supervising a coding agent it never has to stop. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/foreman-agent). A fast decision service watching a slow coding agent, and a policy in ordinary Rust deciding what to do about the numbers. A Framework port of [thruwire/foreman](https://github.com/thruwire/foreman), which placed [TypeSafe’s Jev](https://docs.typesafe.ai/introduction) above a Codex worker and asked whether semantic supervision can run *while* the work happens. ![Foreman terminal demo](https://raw.githubusercontent.com/everruns/everruns/main/examples/foreman-agent/demo/demo.gif) ## What you learn How to run a worker session and observe it at the same time: a [`Decisions`](https://everruns.com/docs/framework/direct-model-calls/#decisions) turning bounded evidence into nine probabilities in one request, and a deterministic policy that owns every threshold, every limit, and the closed vocabulary of things the supervisor may do. ## The two loops The worker keeps its own loop — an Everruns session, or an external CLI in a child process. `session.send` returns a receipt immediately, and a child process is simply left running; either way the supervisory loop reads the evidence beside the live work. Activity is debounced to a floor, and only a worker finishing bypasses it. Nothing stops for the factory to think, and a test holds that claim: it counts readings taken while a worker’s turn is unresolved and fails if supervision waits its turn. ## What it assesses Five questions describe the job (`implementation_complete`, `tests_sufficient`, `requirements_satisfied`, `needs_verification`, `ready_to_finish`) and four describe the floor right now (`meaningful_progress`, `worker_stuck`, `work_off_track`, `needs_human`). Each is a Noul — the probability that a yes/no statement is true — and all nine ride one request, because questions in a decision are answered independently and in parallel. ## What it may do about them The decision service only estimates. The policy decides, safety and hard limits before productivity: escalate when a person is needed or the iteration ceiling is reached, stop a worker that is off track or stuck, retry once after a stop, finish when the completion thresholds hold and verification is resolved, start one independent verifier when a check is warranted, otherwise start or continue work. Thresholds and limits are Foreman’s defaults and are overridable through `FOREMAN_*` environment variables. ## What it looks at Never the repository. One bounded snapshot per reading: worker status, elapsed time, tool calls and output tails, `git status`, a bounded `git diff`, the untracked paths a diff cannot show, recent session events, verification results, and the previous assessment and decision. An unbounded observation would make supervision as slow as the work it is watching. That snapshot goes to the decision service on every reading, so a bounded slice of the repository leaves the machine on every run — point `--repo` at a private repository only if that is acceptable for it. `demo` works on a fixture it materializes itself, so it carries nothing of yours. The repository content in an observation is also untrusted input to the decision service, and that it can only produce a number is the point: the decision service never names an action, and every action the policy can take is in one readable file. ## Run it Foreman’s own two entry points, and they mean the same things here: ```bash cargo run -p everruns-foreman-agent --bin foreman -- demo foreman run --repo ./my-project --job "Add rate limiting, and test it." ``` Both commands use the same worker, decision service, and credentials. They differ in the repository and job they use: | Command | Repository | Job | Needs | | --------------- | ------------------------------------------- | ----------------------- | --------------------------------- | | `foreman demo` | a bundled fixture, in a temporary directory | one it ships with | `TYPESAFE_API_KEY` + the worker’s | | `foreman run …` | yours, named by `--repo` | yours, named by `--job` | the same | `demo` exists because a fixed starting state makes the ending checkable: the job names a rate schedule, so at the end the repository either prices by weight or it does not. Those checks run at the bottom of the run and read the files, not the supervisor’s opinion of them. `demo` writes its fixture into a temporary directory unless `--repo` says otherwise, and `run` never writes a fixture at all — `--repo` is your project, and the only thing that touches it is the worker. It will be modified. ## One supervisor The supervisor uses `Decisions` to ask a model nine questions in one request, within a fixed budget. Every run requires provider credentials; there is no offline mode. ## The supervisor runs the tests itself It already runs `git` rather than asking the worker what changed, and `--tests` applies the same reasoning to the suite: a worker reporting its own green tests is a claim, and an independently run result is a fact. The command runs in a networkless Docker container with no inherited host environment, over a read-only repository mount copied to isolated temporary storage. Docker is therefore required when `--tests` is used. The result lands in the observation as `test_results` — the field Foreman declares and never fills — and `tests_sufficient` moves on it. The suite runs once as a baseline before any worker starts, then whenever the floor is quiet; between times the last result is carried with its age, because a stale pass should not read as a fresh one. ## Who does the work `--worker` picks the crew. All three are watched identically, because what the supervisor reads is a bounded observation and the strongest evidence in one — the repository’s own diff — is gathered by the host either way. | `--worker` | What runs | Independent verification | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- | | `session` (default) | An Everruns session on the [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/), `meta/muse-spark-1.3-contributor` through OpenRouter | A second session under the default read-only workspace policy | | `codex` | `codex exec --cd … --sandbox workspace-write --color never --json …`, the line Foreman itself runs | The same CLI with `--sandbox read-only` | | `yolop` | `yolop -C … -p …`, its one-shot print interface | Mission only — yolop publishes no read-only mode | Anything else is a template: `--worker-command "mycli --cd {repo} --task {mission}"`, where both placeholders are substituted as whole arguments so no shell sees either. A session is observed through its own canonical event stream, which arrives already typed. A CLI offers none of that, so an external worker is observed through stdout and stderr, and a JSONL line that names its own `type` counts as a step. ## The floor The bundled fixture is a small shell project that prices every parcel at one flat rate; the job is to replace that with weight tiers and cover the boundaries. Shell, deliberately: `bash tests/run.sh` needs no framework, no interpreter and no network, so the same suite runs inside the Bashkit sandbox, on the host, and inside an external agent. On the session crew the coding worker mounts it read-write and the verifier mounts the same directory under the default read-only policy, so “independent check” is a property of the mount rather than a request in a prompt. The supervisor runs `git` itself rather than asking the worker what it did, which is also why an external CLI is supervised just as well as a session. ## What the Framework changes The architecture is Foreman’s; the runtime underneath it is not. A worker is a session rather than a subprocess, so stopping one is a cooperative turn cancellation instead of a signal to a process group. Evidence is the canonical event stream rather than parsed JSONL. A retry is a fresh session over the same workspace. Read-only verification is a workspace policy. Steering exists — sending into a live turn applies at the next iteration boundary — and is deliberately left out of the policy’s vocabulary so the comparison with the original stays honest. ## Limits This is an architectural experiment, and porting it does not make it a proven one. Decisions accuracy for this use is unproven and the thresholds are uncalibrated: false positives stop useful workers, false negatives let bad work continue. Observations are bounded and therefore incomplete. One coding worker runs at a time, a verifier reports evidence rather than proof, and the session state is in memory. --- # Host Shell Agent > Fix a failing Rust test suite by compiling and running it, inside a kernel-enforced boundary. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/host-shell-agent). Fix a failing Rust test suite by compiling and running it, through the [Host Shell](https://everruns.com/docs/capabilities/host-shell/) capability: real processes on this machine, bounded by a kernel policy. The agent uses `gpt-6.2-sol`. ## What you learn How to give an agent the machine it is running on without giving it the machine: one execution capability, a kernel-enforced write and network boundary, and independent host assertions as the success condition. This is the example the sandboxed [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/) cannot be. Compiling a crate and running its test binary needs a real toolchain and real child processes; an in-process interpreter over a virtual filesystem has neither. ## Scenario and expected outcome The bundled fixture is a small zero-dependency crate whose `chunk_count` drops the partial final chunk, so two tests fail. The agent must run the suite, read the failure, fix the source, and re-run until it passes. Before the agent starts, the example probes the same containment provider the capability uses and prints what it found: a write inside the workspace succeeds, a write outside it is refused by the kernel, an outbound socket is refused by the kernel. Not a description of the boundary, the boundary. After the turn, Rust code re-runs `cargo test` from the host and fails the process if the suite is still red, if an assertion was edited away, or if a test was marked `#[ignore]`. A confident model answer cannot make a red suite pass. ## Run it Clone the repository and run from its root: ```bash export OPENAI_API_KEY="your-key" cargo run -p everruns-host-shell-agent ``` Type the task interactively, or point it at a directory you keep: ```bash cargo run -p everruns-host-shell-agent -- --interactive cargo run -p everruns-host-shell-agent -- /tmp/chunker ``` With no directory, the fixture is materialized into a fresh temporary one and discarded afterwards. ## Requirements Kernel containment needs macOS, or Linux with Landlock ABI v3 fully enforced (Linux 6.2 or a backport). On a kernel that cannot enforce it, the provider fails closed and the example says so instead of running uncontained. On Linux the policy is applied by a helper process, and this example is its own: `main` routes a `__sandbox-exec` re-exec into the containment worker and names that through `SandboxLauncher::ReexecSelf`, which is what a single-binary embedder does. Nothing else needs to be on disk. --- # Incident Commander Agent > Multi-tool evidence gathering, distinguishing correlation from causation, and a narrowly scoped append-only side effect. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/incident-commander-agent). Investigate a fictional checkout alert using contrasting metrics, deployment history, logs, and a runbook; then persist an evidence-backed status update. The agent must investigate before recommending action. ## What you learn Multi-tool evidence gathering, distinguishing correlation from causation, and a narrowly scoped append-only side effect. ## Scenario and expected outcome Checkout errors rise from 0.2% to 18.4% after v43 reduces the payment timeout from 2,000 ms to 200 ms. Catalog traffic and database metrics stay normal; sampled logs show requests timing out just beyond the new deadline. Read metrics, deployments, logs, and the runbook. Identify the timeout reduction as a likely cause, not proven causation. Propose on-call ownership and approval for rollback consideration. Record that conclusion locally without claiming any production mitigation occurred. ## Run it Install Rust/Cargo, clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient. ```bash git clone https://github.com/everruns/everruns.git cd everruns export MODEL_API_KEY="your-key" cargo run -p everruns-incident-commander-agent ``` The configured model is `muse-spark-1.3`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question: ```bash cargo run -p everruns-incident-commander-agent -- "Investigate checkout and explain what evidence argues against a database-wide incident. Record a concise update." ``` ## Build the agent This builder combines the agent instructions, tools, and model. Tools/capabilities supply evidence and actions; the model chooses how to use them. ```rust let agent = Agent::builder() .name("incident-commander-agent") .instructions(include_str!("instructions.md")) .provider(everruns::Meta::from_env()?) .model(MODEL) .max_iterations(12) .tool(tools::inspect_evidence()) .tool(tools::record_incident_update()) .build()?; ``` ## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline. ```rust let engine = Engine::new(); let session = engine.create(agent); println!("MODEL: {MODEL}"); demo::run(&session, question).await?; ``` This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `inspect_evidence` exposes only four named fixture categories. `record_incident_update` appends a non-empty update of at most 500 UTF-8 bytes to this example’s `src/incident.log`, normalizing newlines. There is deliberately no production rollback tool. ## Validate the behavior ```bash cargo test -p everruns-incident-commander-agent python3 examples/incident-commander-agent/src/render_demo.py --check ``` Tests verify that updates survive multiple writes, oversized/empty updates are rejected before a file is created, and evidence access is scoped. A live run should show all evidence reads followed by a persisted update that matches the observed facts. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording ![Incident Commander Agent recorded run](https://raw.githubusercontent.com/everruns/everruns/main/examples/incident-commander-agent/src/demo.gif) Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/incident-commander-agent/src/demo.txt) for the full recorded output. The GIF shows excerpts from the run. With credentials exported and Python 3, VHS, ffmpeg, and a VHS-compatible browser installed: ```bash cd examples/incident-commander-agent bash src/record.sh ``` The script captures a successful run, generates correctly wrapped pages and page durations, and renders `src/demo.gif`. It preserves the previous transcript when the provider run fails. To replay an existing transcript without another model call, run `(cd src && python3 render_demo.py && vhs demo.tape)`. `src/demo.txt` retains the displayed output; `.demo-pages/` is generated and ignored. Inspect results before sharing: public/demo data is safe here, but adapting tools may expose private data. ## Adapt it Replace each fixture with a read-only monitoring/deployment API scoped to the relevant service. Keep investigation separate from action. Add explicit approval and an audit trail before introducing any production mutation. ## Boundaries All telemetry is fictional. The log is a real local append-only artifact, ignored by Git; it contains exercise text and does not change production. Filesystem permissions and rotation are the application’s responsibility. --- # Research Agent > Composing the first-party Brave Search and WebFetch capabilities, handling external evidence, and reporting uncertainty. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/research-agent). Search for relevant material, open primary sources, and synthesize a short, cited research brief. The model must read source content rather than treating search snippets as sufficient evidence. ## What you learn Composing the first-party Brave Search and WebFetch capabilities, handling external evidence, and reporting uncertainty. ## Scenario and expected outcome The default question asks what durable execution guarantees about retries and external side effects. A useful answer must distinguish replaying recorded results from the risk of retrying an unrecorded side effect. Read at least two primary sources successfully before answering. Cite those pages, distinguish documented guarantees from inference, and explain why idempotency can still be necessary. If a source is inaccessible, try another and disclose the gap. Search snippets alone do not meet the task. ## Run it Install Rust/Cargo, clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient. ```bash git clone https://github.com/everruns/everruns.git cd everruns export OPENROUTER_API_KEY="your-key" export BRAVE_SEARCH_API_KEY="your-key" cargo run -p everruns-research-agent ``` The configured model is `z-ai/glm-5.2`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question: ```bash cargo run -p everruns-research-agent -- "Compare retry guarantees in Temporal Activities and Restate. Read official sources and give two findings plus one caveat." ``` ## Build the agent This builder combines the agent instructions, tools, and model. Tools/capabilities supply evidence and actions; the model chooses how to use them. ```rust let agent = bound_external_calls( Agent::builder() .name("research-agent") .instructions(include_str!("instructions.md")) .provider(everruns::OpenRouter::from_env()?) .model(MODEL), ) .capability(BraveSearch::from_env()?) .capability(everruns::WebFetch::new()) .build()?; // Search and fetched pages are untrusted model input. Keep one injected result from // multiplying paid provider requests or scheduling a batch of outbound calls. fn bound_external_calls(builder: AgentBuilder) -> AgentBuilder { builder .max_iterations(MAX_ITERATIONS) .parallel_tool_calls(false) } ``` ## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline. ```rust let engine = Engine::new(); let session = engine.create(agent); println!("MODEL: {MODEL}"); demo::run(&session, question).await?; ``` This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `BraveSearch::from_env()` provides `brave_web_search`. `WebFetch::new()` provides `web_fetch`, enabled through the Framework `web-fetch` Cargo feature. Search chooses candidates; fetch retrieves page content using the existing integration’s egress controls. Download-to-file is not enabled. ## Validate the behavior ```bash cargo test -p everruns-research-agent python3 examples/research-agent/src/render_demo.py --check ``` Offline tests cover evidence-preview rendering and recording pagination; they do not perform web research. Validate a live run by checking successful `web_fetch` results for at least two primary sources and matching the final citations to pages actually read. Network/provider behavior and factual quality are not guaranteed by a green offline test. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording ![Research Agent recorded run](https://raw.githubusercontent.com/everruns/everruns/main/examples/research-agent/src/demo.gif) Read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/research-agent/src/demo.txt) for the full recorded output. The GIF shows excerpts from the run. With credentials exported and Python 3, VHS, ffmpeg, and a VHS-compatible browser installed: ```bash cd examples/research-agent bash src/record.sh ``` The script captures a successful run, generates correctly wrapped pages and page durations, and renders `src/demo.gif`. It preserves the previous transcript when the provider run fails. To replay an existing transcript without another model call, run `(cd src && python3 render_demo.py && vhs demo.tape)`. `src/demo.txt` retains the displayed output; `.demo-pages/` is generated and ignored. Inspect results before sharing: public/demo data is safe here, but adapting tools may expose private data. ## Adapt it Narrow the research question and source policy, add an evidence store if results need to survive sessions, and validate citation coverage before publishing important findings. Treat fetched text as untrusted data, not as instructions. ## Boundaries Requires both OpenRouter and Brave Search credentials plus outbound HTTPS. Search and fetch can fail; six agent iterations cap the loop, not the bill. Word limits are instructions, not a hard output validator. This is a small research workflow, not an exhaustive literature review. --- # Support Agent > Typed read-only tools, separating facts from policy, and choosing different answers for different inputs. Source: [Browse the complete example](https://github.com/everruns/everruns/tree/main/examples/support-agent). Diagnose a sign-in problem by combining account facts with an explicit recovery policy. The interesting decision is whether the user needs MFA recovery, must wait for a lockout, or should try a clean browser session. ![Support Agent terminal demo](https://raw.githubusercontent.com/everruns/everruns/main/examples/support-agent/demo/demo.gif) ## What you learn Typed read-only tools, separating facts from policy, and choosing different answers for different inputs. ## Scenario and expected outcome The default customer reset their password, but has MFA enabled and neither an authenticator nor recovery codes. A password reset alone cannot solve this case. For `cust_mfa`, direct the user to verified identity recovery, explicitly noting that resetting a password does not disable MFA. Never request passwords or recovery codes. For `cust_locked`, explain the 15-minute wait. For `cust_browser`, suggest a private window and an escalation if it still fails. ## Run it Install Rust/Cargo, clone the repository, and run from its root. These folders are self-contained **within the workspace**: their Cargo manifests reference the local Framework crates, so copying one folder alone is not sufficient. ```bash git clone https://github.com/everruns/everruns.git cd everruns export OPENAI_API_KEY="your-key" cargo run -p everruns-support-agent ``` The configured model is `gpt-6.2-sol`. Provider access and funded credits are required; a model identifier alone does not grant access. Keep keys in your environment, not in source control. Missing variables, provider errors, or unsuccessful turns exit nonzero. Try a contrasting question: ```bash cargo run -p everruns-support-agent -- "cust_locked reset their password but cannot sign in. What should they do?" cargo run -p everruns-support-agent -- "cust_browser cannot sign in after a reset. What next?" ``` Or enter a question interactively: ```bash cargo run -p everruns-support-agent -- --interactive ``` ## Build the agent Tools supply evidence; the model chooses how to use it. ```rust pub fn build(provider: impl Into) -> Result { Agent::builder() .name("support-agent") .instructions(include_str!("resources/instructions.md")) .provider(provider) .model(MODEL) .max_iterations(12) .tool(tools::lookup_customer()) .tool(tools::read_support_policy()) .build() } ``` ## Send, observe, and wait The shared demo helper subscribes before sending, filters events to this turn, shows bounded tool previews, waits for completion, and rejects unsuccessful turns. It changes presentation only; use `session.send_and_wait(question).await?` when you do not need the live tool timeline. ```rust // OPENAI_API_KEY, declared by the OpenAI driver itself. let agent = agent::build(everruns::OpenAI::from_env()?)?; let engine = Engine::new(); let session = engine.create(agent); println!("MODEL: {}", agent::MODEL); demo::run(&session, question).await?; ``` This engine is in-memory. It does not demonstrate durable session storage; the Everruns Support example can explain that API, but does not itself persist its session. ## How the tools work `lookup_customer` reads one of three fictional customer records. It returns facts, not a prewritten recommendation. `read_support_policy` returns the bundled recovery rules. The model combines the two; no tool disables MFA or changes a real account. ## Validate the behavior ```bash cargo test -p everruns-support-agent bash examples/support-agent/demo/record.sh --check ``` Tests cover the distinct account states and rejection of unknown IDs. They do not grade the model’s recommendation: compare a live response with the expected outcomes above. These checks run offline without provider credentials. To check the model’s answer, run the example and compare its output with the expected outcome above. ## Demo and recording Watch the screencast or read the [captured transcript](https://github.com/everruns/everruns/blob/main/examples/support-agent/demo/transcript.txt) to see the tool calls and answer. With credentials exported and VHS, ffmpeg, and a VHS-compatible browser installed: ```bash bash examples/support-agent/demo/record.sh ``` Export `OPENAI_API_KEY` before recording. The script updates `demo/demo.gif` and `demo/transcript.txt` after a successful run. ## Adapt it Replace the fixture lookup with your authorized customer-data service. Keep policy separate from account facts, scope lookups to the authenticated customer, and return only fields needed for the support decision. Add human approval before any account mutation. ## Boundaries All customers, policy rules, and support.example.com URLs are fictional. This is a read-only support exercise, not a live help desk. --- # Typed Extractions > Extract typed values and evidence from documents without an agent loop. Source: Extractions ask for named fields in text or images and return typed values, evidence and uncertainty. They are separate from Decisions (classification) and structured chat (schema-shaped generated output). The Framework exposes the same contract as the Server. No agent, session or tool loop is required. The upstream OpenRouter API is **alpha**. Providers differ in supported fields and assets; an unsupported request fails rather than silently switching to chat or inventing confidence. ## Framework ```rust use everruns::{Extractions, ExtractionField}; async fn example() -> Result<(), Box> { let key = std::env::var("OPENROUTER_API_KEY")?; let provider = everruns_drivers::openrouter::provider("openrouter", key); let result = Extractions::new("cloudflare/clairic", provider) .about("Invoice INV-42. Total USD 120.00.") .field("invoice_number", ExtractionField::text()) .field("total", ExtractionField::number()) .send().await?; println!("{:?}", result.answers); Ok(()) } ``` Enable the driver’s `openrouter` feature. Run the synthetic invoice example: ```bash OPENROUTER_API_KEY=... cargo run -p everruns --features utility-llm --example direct_extractions ``` This sends a paid provider request. Set `EXTRACTION_MODEL` to another extraction model supported by your account. Do not use real private documents in tests without authorizing their transmission. For direct Cloudflare access, construct `everruns_drivers::cloudflare::provider` with your account ID and API token and select `@cf/cloudflare/clairic`. It uses Workers AI directly, without an OpenRouter credential. The Cloudflare driver accepts text and images, not PDFs. Run `cargo run -p everruns --features cloudflare --example direct_cloudflare_extractions` with `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN` to try the synthetic invoice directly (provider charges may apply). Other upstream vendors are available through OpenRouter; they are not advertised as direct integrations here. ## Server and console Configure an organization-owned provider key and add or sync an extraction-capable model. In Platform, first enable **Extractions** in **Settings → Features** for the organization. Open **Extractions** in the console to supply text and field specifications, then inspect values and evidence. Inline assets are supported through the API; the initial console does not include an asset upload widget. Set `EVERRUNS_URL` to the API base URL, for example `http://localhost:27100/api` for a local stack. ```bash curl "$EVERRUNS_URL/v1/extractions" \ -H "Authorization: Bearer $EVERRUNS_API_KEY" \ -H 'Content-Type: application/json' \ -d '{"model_id":"YOUR_MODEL_ID","input":{"state":"Invoice INV-42","fields":{"invoice_number":{"type":"text"}}}}' ``` The API requires model-management permission and an enabled extraction model in the authenticated organization. Read-only credentials cannot make paid calls. This initial endpoint supports organization BYOK providers; hosted/billed models are rejected until their extraction accounting path is available. Model/provider overrides inside `input` are rejected. Credentials stay on the server. Requests carry no arbitrary remote document URL: assets are inline base64. OpenRouter transport additionally caps fields at 128, state at 2 MiB and encoded assets at 12 MiB as client-side safeguards, not vendor limit claims. The contract caps assets at 32 and their aggregate encoded data at 32 MiB before decoding. Driver-specific limits may be lower; the Server limits the whole request body to 8 MiB. ## Values, absence and evidence Field types are `text`, `number`, `date`, `time`, `datetime`, `duration`, `list`, `estimate` and `forecast`. `ExtractionField` validates type-specific options before network access. The outcome preserves provider usage, `truncated`, optional state text and every returned field’s value. * `value: null` is a valid absent answer. Do not replace it with `guess` automatically. * Confidence and `p_null` are optional provider observations, not calibrated guarantees. Missing confidence means unknown, not zero or one. * `span` and per-item `spans` use Unicode **code points**, not byte or JavaScript UTF-16 offsets. In JavaScript use `Array.from(state).slice(start, end).join('')`. A list may have null span entries. * Use returned `state_text` when present. Truncation means the model did not read the whole input. * Estimates and forecasts preserve distribution statistics; they are not exact source facts. * Image/document answers may have no textual span. Do not fabricate evidence. ## Deployment-owned utility With the Framework’s `utility-llm` feature, `utility_extractions::utility_extractions_from_env()` selects the explicit `UTILITY_EXTRACTION_MODEL` and `UTILITY_OPENROUTER_API_KEY`. With no extraction model configured, it is disabled. Tenant extraction endpoints never fall back to these deployment credentials. ## Document conversion and Markkit Keep conversion separate from inference. Convert a local document to Markdown with Markkit, review the text, then pass that text as extraction state. This preserves Markkit’s deterministic offline boundary and keeps provider credentials in Everruns. The example accepts a reviewed Markdown file from a separate converter: ```bash EXTRACTION_INPUT_FILE=reviewed-invoice.md OPENROUTER_API_KEY=... \ cargo run -p everruns --features utility-llm --example direct_extractions ``` This explicitly sends that file’s text to the provider. For scanned documents, use a provider-supported asset input instead: conversion quality and OCR are not guaranteed by the extraction contract. ## Verification Unit and mock-HTTP tests cover typed validation, malformed answers, Unicode spans, auth/tenant checks and sanitized errors. The live qualification harness is opt-in, uses synthetic documents and requires separate credentials and an explicit spending decision. A mock pass does not prove vendor availability or extraction accuracy. After explicitly approving provider spending and setting the provider credentials, run the synthetic-document live qualification harness: ```bash EVERRUNS_EXTRACTIONS_LIVE=1 EXTRACTIONS_LIVE_PROVIDER=openrouter \ cargo bench -p everruns-drivers \ --no-default-features --features openrouter,cloudflare \ --bench extractions_live ``` Select `openrouter` or `cloudflare` with `EXTRACTIONS_LIVE_PROVIDER`; do not supply credentials for a provider you do not intend to call. See [OpenRouter’s extraction contract](https://openrouter.ai/docs/guides/community/extractions) and [Cloudflare Clairic](https://developers.cloudflare.com/workers-ai/models/clairic/) for current limits. --- # GitHub tools > Read pull requests and repository files, search GitHub, and explicitly enable write tools in a Framework application. Source: Give an agent GitHub tools with `everruns::github::GitHub`. The capability uses the same GitHub operations as the Platform, with credentials your application supplies. It requires no Everruns server or database. Until these APIs are included in the next crate release, use the repository’s `main` branch or run the examples from a source checkout: ```toml [dependencies] everruns = { git = "https://github.com/everruns/everruns", branch = "main", features = ["github", "openai"] } tokio = { version = "1", features = ["macros", "rt-multi-thread"] } ``` The `github` feature is optional. It adds the GitHub HTTP integration without enabling other vendors or the hosted capability catalog. ## Read a pull request Set `GITHUB_TOKEN` with repository read access and `OPENAI_API_KEY`, then run: ```rust use everruns::{Agent, Engine, OpenAI, github::GitHub}; #[tokio::main] async fn main() -> Result<(), Box> { let agent = Agent::builder() .instructions("Read the requested pull request and its diff. Report concrete bugs with evidence.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability(GitHub::from_env()?) .max_iterations(12) .build()?; let engine = Engine::new(); let result = engine.create(agent) .send_and_wait("Review pull request 123 in my-org/my-repo.") .await?; println!("{}", result.response); Ok(()) } ``` The [complete PR reviewer example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/github_review.rs) takes the repository and PR number as arguments: ```bash cargo run -p everruns --features github,openai --example github_review -- OWNER/REPO NUMBER ``` It runs a real model against GitHub and prints its review. Its GitHub capability is read-only, so it cannot post comments or reviews. ## Tool selection and write controls | Configuration | Available tools | | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | Default | `get_github_pull_request`, `get_github_pull_request_diff`, `search_github_code`, `read_github_file`, `search_github_issues` | | `.allow_writes(true)` | Also `upsert_github_comment`, `submit_github_pull_request_review`, `upsert_github_issue` | | `.allow_writes(true).allow_pull_requests(true)` | Also `create_github_pull_request` | | `.allow_writes(true).allow_merge(true)` | Also `mark_github_pull_request_ready`, `merge_github_pull_request` | Enable writes only for an application that should modify GitHub: ```rust use everruns::github::GitHub; let github = GitHub::from_env()? .allow_writes(true) .private_issues_only(true) .allow_pull_requests(true); ``` These options decide which tools exist; instructions cannot enable an absent tool. The GitHub token also needs the corresponding repository permissions. `private_issues_only(true)` checks repository visibility before filing an issue. Reviews can comment or request changes, and cannot approve. PR creation defaults to a draft and returns an existing open PR from the same head branch. Comments and findings carry markers for updates and duplicate suppression. Finding/review matching trusts bot-authored records. Use a GitHub App installation token for repeatable bot writes; personal-token runs do not gain a bot identity. ## Tokens and refresh `GitHub::new(token)` keeps a token in application memory. `GitHub::from_env()` reads `GITHUB_TOKEN` once at startup and rejects empty values. Neither stores credentials in capability configuration or includes them in debug output. For short-lived installation tokens or rotating credentials, implement `GitHubTokenProvider`. It is called on each operation with the repository and requested permissions. An application can mint the least-privileged token it needs; a static personal token retains its original permissions. ```rust use everruns::capability::async_trait; use everruns::github::{GitHub, GitHubTokenProvider, GitHubTokenRequest}; struct ApplicationTokens; #[async_trait] impl GitHubTokenProvider for ApplicationTokens { async fn token(&self, request: &GitHubTokenRequest) -> Result { // Replace this with your application's refresh or installation-token service. // request.repository and request.permissions describe this operation's needs. let _ = request; std::env::var("GITHUB_TOKEN").map_err(|error| error.to_string()) } } let github = GitHub::with_token_provider(ApplicationTokens); ``` Credential-provider failures are returned as generic errors; raw provider errors and GitHub error response bodies are not exposed to the model. A failed provider does not fall back to process or session credentials. `.api_base("https://github.example.com/api/v3")` selects a trusted GitHub Enterprise API root. It receives the credential, so configure it in application code. Search availability follows your GitHub installation and token permissions. ## Platform boundary The Framework adapter shares tool schemas, validation, diff limits, review policy, and vendor operations with the Platform. The Platform still owns GitHub App setup, webhooks/triggers, user-versus-agent identity selection, and connection storage. Managed sandbox checkout requires an attested installation credential and a sandbox service; this Framework capability does not advertise that tool. See [capability integrations](https://everruns.com/docs/framework/capability-integrations/) for other optional integrations and [moving to Platform](https://everruns.com/docs/framework/moving-to-platform/) for deployment differences. --- # Host Services > Compose an embedding host's own capabilities, provider resolution, retry budgets, MCP credentials, request metadata and observers into Engine sessions. Source: An embedding host often owns more than an agent definition: a curated set of capability implementations with guards and context controls, its own provider and credential resolution, provider recovery budgets, refreshing MCP tokens, request attribution and event observers. `everruns::hosting::services` composes those into the runtimes `Engine` builds, so the host keeps its behavior and still uses Engine’s sessions, durability and turn runner. Stability: experimental, together with [session hosting](https://everruns.com/docs/framework/hosting/). ## What host services can and cannot change `HostServices` is an immutable value attached to an Agent with `AgentBuilder::host_services`. Engine applies it every time it constructs that Agent’s runtime: for new sessions, resumption, durable activations and context inspection. | Host services contribute | Engine keeps owning | | ------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- | | Capabilities written against the Core `Capability` contract, with pre-tool hooks, model views, slash commands and instructions | Session identity, admission, leases and the turn runner | | Provider resolution through a host `ProviderStore` and `DriverRegistry` | The Environment-selected session filesystem and its containment | | Provider retry and stream-stall budgets | Canonical history: observers only observe | | Call-time MCP credentials | Framework capability ids, aliases and tool names | | The outbound boundary for runtime traffic (MCP servers, web tools) | The address checks the MCP transport makes before a request reaches that boundary | | Credential-free provider-request metadata and OpenRouter attribution | Request correlation ids such as `session_id` and `turn_id` | There is no callback that receives a runtime builder, and no runtime handle to drive. A host capability whose id, alias or tool name collides with a Framework capability or another input fails `AgentBuilder::build`, so a host cannot replace, for example, the filesystem capability by registering the same id. Capability tools reach files only through the session filesystem in their `ToolContext`, which is the one the session’s Environment selected. ## Add a host capability Implement `Capability` for the host’s policy. This one blocks one tool before it runs and adds a reminder to every model call: ```rust use std::sync::Arc; use everruns::hosting::services::{ Capability, HostServices, ModelViewContext, ModelViewProvider, PolicyGate, PreToolUseDecision, PreToolUseHook, RuntimeMessage, ToolCall, ToolContext, ToolDefinition, async_trait, }; use everruns::{Agent, Engine, OpenAI}; use serde_json::Value; struct NoForcePush; #[async_trait] impl PreToolUseHook for NoForcePush { async fn before_exec( &self, tool_call: ToolCall, _definition: &ToolDefinition, _context: &ToolContext, ) -> PreToolUseDecision { if tool_call.name == "git_push" && tool_call.arguments["force"] == true { return PreToolUseDecision::Block { tool_call, reason: "force pushes are disabled by policy".into(), user_message: None, }; } PreToolUseDecision::Continue(tool_call) } } struct Reminder; impl ModelViewProvider for Reminder { fn apply_model_view( &self, mut messages: Vec, _config: &Value, _context: &ModelViewContext<'_>, ) -> Vec { messages.push(RuntimeMessage::system("Keep changes reviewable.")); messages } } struct RepoPolicy; impl Capability for RepoPolicy { fn id(&self) -> &str { "repo_policy" } fn name(&self) -> &str { "Repository policy" } fn description(&self) -> &str { "Guards repository operations." } fn system_prompt_addition(&self) -> Option<&str> { Some("Never rewrite published history.") } fn pre_tool_use_hooks(&self) -> Vec> { vec![Arc::new(PolicyGate(NoForcePush))] } fn model_view_provider(&self) -> Option> { Some(Arc::new(Reminder)) } } let agent = Agent::builder() .instructions("You maintain this repository.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .host_services(HostServices::new().capability(RepoPolicy)) .build()?; let session = Engine::new().create(agent); let context = session.inspect().await?; assert!(context.instructions.contains("Never rewrite published history.")); ``` `PolicyGate` marks the hook as a policy decision, so it runs after any hook that rewrites arguments and decides on the call that will execute. A blocked call never reaches its tool; the model receives the reason as the call’s result. Commands a capability contributes appear in `SessionContext::commands`. `HostServices::capability` registers and activates. `HostServices::register` only registers, and the Agent activates it with an ordinary `.capability("repo_policy")` reference or through its Harness. ## Resolve providers through the host With `HostProviders`, the host’s `ProviderStore` owns endpoints and credentials and the Agent names its model with `Model::on(provider_key, model_id)`. The store is asked under the calling session’s identity on every provider call, so rotated credentials and provider metadata reach the next call without rebuilding anything. ```rust use everruns::hosting::services::{HostProviders, HostServices, ProviderStore}; use everruns::{Agent, DriverRegistry, Model}; let mut drivers = DriverRegistry::new(); everruns::drivers::openai::register_driver(&mut drivers); let agent = Agent::builder() .instructions("You are concise.") .model(Model::on("team-openai", "gpt-6.2-sol")) .host_services(HostServices::new().providers(HostProviders::new(store, drivers))) .build()?; ``` Selection is never implicit. Building fails with `ConflictingProviders` when a host-resolved model also carries an Agent provider, or when `Model::on` names a different provider than `.provider(..)`. A provider key the host cannot serve fails runtime construction before any model call, rather than falling back to another driver. ## Attribute provider requests `RequestMetadata` is a small set of credential-free strings that every provider request of the Agent’s sessions carries: turn model calls, out-of-band completions, and sessions Engine rebuilds or resumes, on both the in-process and the hosted lanes. `openrouter_attribution` sets the entries the OpenRouter driver sends as its `HTTP-Referer` and `X-Title` headers; other providers receive them as ordinary request metadata. ```rust use everruns::hosting::services::{HostServices, RequestMetadata}; use everruns::{Agent, Engine, OpenAI}; let metadata = RequestMetadata::new() .entry("app", "my-app") .entry("my_app_version", env!("CARGO_PKG_VERSION")) .openrouter_attribution("https://my-app.example", "My App"); let agent = Agent::builder() .instructions("You are concise.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .host_services(HostServices::new().request_metadata(metadata)) .build()?; let session = Engine::new().create(agent); let context = session.inspect().await?; assert_eq!(context.request_metadata["app"], "my-app"); ``` `SessionContext::request_metadata` shows the entries as assembled for the next request. The runtime adds its own correlation ids to each request after them. Building fails with `InvalidRequestMetadata` when an entry claims one of those ids (`RequestMetadata::RESERVED_KEYS`: `session_id`, `harness_id`, `turn_id`, `exec_id`, `org_id`, `agent_id`, `model_id`), when a key is not 1 to 64 ASCII letters, digits, `_`, `.`, `-` or `:`, when a value is longer than 512 bytes or holds control characters, or when there are more than nine entries. Providers and request capture see these values, so never put a secret in them. ## MCP readiness Cold MCP discovery waits for complete tool catalogs by default. Interactive hosts can opt into a short readiness budget with the `mcp` feature enabled: ```rust use std::time::Duration; use everruns::hosting::services::HostServices; let services = HostServices::new() .mcp_readiness_budget(Duration::from_millis(250)); ``` The budget applies once to the whole eager discovery pass, not separately to every server. Expiry does not turn a pending connection into an error or expose partial catalogs. Discovery continues with bounded concurrency, queue size, response size and its existing hard deadline. Completed tools are available on subsequent turns, including when an Engine rebuilds its runtime between turns. Unrevealed deferred servers still connect only when revealed. With this option enabled, `session.mcp_discovery_statuses().await` reports `Pending`, `Ready`, `AuthenticationRequired`, or `Failed` for eager and revealed servers. It does not initiate discovery. `Pending` also covers servers whose discovery has not started. Authentication failures retain their reconnect hold; transient failures retain the existing retry cooldown. The shared cache remains scoped by session and attachment configuration. Reuse the same `HostServices` value for runtimes that should share completion state. ### MCP status, reconnect and doctor A host can drive an MCP status display, a reconnect action and doctor output from the session alone: ```rust use everruns::{McpDiscoveryStatus, Session, SessionControlError}; async fn render(session: &Session) -> Result<(), SessionControlError> { let readiness = session.mcp_readiness().await?; for server in &readiness.servers { let state = match server.status { McpDiscoveryStatus::Ready => format!("ready, {} tools", server.tool_count), McpDiscoveryStatus::Pending if server.deferred => "deferred".into(), McpDiscoveryStatus::Pending => "connecting".into(), McpDiscoveryStatus::AuthenticationRequired => "sign-in required".into(), McpDiscoveryStatus::Failed => "failed".into(), }; println!("{}: {state} {}", server.name, server.reason.as_deref().unwrap_or("")); } Ok(()) } async fn reconnect(session: &Session, name: &str) -> Result<(), SessionControlError> { let after = session.probe_mcp(Some(name)).await?; if let Some(server) = after.server(name) { println!("{name}: {:?} at {:?}", server.status, server.last_probe); } Ok(()) } async fn follow(session: &Session) -> Result<(), SessionControlError> { let mut changes = session.watch_mcp_readiness().await?; while changes.changed().await.is_some() { render(session).await?; } Ok(()) } ``` * `mcp_readiness` reads, and never starts discovery. It returns one entry per configured server that discovers tools, including unrevealed deferred ones. Each entry carries the status, `deferred`, `probing` (an attempt is running), `tool_count`, a bounded failure or authentication `reason` and `last_probe` (when the last attempt completed). A server the runtime cannot serve, for example a stdio server in a build without `mcp-stdio`, reports `Failed` with a reason. * `probe_mcp(Some(name))` reconnects one server now, and `probe_mcp(None)` reconnects every eager and revealed server. The probe asks your `McpAuthProvider` again and skips the authentication hold and the retry cooldown. It waits up to the discovery deadline and returns the updated readiness. If an attempt is already in flight, the probe returns that attempt’s result. A transient failure keeps the previous tools for turns. Unknown names, and servers that do not discover tools, are rejected. * `watch_mcp_readiness` wakes when readiness may have changed. Every snapshot also carries a `revision` that you can poll and compare instead. Revisions are shared by sessions that use the same `HostServices`, so re-read readiness after each change. For hosted sessions, set `mcp_readiness_budget` so that readiness survives runtime rebuilds. For doctor output, print the full snapshot: `McpReadiness` and its entries implement `Debug`. `mcp_discovery_statuses` remains as the short status list for eager and revealed servers. ## Budgets, MCP credentials and observers ```rust use std::time::Duration; use everruns::hosting::services::{ HostEvent, HostEventListener, HostServices, LlmRetryConfig, McpAuthProvider, McpAuthRequest, McpAuthResult, McpCredential, async_trait, }; use everruns::Engine; struct Vault; #[async_trait] impl McpAuthProvider for Vault { async fn authorization( &self, request: &McpAuthRequest<'_>, ) -> McpAuthResult> { // Called before every discovery and tool call: return the current token. Ok(Some(McpCredential::bearer(format!("token-for-{}", request.server_name)))) } } struct Audit; #[async_trait] impl HostEventListener for Audit { async fn on_event(&self, event: &HostEvent) { println!("{} {:?}", event.event_type, event.sequence); } } let services = HostServices::new() .provider_retry(LlmRetryConfig { max_retries: 4, ..LlmRetryConfig::default() }) .provider_stall_timeout(Duration::from_secs(45)) .mcp_auth(Vault); let engine = Engine::builder().host_listener(Audit).build(); ``` Host listeners receive the same committed canonical envelopes as `EngineBuilder::listener`, in Core event form, on their own bounded queue. A slow, full or failing listener cannot fail a turn or delay other listeners, and nothing a listener does is written to session history. Drops are counted in `Engine::observer_stats`. `McpAuthProvider` requires the `mcp` feature. `HostServices::egress` replaces the Framework’s direct, policy-aware transport for the Agent’s runtime traffic with the host’s own `EgressService`: a proxy the host already routes through, or a test host that answers a stub server in process. The MCP transport still validates and pins a server’s address before its request reaches the egress, so an address that is private or does not resolve is refused either way. For outbound MCP OAuth login, `everruns_core::mcp::oauth::prepare_login_with_options` accepts `LoginOptions` with an optional `client_metadata_url`. Publish your public-client metadata at that HTTPS URL first, including the exact URL as `client_id`, `client_name`, your `redirect_uris`, and `token_endpoint_auth_method: "none"`. The host owns document hosting and must ensure its callback is covered by the document. The helper prefers a supplied configured or saved client, then the metadata URL when the authorization server advertises `client_id_metadata_document_supported`, then dynamic registration. Keep saved registrations scoped to their issuing authorization server. The client does not fetch the document; the authorization server does. The existing `prepare_login` API retains its configured-client/DCR behavior. Both APIs use the same PKCE, callback validation and token exchange. ## Related * [Advanced capabilities](https://everruns.com/docs/framework/advanced-capabilities/) for typed capability packages that need no Core contract. * [Session controls](https://everruns.com/docs/framework/session-controls/) to hot-install, activate and deactivate capabilities and run their commands on a live session. * [Session hosting](https://everruns.com/docs/framework/hosting/) for durable session authority under Engine. * [Custom backends](https://everruns.com/docs/framework/custom-backends/) when the host itself replaces execution components. --- # Pluggable session hosting > Implement durable hosting beneath Engine without implementing an agent runtime. Source: Everruns owns agent execution. A host supplies a coherent session authority through `everruns::hosting::SessionHosting` and a claimed write capability through `SessionEpoch`. Applications define ordinary `Agent` values and call the same `Engine` and `Session` API regardless of the hosting implementation. Without hosting configuration, `Engine::builder().build()` uses in-memory hosting. An embedded application needs no database, Durable Object or external hosting service. Its sessions live for the lifetime of that process. Durable hosting is an opt-in. The hosting SPI is experimental. Framework has no dependency on a Cloudflare or celld adapter. [Durable Object examples](https://everruns.com/docs/framework/durable-objects/) demonstrate how an external implementation can use this boundary; they are source-only references, outside the product publish set. ## Responsibilities | Framework Engine | Hosting implementation | Application/deployment | | --------------------------------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------ | | Planning, shared actor recovery and tool execution | Immutable session binding and transactional commands | Trusted Agent/build registry | | Lifecycle hooks and finalized outcomes | Deduplicated input receipts and canonical journal | Retained code and Environment factories | | Session command rules, approval/client wait interpretation and context projection | Exclusive epochs, fenced storage and bounded snapshot reads | Workspace, compute, secrets and provider transport | | Execution entry through `Engine::activate` | Independent wakeups, expiry discovery and activation retries | Authentication, authorization and operational recovery | A hosting implementation must commit input, receipt and wakeup obligation before acknowledging admission. Every execution mutation checks the same unexpired epoch inside the authority transaction. Frontend readers reject mutations. Durable hosts arrange activation independently of frontend connections and process-local queues. An alarm or supervisor calls `Engine::activate(session_id)`; it never implements a second planner or turn loop. Registered builds and Environment references are immutable. Missing retained code must produce visible suspension instead of falling back to the latest Agent. A durable journal does not make tools, workspace files or external effects durable; those services retain their own guarantees and idempotency requirements. ## SQLite per session For a local code-registered host, enable `local` and open a session directory: ```rust use everruns::{SessionId, hosting::SqliteHosting}; fn main() -> Result<(), Box> { let _hosting = SqliteHosting::open_session("sessions", SessionId::new())?; Ok(()) } ``` Each session, including a child, owns `//engine.sqlite`. The parent retains its child registry; commands, history, task reports and schedules route to the session that owns them. Run `SqliteHosting::supervise` at startup to recover the root and its registered descendants. Unrelated sessions have independent SQLite writer locks. The host still supplies registered builds, Environments and stable secret keys. Keep the directory, its WAL files and encryption keys together when backing up. To transfer an older shared authority or session-directory layout, use `open_session_from` with the host’s exclusive source-session lease callback. It acquires leases for the complete source family before copying any database, including ancestors and siblings when a child opens first, so late task reports survive handoff. Each transfer copies canonical SQLite state and publishes a complete private database atomically. Live epochs and shared maintenance state refuse transfer; completed session databases reopen without consulting the old source. The source is retained, and transferred sessions must use the updated host. Use `open_session_from_sources` for multiple earlier layouts in priority order; existing destination databases win. Directory transfers use SQLite backup so committed WAL frames and child registrations survive without a file-copy race. For offline history inspection, `SqliteHosting::visit_session_events` streams the full raw journal, including abandoned branches, from a read-only SQLite snapshot. It does not claim execution or recover pending commands and can read while a different connection holds a writer lock. Hosts can search this journal directly without continuously rebuilding a JSONL export. The directory profile does not publish shared agent definitions. Use a host control plane for shared definitions and organization-wide policy. The original `SqliteHosting::open` remains available when a host deliberately chooses one shared transaction domain. ## Waiting without keeping execution alive Engine stores safe provider retry deadlines and budgets in the session authority, then releases execution. The host wakes the same session when its deadline is due; an early wake sends no model request, and cancellation can interrupt the wait. Retry limits survive restarting Engine. An explicit provider policy that disables retries remains disabled. If a stream fails after answer text, a safe retry keeps the message ID and emits `output.message.started` again before parking. Clients replace that attempt’s partial text. A trailing stream error keeps the partial answer when no retry fits the remaining budget. Tool calls and provider-executed effects retain their existing replay restrictions. `HostingStatus::wake()` reports whether compute is needed now, at a deadline, or only after an external event. Hosts deliver wakeups; Engine retains execution and recovery policy. Default memory hosting uses its local scheduler and needs no external backend. This does not suspend arbitrary running tools or make external effects safe to repeat. The reference Workers SSE transport uses hibernating WebSocket notifications to read committed events when they change. Its frontend heartbeat does not poll an idle session object. Clients that perform their own polling still generate object activity. ## Implementing a host Depend on `everruns` with only the features the placement needs. The supported storage/ownership paths are under `everruns::hosting::spi`: canonical events and paging, actor leases/journal access, execution views, compaction checkpoints and session values. `SessionHosting` and `SessionEpoch` describe the semantic contract. Host implementations need no direct `everruns-core` or `everruns-contracts` dependency, and do not construct the Engine runtime. The host’s required `command` method runs the shared Rust command implementation inside its authority transaction. Capture a bounded `spi::session_commands::CommandView`, call `spi::session_commands::prepare`, and apply its complete plan before releasing the transaction or revision guard. This keeps binding checks, command fingerprints, approval decisions, pending-call validation and partial-wait behavior in Everruns. The host translates the resulting journal, value, receipt and activation writes into its own storage operations. It does not reproduce those decisions. For a remote authority, send the semantic command to that authority. Run the Rust implementation there, against its locked canonical state and clock; preparing a plan in an executor and sending its effects creates a validation/commit race. The object reference compiles this same command implementation to Wasm and invokes it inside the object’s storage transaction, including when Engine runs on a native executor. JavaScript supplies transaction and wakeup plumbing. `SessionEpoch::backends()` supplies the low-level storage ports that Engine uses internally. `HostBackends` is an implementation detail of writing a hosting adapter, not Agent configuration. Its in-memory defaults are appropriate for reconstructed definitions and explicitly volatile services. Replace every store whose state your durability claim covers with the authority’s fenced version. Selecting individual stores through legacy Agent configuration is a separate, weaker embedding path and does not establish coherent durable execution. ## Qualifying a host Enable `hosting-conformance` in the host crate’s **dev-dependencies**. The native suite runs against a fresh test authority using the public Framework SPI: ```rust async fn qualify(host: impl everruns::hosting::SessionHosting + Clone) { everruns::hosting::conformance::authority(&host).await; everruns::hosting::conformance::lease_expiry(&host).await; everruns::hosting::conformance::engine_wait_recovery(host).await; } ``` The scenarios check immutable bindings, admission retries/conflicts, bounded snapshot paging, read-only frontend ports, exclusive epochs, lease expiry, stale writes, Engine reconstruction, mixed approval/client waits and atomic retry handling. The same suite runs against built-in memory/SQLite hosts and the independently resolved DO reference. The feature is native test tooling; it does not add a simulated provider to Wasm deployments or certify storage durability. Also run deployment-specific failure tests: kill the executor or isolate after a committed effect, allow no frontend requests during recovery, and require the original admission to finish through an independent wakeup. Test lease expiry, platform owner loss and retained-build failure according to the claimed failure domain. The reference includes workerd, isolate and native-process harnesses; local emulators alone do not establish production platform guarantees. ## Framework, serve and the platform server Framework applications and serve both enter `everruns::Engine`; selecting a host does not change their Agent definitions. A DO application is the same Engine and Agent compiled to Wasm, with an object host and host-compatible tools. The celld example supplies that hosting integration, or a native executor when tools need processes and files. The full Everruns platform server has its own control plane and PostgreSQL actor host. It consumes the same Core actor driver directly, shares the Rust command validation for client results and the approval/question policies, and retains its platform authorization and durable resolution claims. Supporting external hosts does not require routing the whole platform through the Framework facade. ## Deployments and retained behavior Keep three identities separate: | Identity | Changes when | Persisted binding | | -------------------------- | ----------------------------------------------------- | ------------------------ | | Deployment revision | Worker/container is rebuilt or redeployed | Deployment metadata only | | `AgentBuild` | Agent behavior or its immutable configuration changes | Exact Agent registration | | `SessionBinding.framework` | Persisted runtime format becomes incompatible | Runtime format version | A compatible Engine upgrade can serve several retained Agent definitions in one binary. Register the original definitions alongside the new one; an old key must still construct its original tools, hooks, Harness, Environment and model settings. Do not alias old keys to the current Agent. Retaining a frontend Worker revision alone does not retain the implementation running inside its Durable Objects. serve’s default durable build identity conservatively includes its executable. It does not automatically declare rebuilt behavior compatible. Hosts can supply an application-owned Engine with retained registrations through `everruns_serve::ServerBuilder::engine`; each old registration still needs its original Agent, Harness and Environment reference. `Server::engine()` exposes that same Engine for authorized compatibility inspection and retry operations. ```rust use everruns::{Agent, Engine}; use everruns::hosting::AgentBuild; fn deployment(original: Agent, current: Agent) -> Engine { Engine::builder() .register(AgentBuild::new("assistant", "v1"), original) .register(AgentBuild::new("assistant", "v2"), current) .build() } ``` Use `engine.session_compatibility(id).await?` to inspect the required build, unsupported runtime version or missing immutable configuration without starting execution. History and accepted receipts remain readable when code is missing. Activation suspends before running an unsupported format or substituting behavior. After restoring the exact registration and a compatible Engine, an authorized host can call `engine.retry_session(id, stable_command_id).await?`. This checks compatibility, then atomically checks the unchanged binding and absence of an active epoch, retains the retry receipt and restores the activation obligation. Pending tools, approvals, waits, history and accepted inputs retain their identities. The command refuses suspensions caused by uncertain effects: those require their own reconciliation policy. This privileged method needs application authorization. The framework does not provide an in-place build migration. A migration would need a separate compatibility and fencing protocol covering active execution, accepted inputs and outstanding effects. Overwriting a build ID or copying only completed messages does not preserve interrupted execution. --- # Framework integrations > Every integration a Framework app can use, the Cargo feature that turns it on, and where its documentation lives. Source: These integrations run inside your own Framework application. Turn on the listed Cargo feature (on the `everruns` crate unless the row names another), then add the capability to the agent. Turning a feature on compiles the integration in, but no agent gets it until you add the capability. Feature and boundary details are in [Capability integrations](https://everruns.com/docs/framework/capability-integrations/). ```toml [dependencies] everruns = { version = "0.52", features = ["openai", "web-search", "web-fetch"] } ``` ## Search and web | Integration | Cargo feature | Docs | | ----------------------------------------------------------------------- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Web search: Brave, Parallel, provider-native search, or a free fallback | `web-search` | [Web Search](https://everruns.com/docs/capabilities/web-search/#framework) | | Brave Search as its own typed capability | `everruns-integrations` crate, `brave-search` feature | [Provider integrations](https://everruns.com/docs/framework/capability-integrations/#provider-integrations) | | DuckDuckGo instant answers | `duckduckgo` | [DuckDuckGo](https://everruns.com/docs/integrations/duckduckgo/) | | Web fetch | `web-fetch` | [Web Fetch](https://everruns.com/docs/capabilities/web-fetch/) | ## Code and files | Integration | Cargo feature | Docs | | ------------------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------- | | Session filesystem | `filesystem` (default) | [File System](https://everruns.com/docs/capabilities/file-system/) | | Bashkit sandboxed shell | `bashkit` | [Bashkit Shell](https://everruns.com/docs/capabilities/bashkit-shell/) | | Shell on the host machine | `host-shell` | [Host Shell](https://everruns.com/docs/capabilities/host-shell/) | | Lua sandbox | `lua` | [Capability integrations](https://everruns.com/docs/framework/capability-integrations/#framework-features) | ## Models, media and judgment | Integration | Cargo feature | Docs | | ------------------------------------------------------------------------ | ------------------------ | ------------------------------------------------------------------------------------ | | Model providers: OpenAI, Anthropic, Gemini, Bedrock, OpenRouter and more | one feature per provider | [Models and Providers](https://everruns.com/docs/framework/models-and-providers/) | | Image generation: stills, GIFs and SVGs | `images` | [Image Generation](https://everruns.com/docs/capabilities/image-generation/) | | TypeSafe typed decisions | `typesafe` | [TypeSafe](https://everruns.com/docs/integrations/typesafe/#embedding-the-framework) | ## Tools and other agents | Integration | Cargo feature | Docs | | --------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------- | | MCP servers over HTTP | `mcp` | [MCP and plugins](https://everruns.com/docs/framework/agents/#mcp-and-plugins) | | Local-process (stdio) MCP servers | `mcp-stdio` | [MCP and plugins](https://everruns.com/docs/framework/agents/#mcp-and-plugins) | | Delegate to remote A2A agents | `a2a` | [Call remote A2A agents](https://everruns.com/docs/framework/a2a/#call-remote-a2a-agents) | | Personal agent (Poppy) | `poppy` | [Poppy in the Framework](https://everruns.com/docs/capabilities/poppy-personal-agent/#in-the-framework) | ## Put an agent in front of people | Integration | Cargo feature | Docs | | ----------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------- | | Serve over A2A | `everruns-serve` crate, `a2a` feature | [Serve your agents over A2A](https://everruns.com/docs/framework/a2a/#serve-your-agents-over-a2a) | | AG-UI frontends | `ag-ui`, `ag-ui-axum` | [Serve AG-UI](https://everruns.com/docs/framework/ag-ui/) | | Slack and webhook channels | `channels` | [`everruns::channels` API docs](https://docs.rs/everruns/latest/everruns/channels/) | | Voice through a speech-to-speech provider | `voice` | [Build a Voice Agent](https://everruns.com/docs/framework/voice/) | ## Observability | Integration | Cargo feature | Docs | | ------------- | ------------- | --------------------------------------------------------------------------------- | | OpenTelemetry | `otel` | [OpenTelemetry](https://everruns.com/docs/framework/observability/#opentelemetry) | | Braintrust | `braintrust` | [Braintrust](https://everruns.com/docs/framework/observability/#braintrust) | ## Platform only The integrations below run in a Platform deployment, where the server holds the connection and its credentials: cloud sandboxes (Daytona, E2B, Container Sandbox, Modal, boxd), Browserless, Cursor, GitHub, Cloudflare Web Search, the Parallel tool set, the hosted Slack app, AgentID and ARD. A Framework app reaches Parallel through [web search](https://everruns.com/docs/capabilities/web-search/#framework). To use any of the others, [move to the Platform](https://everruns.com/docs/framework/moving-to-platform/). The full catalog is in [Integrations](https://everruns.com/docs/integrations/). --- # Lifecycle Hooks > Run typed application handlers at agent, turn, tool, and completion boundaries. Source: Lifecycle hooks run trusted application code at defined execution boundaries. Register them on `Agent::builder()` when work must finish before execution can continue, or when an application needs a typed failure from a lifecycle action. ```rust use everruns::prelude::*; let agent = Agent::builder() .instructions("You are concise.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .on_agent_start(|context| async move { println!("starting {}", context.session_id); }) .on_turn_start(|context| async move { if context.input.content.is_empty() { Err("empty input") } else { Ok(()) } }) .on_completion(|context| async move { println!("turn stopped with {:?}", context.turn.stop_reason); }) .build()?; ``` Handlers are async `Fn` closures. An infallible handler returns `()`; a fallible handler returns `Result<(), E>` where `E` implements `Display`. Wrap synchronous work in an async block, such as `|context| async move { record(context) }`. ## Hooks or events? Hooks and [session events](https://everruns.com/docs/framework/events-and-cancellation/) serve different jobs. | | Lifecycle hooks | Session events | | --------------- | ---------------------------------------------- | -------------------------------------------- | | Purpose | Extend execution with application behavior | Observe execution for UI, telemetry, or logs | | Delivery | Awaited at a lifecycle boundary | Non-blocking stream from `Session::events()` | | Effect on a run | A pre-effect error may prevent its scoped work | Never changes or delays a run | | Registration | `AgentBuilder::on_*` | Subscribe on each `Session` | Do not register a hook merely to mirror the event feed. Use hooks when ordering or failure semantics matter; use events for observation. ## Lifecycle points Handlers at one lifecycle point run sequentially in builder registration order. | Builder method | Runs | Error behavior | | ---------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | | `on_agent_start` | Before the first turn attempted by each session | The first error returns `RunError::Hook`; the next run retries the complete chain | | `on_turn_start` | Before every turn enters the runtime | The first error returns `RunError::Hook` and prevents that turn | | `on_tool_start` | Before a model-requested tool call executes | The first error blocks only that call, skips later start handlers for it, and records a `HookFailure` | | `on_tool_end` | After a tool call reaches a terminal result, including a blocked call | Errors are isolated, recorded, and do not skip later end handlers | | `on_completion` | After a non-cancelled runtime turn reaches a terminal outcome | Errors are isolated, recorded, and do not skip later completion handlers | `Session::inspect()` may materialize a runtime but invokes no lifecycle handler. A successful agent-start chain runs once for that session. If it fails or is cancelled, the next run starts the complete chain again, so external agent-start effects should be idempotent. Tool-start runs after any earlier execution gates configured by the host. Tool-end runs for every call that reaches a terminal result, including a call blocked by an earlier gate; in that case the Framework tool-start handler might not have run. Independent calls in a parallel tool batch can run their hook chains concurrently. Completion receives terminal `Turn` values whether `turn.success` is true or false. A runtime error that produces no `Turn`, and a cancelled in-flight turn, do not run completion handlers. Every completion handler receives the same pre-completion `CompletionContext` snapshot. ## Failures and execution effects Hook contexts are owned, read-only snapshots. A hook cannot rewrite input, tool arguments, tool results, or the returned turn. Errors affect execution only where work has not happened: * agent-start and turn-start errors prevent the turn and return `RunError::Hook`; * tool-start errors prevent that one tool call and appear in `Turn::hook_failures`; * tool-end and completion errors cannot roll back completed work, so they are isolated in `Turn::hook_failures` and the remaining handlers still run. Each `HookFailure` identifies the lifecycle point and its zero-based handler index. Tool failures also identify the tool and call. A tool-start error shown to the model is deliberately generic; the handler’s detailed message remains application-facing on `HookFailure`. With no registered hooks, execution behavior is unchanged and `Turn::hook_failures` is empty. ## Cancellation, concurrency, and panics A token cancelled before `run_with` skips all handlers. Cancellation during agent-start, turn-start, or an in-flight tool chain drops the active handler future, skips the remaining turn work, and does not run completion. Side effects that finished before cancellation are not rolled back. The synthesized cancelled `Turn` does not report partial failures from the dropped in-flight hook chain. Once the runtime commits a turn, completion handlers finish in order even if that run token is then cancelled. This makes post-turn delivery predictable. The same `Fn` handlers are shared by every session. Separate sessions, and separate calls in a parallel tool batch, may invoke a handler concurrently. Protect shared mutable state inside the closure and do not depend on failure ordering across parallel tool calls. The Framework adds no hook timeout and does not catch panics. Apply an application timeout inside a handler when external work must be bounded; let ordinary Rust panic behavior handle programming defects. ## Sensitive data Lifecycle handlers are trusted in-process application code. Turn and tool contexts can contain user input, model-selected arguments, tool results, or backend error text. Do not log or export whole contexts without applying the same redaction and access controls as the underlying data. ## Runnable example The focused [`lifecycle_hooks.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/lifecycle_hooks.rs) example registers all five lifecycle points around a typed tool: ```bash cargo run -p everruns --features openai --example lifecycle_hooks ``` It uses `gpt-6.2-sol` and requires `OPENAI_API_KEY`. --- # MCP authentication > Bind application credentials or an OAuth token store to an exact MCP endpoint, with shared PKCE login and token refresh. Source: Enable `mcp-auth` to use `everruns::mcp::auth`. The MCP transport, OAuth discovery, PKCE exchange, resource/issuer checks, refresh and token rotation are shared with Server. Your application owns login navigation, the callback route, and secret storage. Server uses the same protocol client with its organization-scoped network policy, encrypted records and cross-replica refresh locks; these host adapters remain separate. See the [implementation-sharing matrix](https://everruns.com/docs/framework/service-integrations/#shared-implementations-and-host-boundaries). ## Bearer credentials ```rust use everruns::{Agent, McpServer, Model, mcp::auth::McpAuth}; let url = "https://mcp.example.com/mcp"; let auth = McpAuth::new().bearer("docs", url, std::env::var("DOCS_MCP_TOKEN")?); let agent = Agent::builder() .instructions("Consult the docs server before answering.") .model(Model::new("gpt-6-luna", everruns::OpenAI::from_env()?)) .mcp_server(McpServer::http("docs", url)) .mcp_auth(auth) .build()?; ``` Names and endpoints must match at build time and again when credentials are requested. Replacing a live MCP endpoint cannot redirect its existing credential. Unbound servers get no credential from this value. For rotating keys, implement `Credentials::authorization` and use `.credentials(name, url, provider)`. Acquisition happens for each operation; provider errors are sanitized. ## OAuth `OAuthLogin::begin` discovers metadata and prepares a PKCE login. Use `begin_with_client` when the authorization server requires a pre-registered public client. Its redirect URI must be permitted by that server. Open `authorization_url()` in the user’s browser, retain the login in the same trusted application, and give `complete(code, state, issuer)` the callback’s values. `parse_callback_query` rejects duplicate parameters and OAuth error callbacks. Completion consumes the prepared login and validates state, issuer, PKCE and resource binding. Save the returned `OAuthTokens` into an application-owned secret store implementing `TokenStore`. Bind it with `McpAuth::oauth(name, url, store)`. Refreshed and rotated grants are saved through the same store. A production store must protect access/refresh tokens and coordinate concurrent writers; store serialization is never agent configuration or event history. `MemoryTokenStore` is explicitly process-local. Outstanding logins and grants do not survive a process restart unless your application persists them securely. The [complete OAuth example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/mcp_oauth.rs) performs login, accepts a callback URL on stdin, stores the grant in memory, and runs a real agent against the authenticated MCP server: ```bash cargo run -p everruns --features mcp-auth,openai --example mcp_oauth -- \ https://MCP_HOST/mcp http://localhost:8080/callback ``` Treat login/callback URLs as credentials. Do not place them in logs, chat messages, or analytics. Discovery tries the MCP endpoint’s path-specific protected-resource document, then the origin’s root document. Authorization-server discovery supports issuer paths and OpenID fallbacks, and rejects an advertised issuer that differs from the selected issuer. Callback issuer mismatches are rejected before code exchange. Shared HTTP client errors expose recognized OAuth codes such as `invalid_grant`, while provider descriptions, response bodies and transport URLs are omitted. Server presets with manually configured OAuth endpoints must also configure the issuer when the provider returns callback `iss`. Automatic discovery records it. A callback issuer without a trusted configured or discovered issuer is rejected; signing in again without correcting the preset cannot establish that trust. Providers that omit `iss` remain supported. --- # Models and Providers > Select model identities, attach provider drivers and credentials, list provider catalogs, and write a custom provider. Source: The Framework separates **what model to use** from **how to reach it**: * `.model("id")` selects the provider-visible model with a credential-free string. * `Provider` supplies the driver, endpoint, and authentication needed by the host. * An agent currently accepts one provider, configured separately with `.provider(...)`. This boundary is open: a new provider does not require a new closed enum variant or provider-specific branch in application code. The Framework constructs its execution-facing model specification internally when the agent builds. This page covers the provider types the `everruns` crate bundles, the drivers that ship today, the environment variables each driver reads, provider model catalogs, and writing your own driver. To call a model once without building an agent, see [Direct model calls](https://everruns.com/docs/framework/direct-model-calls/). ## Provider configuration Each production provider has a typed configuration under `everruns::providers`, also re-exported at the crate root. Enable its Cargo feature. Configuration owns credentials, endpoint and protocol; `Model` and `LlmCallConfig` own the model and per-call generation settings. Pass a configuration to `.provider(...)`, or convert it into `Provider` with `.into()` for direct calls. | Feature | Configuration | Credential setup | | ------------ | ------------- | ----------------------------------------------------------------------- | | `openai` | `OpenAI` | `new(key)` or `from_env()` | | `openai` | `AzureOpenAI` | `new(versioned_url, key)` or `from_env(versioned_url)` | | `anthropic` | `Anthropic` | `new(key)` or `from_env()` | | `gemini` | `Gemini` | `new(key)` or `from_env()` | | `openrouter` | `OpenRouter` | `new(key)` or `from_env()` | | `mistral` | `Mistral` | `new(key)` or `from_env()` | | `groq` | `Groq` | `new(key)` or `from_env()` | | `fireworks` | `Fireworks` | `new(key)` or `from_env()`; `.batch(config)` enables native batches | | `meta` | `Meta` | `new(key)` or `from_env()` | | `vercel` | `Vercel` | `new(key_or_oidc_token)` or `from_env()` | | `typesafe` | `TypeSafe` | `new(key)` or `from_env()`; Decisions service only | | `mai` | `Mai` | `new(url, MaiAuth)` or `from_env()`; API key or Entra auth | | `cloudflare` | `Cloudflare` | `new(account, token)` or `from_env()`; `.gateway(id)` selects a gateway | | `bedrock` | `Bedrock` | `new(access, secret, region)` or `default_chain()` | | `chatgpt` | `ChatGpt` | `new(auth)` with host-managed OAuth and refresh | | `codex` | `Codex` | `new(auth)` with host-managed legacy Codex authentication | HTTP wrappers offer `.base_url(...)`. Bedrock uses `.region(...)` instead. ChatGPT and Codex intentionally have no environment credential loader: the host owns login and token storage. Azure keeps its versioned resource URL explicit. The offline test double remains `Model::simulated(...)`. OpenAI and Mistral expose the same protocol selection method with provider-owned choice types: ```rust use everruns::{OpenAI, OpenAIApi, Mistral, MistralApi}; let openai = OpenAI::from_env()?.api(OpenAIApi::ChatCompletions); let mistral = Mistral::from_env()?.api(MistralApi::Responses); ``` Without `.api(...)`, OpenAI uses Responses and Mistral uses Chat Completions. The selection affects chat only; OpenAI’s other services remain available. Providers with one supported chat API need no selector. No provider-specific extension methods are added to shared call configuration. Hosted provider create/update requests use the same connection setting: ```json { "name": "OpenAI Chat Completions", "provider_type": "openai", "api": "chat_completions" } ``` Supply credentials through the normal `api_key` or `credentials` fields. `GET /v1/providers/config` advertises each driver’s supported `apis`; unsupported choices return a validation error. Omit `api` on creation to use the driver default, omit it on PATCH to keep the current choice, or PATCH `"api": "default"` to reset it. The setting follows the connection through hosted workers and does not change other model services. Upgrade both server and workers before using the selector. The Settings UI does not expose the selector yet. ## OpenAI convenience With the `openai` feature, `OpenAI::from_env` reads `OPENAI_API_KEY` and the optional `OPENAI_BASE_URL`: ```rust use everruns::{Agent, OpenAI}; let agent = Agent::builder() .instructions("Be concise.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; ``` Use `OpenAI::new(key)` when the host already owns an explicitly resolved credential. Never put credentials in a model id, log them as model identity, or select provider behavior with vendor-specific detection. The `anthropic` and `gemini` features add `Anthropic` and `Gemini` with the same shape, and `openrouter` adds `OpenRouter`: | Feature | Type | Reads | | ------------ | ------------ | ------------------------------------------------------- | | `openai` | `OpenAI` | `OPENAI_API_KEY`, `OPENAI_BASE_URL` | | `anthropic` | `Anthropic` | `ANTHROPIC_API_KEY` | | `gemini` | `Gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY`, `GEMINI_BASE_URL` | | `openrouter` | `OpenRouter` | `OPENROUTER_API_KEY`, `OPENROUTER_BASE_URL` | The `bedrock` feature adds `Bedrock`, which takes either static keys or the AWS default credential chain, so a process with an IAM role (an ECS task, an EC2 instance profile, an Amazon Bedrock AgentCore Runtime execution role) needs no keys. The model id is a Bedrock model id or inference profile: ```rust use everruns::{Model, providers::bedrock::Bedrock}; // Static keys. let model = Model::new( "us.anthropic.claude-sonnet-4-6", Bedrock::new("AKIA...", "secret", "us-east-1"), ); // The AWS default chain. Region: `.region(..)`, else AWS_REGION, else // AWS_DEFAULT_REGION, else us-east-1. let model = Model::new( "us.anthropic.claude-sonnet-4-6", Bedrock::default_chain().region("us-east-1"), ); ``` `from_env` is not OpenAI-specific: every driver declares the variables its own vendor SDK reads, and each driver crate exposes the same entry point. See [Credentials](#credentials) for the per-driver table. Applications with their own driver pass `Provider::new(key, driver)` to `.provider(...)`; see [Custom providers](#custom-providers). ## Simulated models, for tests ```rust use everruns::{Agent, Model}; let agent = Agent::builder() .instructions("Answer deterministically.") .model(Model::simulated("fixed response")) .build()?; ``` `Model::simulated` is backed by the focused `everruns-llmsim` crate. It is a **test double that runs no inference** — it replays canned responses so tests can assert on agent behavior without a network call or an API key. It is not a local model and not a way to run Everruns without a provider. Depend on the crate directly when building a low-level host or scripting multi-turn provider behavior; ordinary Framework applications need only `everruns`. It registers as a driver, but it reaches no network. For real work, pick a provider from [Supported providers](#supported-providers). ## Supported providers Everruns talks to model vendors through **drivers**. A driver owns one vendor’s wire protocol; a `Provider` pairs typed service drivers with one endpoint and authentication boundary. The set below is what ships today. The boundary is open, so a [custom driver](#custom-providers) is a first-class peer of these. ### Drivers | Driver | Crate | Wire protocol | Services | Model discovery | | ------------------------- | --------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------- | --------------------------------------------- | | OpenAI | `everruns-drivers` (`openai`) | OpenAI Responses, OpenAI Decisions API, OpenAI Images | chat, embeddings, realtime, decisions, images | yes | | ChatGPT Plan | `everruns-drivers` (`chatgpt`) | Stateless Responses + open-source OAuth | chat | account-visible models | | OpenAI (Chat Completions) | `everruns-drivers` (`openai`) | OpenAI Chat Completions | chat | yes | | Azure OpenAI | `everruns-drivers` (`openai`) | OpenAI Responses, OpenAI Images | chat, images | yes | | Anthropic | `everruns-drivers` (`anthropic`) | Anthropic Messages | chat | yes | | Google Gemini | `everruns-drivers` (`gemini`) | Gemini `generateContent`, Decisions API | chat, decisions | yes | | AWS Bedrock | `everruns-drivers` (`bedrock`) | Bedrock `ConverseStream` (SigV4) | chat | no | | OpenRouter | `everruns-drivers` (`openrouter`) | OpenAI Responses-compatible, System One, OpenRouter Images, native Extractions | chat, decisions, images, extractions | yes, including decision and extraction models | | TypeSafe | `everruns-drivers` (`typesafe`) | System One | decisions | yes, plus curated Jev profiles | | Microsoft MAI | `everruns-drivers` (`mai`) | OpenAI Chat Completions, System One (Azure AI Foundry) | chat, decisions | yes | | Fireworks AI | `everruns-drivers` (`fireworks`) | OpenAI Chat Completions-compatible | chat | yes | | Meta | `everruns-drivers` (`meta`) | OpenAI Responses-compatible, OpenAI-compatible Images | chat, images | yes | | Mistral AI | `everruns-drivers` (`mistral`) | Chat Completions; connection-selected stateless Responses | chat | yes | | Cloudflare AI Gateway | `everruns-drivers` (`cloudflare`) | OpenAI Chat Completions-compatible, Workers AI Clairic | chat, extractions | Workers AI only | | Vercel AI Gateway | `everruns-drivers` (`vercel`) | Open Responses | chat | yes | | LLM Simulator | `everruns-llmsim` | none — in-process test double | chat, realtime (simulated) | no | Every chat driver produces an incremental stream — server-sent events for the HTTP protocols, `ConverseStream` for Bedrock — so token-by-token output works everywhere, not just on one vendor. Tool calling and multi-turn tool results work across all of them: each driver normalizes its vendor’s shape into the same typed events, which is why swapping a provider does not change application code. The environment variables each API driver reads are in [Credentials](#credentials). ChatGPT Plan connections use host-owned OAuth credentials; see [ChatGPT Plan](https://everruns.com/docs/features/chatgpt/). The optional `codex` SDK feature provides the legacy Codex backend for hosts such as Yolop. The Platform UI does not offer ChatGPT Plan because Everruns does not have a registered web OAuth client and callback for that flow. ### Beyond chat Some drivers offer services beyond chat. The table above is the supported service inventory; each service has its own input and output contract: **Embeddings.** The OpenAI driver powers embedding models for knowledge-base retrieval alongside its chat models. **Realtime voice (WebRTC + WebSocket).** A realtime voice session is negotiated by the platform server, not the Framework. The browser posts its SDP offer to `POST /v1/sessions/{session_id}/voice/calls` and the server answers it; a separate route mints a short-lived client secret from the vendor. The organization’s own API key is used only server-side and never reaches the browser. The server then opens a WebSocket sideband (`wss://…/realtime?call_id=…`) to drive the call and collect transcripts, which land in the session as ordinary events — so a voice turn and a typed turn are the same session, readable through the same history and event streams. This needs the platform server; an embedded Framework process does not expose it. **Images.** OpenAI, Azure OpenAI, OpenRouter and Meta also serve image models (GPT Image 2.6, Muse Image) for the [image generation capability](https://everruns.com/docs/capabilities/image-generation/). **Extractions.** OpenRouter’s native alpha API and direct Cloudflare Workers AI Clairic return typed values with optional confidence and source evidence. Native Rust hosts expose the [Framework Extractions service](https://everruns.com/docs/framework/extractions/); the Server and console use organization-owned provider credentials. This does not add native extraction support to WebAssembly builds. ### Interactive connect OpenRouter declares an OAuth connect flow, so an operator can choose “Connect with OpenRouter” instead of pasting a key. Every other driver takes a credential directly, entered in Settings or supplied in code. ### Choosing one Any OpenAI-compatible gateway that speaks Responses or Chat Completions can usually be reached by pointing the matching driver’s `base_url` at it, rather than writing a driver. Write a [custom driver](#custom-providers) when the vendor’s protocol genuinely differs, or when it needs authentication that a bearer token cannot express. ## Credentials Every provider driver declares the environment variables it reads, on its own descriptor, following **its vendor’s own SDK convention**. There is no Everruns naming scheme to learn: if your shell already runs the `openai` CLI, the AWS CLI, or an Azure service principal, it already configures the matching driver. `OpenAI::from_env()` in the example above reads `OPENAI_API_KEY`, and `OPENAI_BASE_URL` when set. Each driver crate offers the same entry point, returning a ready `Provider`. The facade exposes the typed configurations listed above. AWS Bedrock uses `bedrock`, where `Bedrock::default_chain()` resolves credentials the way the AWS SDK does (an IAM role, SSO or a profile, or the `AWS_*` variables) instead of reading declared names. Application code uses the same typed setup for other vendors: ```rust use everruns::{Agent, Model}; // Reads FIREWORKS_API_KEY, and FIREWORKS_BASE_URL when set. let model = Model::new( "accounts/fireworks/models/llama-v3p1-70b-instruct", everruns::Fireworks::from_env()?, ); ``` ### What each driver declares | Driver | Credential | Endpoint | | ------------------------- | --------------------------------------------------------------------------------------------------------- | ---------------------------- | | OpenAI | `OPENAI_API_KEY` | `OPENAI_BASE_URL` | | OpenAI (Chat Completions) | `OPENAI_API_KEY` | `OPENAI_BASE_URL` | | Azure OpenAI | `AZURE_OPENAI_API_KEY` | — (see below) | | Anthropic | `ANTHROPIC_API_KEY` | — (see below) | | Google Gemini | `GEMINI_API_KEY`, or `GOOGLE_API_KEY` | `GEMINI_BASE_URL` | | OpenRouter | `OPENROUTER_API_KEY` | `OPENROUTER_BASE_URL` | | TypeSafe | `TYPESAFE_API_KEY` | `TYPESAFE_BASE_URL` | | Fireworks AI | `FIREWORKS_API_KEY` | `FIREWORKS_BASE_URL` | | Meta | `LLAMA_API_KEY`, or `META_API_KEY` | `LLAMA_BASE_URL` | | Mistral AI | `MISTRAL_API_KEY` | `MISTRAL_BASE_URL` | | AWS Bedrock | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` (or `AWS_DEFAULT_REGION`), `AWS_SESSION_TOKEN` | — (the region selects it) | | Microsoft MAI | `AZURE_AI_API_KEY`, **or** `AZURE_TENANT_ID` + `AZURE_CLIENT_ID` + `AZURE_CLIENT_SECRET` | `AZURE_AI_ENDPOINT` | | Cloudflare AI Gateway | `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_AI_GATEWAY_ID` | — (derived from the account) | | Vercel AI Gateway | `AI_GATEWAY_API_KEY` | — (one fixed gateway host) | A driver is not limited to one key. Bedrock needs four AWS fields; MAI accepts either a resource key or a full Entra ID service principal. Alternates listed with “or” are variables the vendor itself also honors, tried in the order shown — not a second credential. Anthropic and Azure OpenAI declare no endpoint variable on purpose. A `base_url` here is the *versioned* API root — drivers append bare operation paths to it, and the defaults end in `/v1` — whereas `ANTHROPIC_BASE_URL` and `AZURE_OPENAI_ENDPOINT` name the bare host, because those SDKs add the version segment themselves. Importing either verbatim would resolve to `https://api.anthropic.com/messages` and fail. Point those drivers at a proxy with an explicit `Provider::new(...).base_url(...)`, or through the Settings UI, instead. A credential resolves whole or not at all. If any required variable is missing the driver is simply not configured from the environment, rather than being half-configured into a provider that fails at its first request. A shell carrying `AWS_REGION` but no AWS keys does not configure Bedrock, and a half-populated Entra block does not configure MAI — the same rule the schema applies to an operator-entered form. This table is pinned by a test against the drivers’ own declarations, so it cannot drift from what they read. ### Custom drivers A [custom driver](#custom-providers) declares its variables the same way, on the credential field itself: ```rust use everruns::{CredentialFormSchema, DriverDescriptor, DriverId, FormField}; DriverDescriptor { display_name: "Acme".into(), credential_schema: CredentialFormSchema { fields: vec![ FormField::password("api_key", "API Key") .required() .env("ACME_API_KEY"), ], instructions_markdown: "Create a key in the Acme console.".into(), }, base_url_env: Some("ACME_BASE_URL".into()), ..DriverDescriptor::chat_only(DriverId::external("acme"), factory) } ``` A driver that declares nothing is never configured from the environment, whatever its id is spelled. That is the safe default: the registry’s built-in schema declares no variable, so a driver opts in by naming what its vendor reads. ### Server deployments never read the environment Credential loading is an injected concern, not something a driver does. A driver only *declares* names; the declaration reads nothing and is simply never consulted on the server. `EnvCredentialProvider` is the single place in the workspace that pairs a driver’s declarations with a real environment lookup, and it is for standalone, CLI, and development use. The multitenant server resolves credentials from its encrypted database and constructs no `CredentialProvider` at all. This is the fail-closed Key Resolution Contract: a platform-level key reachable from a shared host environment would silently fund tenant execution. So `OpenAI::from_env`, each driver’s `from_env`, and `EnvCredentialProvider` belong in your own binaries and dev entrypoints. Hosted deployments configure providers through the Settings UI, which renders the same declared schema. ### Resolving credentials yourself To build the provider without going through a driver crate’s `from_env` — a custom `ProviderKey`, or your own credential source — resolve against the descriptor: ```rust use everruns::{CredentialProvider, EnvCredentialProvider, provider_from_env}; let driver = everruns_drivers::anthropic::descriptor(); // What this driver reads, for an error message or a setup check. let names = driver.declared_env_vars(); // The same resolution `from_env` performs. let provider = provider_from_env(&driver, "primary")?; // Or inspect the resolved fields first. if let Some(credentials) = EnvCredentialProvider.resolve(&driver) { let _ = credentials.api_key(); } ``` `ProviderCredentials` carries every declared field, so multi-field drivers stay expressible; `document()` produces the exact credential shape the server stores, which is why an env-resolved credential and an operator-entered one reach the driver through one path. ## Model catalogs Selecting a model needs an exact, provider-visible id. Anything that lets a person choose one — a picker, a `--model` flag, a settings page — needs the catalog behind it: which ids this provider serves, what they are called, and what each one supports. That is the same provider edge an [agent](https://everruns.com/docs/framework/agents/) or a [direct call](https://everruns.com/docs/framework/direct-model-calls/) uses, asked a different question: ```rust use everruns::{OpenAI, models}; for model in models::list(OpenAI::from_env()?).await? { println!("{} — {}", model.id(), model.display_name().unwrap_or("?")); } ``` Ids come back exactly as chat calls expect them, newest first, merged with the model profile registry so a provider that returns bare ids still renders human-readable names and descriptions. `list` is a provider call: it costs a round trip and catalogs change rarely, so cache the result instead of asking per keystroke. ### What each entry carries `ModelInfo` separates the id from everything around it. The id is the provider’s own; the rest is display and capability metadata, absent when neither the provider nor the registry knows it. ```rust use everruns::ModelInfo; fn describe(model: &ModelInfo) -> String { let name = model.display_name().unwrap_or(model.id()); let window = model.context_window().unwrap_or_default(); let tools = if model.supports_tools() { "tools" } else { "no tools" }; format!("{name}: {window} tokens, {tools}") } ``` * `id` — pass back unchanged. * `display_name`, `description` — for rendering. * `vendor` — who trained the model, which is not always who serves it: an aggregator offers many vendors’ models. * `context_window`, `supports_tools`, `supports_reasoning` — the common checks, from the profile registry. * `profile` — the full `ModelProfile` behind those: limits, per-million-token prices, modalities, and capability flags. Capability answers come from curated data, so `false` also covers “not in the registry”. Treat them as display hints rather than guarantees. ### From a selection to a run A selection converts straight back into the `Model` the rest of the API takes, bundled with the provider it was discovered through: ```rust use everruns::{Agent, OpenAI, models}; let catalog = models::list(OpenAI::from_env()?).await?; let picked = catalog .into_iter() .find(|model| model.supports_tools()) .ok_or("no tool-calling model")?; let agent = Agent::builder() .instructions("Be concise.") .model(picked.model()) .build()?; ``` No string handling, and nothing to reconfigure: the model already knows how to be reached. ### Providers without a catalog Not every provider can enumerate its models. That is not a failure of the request, so it is a distinct variant rather than an error to log: ```rust use everruns::{Provider, models}; let ids = match models::list(provider).await { Ok(catalog) => catalog.iter().map(|model| model.id().to_string()).collect(), // Keep the application's curated suggestions. Err(models::CatalogError::NoCatalog) => curated, Err(error) => return Err(error.into()), }; ``` `CatalogError::Call` carries the provider failure verbatim, with the full `LlmError` decision intact. ### Metadata without a provider call The profile registry is static data, so a model’s identity can be read offline: ```rust use everruns::{DriverId, models}; let profile = models::profile(&DriverId::OpenAI, "gpt-6.2-sol"); assert!(profile.is_some()); ``` A `Model` that bundles its provider answers the same question directly with `model.profile()`. A bare model id has no profile: nothing says which vendor’s registry to consult. ### Drivers and the vendor behind a provider A provider’s runtime key is the application’s own name for it, so `Provider::new("my-gateway", ...)` says nothing about which vendor’s models it serves. Declare the driver kind when they differ, and profile lookups resolve against the vendor: ```rust use everruns::{ChatDriver, DriverId, Provider}; let provider = Provider::new("my-gateway", driver) .base_url("https://gateway.example/v1") .with_driver_id(DriverId::OpenAI); ``` Unset, the driver kind falls back to the runtime key, which is the conventional case (`OpenAI::from_env()` and every driver crate’s `from_env` already declare it). A custom driver joins in by implementing `ChatDriver::list_models` and returning `DiscoveredModel` values; returning `None` — the default — is how a driver says it has no catalog. See [Custom providers](#custom-providers). The runnable version of this section is [`model_catalog.rs`](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/model_catalog.rs), which runs offline without an API key. ## WASM and host fetch The Chat Completions and Responses protocol drivers, plus OpenAI and OpenRouter chat drivers, build for `wasm32-unknown-unknown`. Enable `http` on `everruns-contracts`, or `openai` / `openrouter` on `everruns-drivers`. The native socket and TLS dependencies are excluded on that target. Supply an `HttpTransport` backed by your host’s `fetch`. It receives the encoded method, URL, headers and body, and returns the HTTP status, headers and a lazy byte stream. The existing driver still owns provider options, tool replay, request encoding, response parsing, SSE, retry policy and error classification. ```rust use std::sync::Arc; use everruns_contracts::{BearerAuth, Provider}; use everruns_contracts::http_transport::HttpTransport; use everruns_drivers::openrouter::OpenRouterChatDriver; fn provider(api_key: String, host_fetch: Arc) -> Provider { Provider::new( "openrouter", OpenRouterChatDriver::new().with_transport(host_fetch), ) .base_url("https://openrouter.ai/api/v1") .auth(BearerAuth::new(api_key)) } ``` The same `.with_transport(...)` method is available on `OpenAIChatDriver`, `OpenAICompletionsChatDriver`, `OpenAIProtocolChatDriver` and `OpenResponsesProtocolChatDriver`. Native applications may also inject a transport; otherwise they keep the shared HTTP client. WASM requires an adapter and reports an error if none is supplied. The adapter owns endpoint and DNS policy, redirect refusal, network timeouts, and cancellation when a request or response stream is dropped. Treat it as trusted application code: it handles provider credentials. Return response headers before consuming the body and stream bytes incrementally. Host JS futures must be wrapped for the single-threaded WASM `Send` contract. OpenAI WebSocket, realtime, speech, embeddings and decision transports remain native; this support covers HTTP chat. The [Workers/Durable Object fetch adapter](https://github.com/everruns/everruns/blob/main/examples/durable-object-agent/src/fetch.rs) shows exact bytes, manual redirects, cancellation and read deadlines. The [workerd integration tests](https://github.com/everruns/everruns/tree/main/tests/wasm-providers) run model/tool/model exchanges in both a Worker and a Durable Object without reimplementing either provider protocol. ## Custom providers Use a custom provider when an application talks to a model service that the Framework does not configure for you. The extension boundary is the public `ChatDriver` trait plus a `Provider` value. The agent selects that provider’s model with a plain credential-free string id. At a high level: ```rust use everruns::{Agent, BuildError, ChatDriver, Provider}; fn agent_for(driver: impl ChatDriver + 'static) -> Result { Agent::builder() .instructions("Use the company model gateway.") .provider(Provider::new("company-gateway", driver)) .model("assistant-v2") .build() } ``` A driver implements the streaming chat-completion contract. It receives the resolved endpoint, model-facing messages (`everruns::llm::Message`), and call configuration, and returns an `LlmResponseStream`. Exact trait methods and event shapes live in the [`everruns::ChatDriver` API reference](https://docs.rs/everruns/latest/everruns/trait.ChatDriver.html). Keep credential lookup and refresh in trusted host/provider configuration. Model ids must remain safe to log, compare, store, and pass across application boundaries. Provider errors should preserve useful decisions without including secrets. A driver registered through a `DriverDescriptor` also declares which environment variables it reads, on its own credential fields. Declaring is inert — the driver never reads them — and it is what lets a caller resolve the provider from the environment without any central name mapping. See [Credentials](#credentials). Use focused provider crates when they already implement the protocol you need. Custom backends and provider registry topology belong to [low-level host composition](https://everruns.com/docs/framework/custom-backends/), not ordinary model selection. --- # Moving to Platform or Cloud > What carries over when a Framework agent moves to a self-hosted Everruns Platform or the Everruns demo instance, what has to be rebuilt, and what stays behind. Source: The Framework and the Platform run the same runtime, so an agent’s behavior moves with it: instructions, capabilities and their configuration, files, MCP servers, and loop settings all have Platform counterparts. What does not move is code. A Platform agent is configuration stored by the server, so anything your Framework app expresses as Rust (function tools, custom capabilities, lifecycle closures, an approver) is rebuilt with a Platform mechanism instead. The Everruns demo instance runs the Platform for you, as a demo of the open-source code rather than a hosted service. Everything here applies to both, except that Cloud runs the production deployment grade: capabilities marked dev-only, such as [Computer Use](https://everruns.com/docs/capabilities/computer-use/), are not offered there. ## What transfers | Framework | Platform agent field | | --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `.name("support")` | `name`. Must be lowercase letters, digits, and hyphens, unique in the organization | | `.instructions(...)` | `system_prompt` | | `.model("gpt-6.2-sol")` | `default_model_id`, the ID of a model configured for the organization. A model name string is not accepted | | `.capability(...)` | `capabilities`, the same `{ "ref": ..., "config": ... }` shape | | `.initial_file(...)`, `.file(...)`, `.readonly_file(...)` | `initial_files` | | `.mcp_server(...)` for a remote server | `mcpServers` | | `.max_iterations(n)` | `max_iterations` | | `.parallel_tool_calls(b)` | `parallel_tool_calls` | | A `Harness` | A Platform harness (`harness_id` or `harness_name`). `Harness::base()`, `conversation()`, `worker()` and `bashkit_worker()` share the Platform preset definitions; `generic()` is deprecated and preserves the legacy bundle | `CapabilityRef` serializes as `{"ref": id, "config": {...}}`, which is the format of a Platform agent’s `capabilities` entries, so a capability list can be copied across as JSON. Check each ID against the [capability index](https://everruns.com/docs/capabilities/): a few Framework capabilities are Framework-only (for example host shell), and a capability whose page lists only Platform or Cloud adds features the Framework never had. To create the agent, write it as a file and import it with `POST /v1/agents/import`, the CLI, or an SDK. See [Define agents as files](https://everruns.com/docs/how-to/define-agents-as-files/). ```rust use everruns::{Agent, CapabilityRef, OpenAI}; use serde_json::json; let agent = Agent::builder() .name("support") .instructions("Answer support questions.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability("current_time") .capability(CapabilityRef::new("loop_detection").config(json!({ "threshold": 4 }))) .build()?; ``` becomes: ```markdown --- name: support capabilities: - current_time - ref: loop_detection config: threshold: 4 --- Answer support questions. ``` Pick the default model in the Platform after import, from the models the organization has configured. ## What has to be rebuilt | Framework | On the Platform | | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Function tools (`.tool(...)`, `#[everruns::tool]`) | Client-side tools in the agent’s `tools` field, which the model calls and your client executes when it receives `tool.call_requested`; or a remote MCP server that exposes the same tools | | Custom capabilities written with `everruns::capability` | Built-in capabilities, a remote MCP server, or client-side tools. The Platform does not load application code | | Lifecycle hooks (`.on_turn_start`, `.on_tool_start`, …) | [User Hooks](https://everruns.com/docs/capabilities/user-hooks/), which run shell commands at lifecycle and tool events | | `.approver(...)` | [Tool Approval](https://everruns.com/docs/capabilities/tool-approval/), answered by a person in the UI or by your client over the API | | `.ask_user(...)` | [Ask User](https://everruns.com/docs/capabilities/ask-user/), answered by the session’s user | | stdio MCP servers | Not supported. The Platform connects only to remote MCP servers | | `OpenAI::from_env()` and other `from_env` credentials | Provider keys set in the organization’s settings and stored encrypted. Drivers do not read them through `from_env` on the server | [Client-side tools example](https://github.com/everruns/everruns/blob/main/examples/client_side_tools.sh) shows the client-side tool loop over the API. ## What does not transfer * **Sessions and history.** There is no session import. A Framework session’s event log, whether in memory, `SqliteHosting` or a legacy `LocalConfig` data directory, stays with the Framework app. Platform sessions start fresh; a Platform session can be exported (`GET /v1/sessions/{id}/export`) but not the reverse. * **Workspace files.** A local workspace directory is not copied. Put files the agent needs at start in `initial_files`. * **Application wiring.** Event listeners, `Engine` configuration, and your own HTTP layer have no Platform equivalent. Platform sessions are driven through the [API](https://everruns.com/docs/api/), SDKs, CLI, or UI, and observed through its [observability](https://everruns.com/docs/observability/) exporters. ## Keeping both The same capability IDs and configuration work in both places, so an agent can be developed in the Framework with simulated models and tests, then registered on the Platform. `CapabilityRef` implements `Deserialize`, so one JSON file of capability entries can feed both the Framework builder and the Platform agent. ## See also * [Portable and hosted capabilities](https://everruns.com/docs/framework/capability-boundaries/) * [Deploying a Framework app](https://everruns.com/docs/framework/deployment/) * [Define agents as files](https://everruns.com/docs/how-to/define-agents-as-files/) --- # Native services > Bind application-owned credentials to Cursor, Browserless and E2B tools, with optional image generation using the shared native integrations. Source: Enable each integration independently with `cursor`, `browserless`, or `e2b`. These values install the existing native tool implementations on real Engine contexts. They do not require Server connection records or its account-selection UI. ```rust use everruns::{Agent, Model, services::ApiKey}; let agent = Agent::builder() .instructions("Delegate coding tasks, inspect results, and clean up resources when done.") .model(Model::new("gpt-6.2-sol", everruns::OpenAI::from_env()?)) .cursor(ApiKey::new(std::env::var("CURSOR_API_KEY")?)) .browserless(ApiKey::new(std::env::var("BROWSERLESS_API_TOKEN")?)) .e2b(ApiKey::new(std::env::var("E2B_API_KEY")?)) .build()?; ``` `ApiKey` redacts `Debug`; credentials are never serialized with agent configuration. Implement `services::Credentials` for vault-backed or rotating keys. Credential acquisition happens when a vendor tool executes; errors are sanitized, and an explicitly bound provider does not fall back to another application’s account. These credentials grant access to the selected vendor account, so use scoped vendor keys and agent instructions appropriate to that account. | Service | Native behavior | | ----------- | ----------------------------------------------------------------------------------------------------------------------- | | Cursor | Launch, inspect, follow up, list and delete Cloud Agents; inspect conversation/model/repository/key metadata | | Browserless | Open/reconnect/close browser sessions, navigate, read, interact and capture results using the shared browser operations | | E2B | Create/list sandboxes, execute commands, read/write files, and manage pause/resume/delete lifecycle | Browser and sandbox state belongs to the Engine session’s service storage. Use a durable Engine profile and protected secret storage when resources must survive process loss. E2B execution uses the provider’s isolation and timeout policies. Explicitly close browsers and delete sandboxes when finished; application operators remain responsible for account quotas and orphan cleanup after a host crash. The Framework does not run the Server’s fleet cleanup scheduler. The [native services example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/service_credentials.rs) uses a real model and one selected service: ```bash cargo run -p everruns --features openai,cursor,browserless,e2b \ --example service_credentials -- e2b ``` Replace `e2b` with `cursor` or `browserless`. The default Cursor task only lists models. The browser task opens a public page and closes the browser; the E2B task creates a sandbox, runs Python, then deletes it. Custom tasks can incur vendor costs. Image generation is available independently through `images` and `everruns::images`. See the [image generation example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/image_generation.rs) for real provider credentials and generated artifacts. Named agent delegation uses the existing [coordinator API](https://everruns.com/docs/framework/coordination/); members retain their own model, instructions and capabilities. --- # Observability > Export Framework sessions to OpenTelemetry and Braintrust, register your own event listeners, and flush them on shutdown. Source: An Engine delivers every event from every session it owns to the listeners registered on it. The `everruns` crate ships two listeners, OpenTelemetry and Braintrust, each behind its own Cargo feature. Without either feature nothing leaves the process. ```toml [dependencies] everruns = { version = "0.34", features = ["openai", "otel", "braintrust"] } ``` The features are independent: enable only the one you export to. ## OpenTelemetry The `otel` feature adds `everruns::observability::OpenTelemetry`. Register it on the Engine builder: ```rust use std::time::Duration; use everruns::observability::{OpenTelemetry, install_otlp_from_env}; use everruns::{Agent, Engine, Model}; let telemetry = install_otlp_from_env(); let provider = telemetry.tracer_provider() .ok_or("OTEL_EXPORTER_OTLP_ENDPOINT is required")?; let engine = Engine::builder() .observe(OpenTelemetry::from_env().tracer_provider(provider)) .build(); let agent = Agent::builder() .instructions("Answer concisely.") .model(Model::simulated("Hello.")) .build()?; engine.create(agent).run("Say hello.").await?; let report = engine.shutdown(Duration::from_secs(10)).await; assert!(!report.timed_out); assert!(report.stats.listeners.iter().all(|listener| listener.flushed)); telemetry.shutdown().await?; ``` Pick the constructor by who owns the tracer provider: | Constructor | Tracer provider | Content capture | | ----------------------------------------------------- | ----------------------------------------------------------------------- | -------------------- | | `OpenTelemetry::global()` | The global provider your application installed | Off | | `OpenTelemetry::from_env().tracer_provider(provider)` | An `SdkTracerProvider` you pass in. `Engine::shutdown` force-flushes it | Off | | `OpenTelemetry::from_env()` | The global provider | From the environment | None of them installs global state. `install_otlp_from_env()` is the one helper that does: it installs an OTLP exporter as the global tracer provider and a console `tracing` subscriber, and returns a `TelemetryGuard`. Keep the guard alive until after `Engine::shutdown`. Skip it when your application already sets up OpenTelemetry. `install_otlp_from_env()` reads: | Variable | Default | Effect | | ----------------------------- | ---------- | --------------------------------------------------------------------------------------------- | | `OTEL_EXPORTER_OTLP_ENDPOINT` | none | OTLP endpoint. Without it no spans are exported, and only the console subscriber is installed | | `OTEL_SERVICE_NAME` | `everruns` | Service name on the spans | | `OTEL_SERVICE_VERSION` | none | Service version | | `OTEL_ENVIRONMENT` | none | Deployment environment label | | `OTEL_SDK_DISABLED` | `false` | `true` turns exporting off without unsetting the endpoint | | `RUST_LOG`, then `LOG_LEVEL` | `info` | Console log filter | `OpenTelemetry::from_env()` reads `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` (or its older alias `OTEL_RECORD_CONTENT`) and `EVERRUNS_TRACE_CONVENTIONS`. Content capture puts instructions, messages, reasoning, and tool arguments and results on spans; it is off by default. Turn it on in code with `.record_content(true)`. The span layout and attribute vocabularies are the same as on the platform: see [OpenTelemetry](https://everruns.com/docs/observability/opentelemetry/). ## Causal links across durable turns Each turn starts an independent trace with a short `durable.turn` anchor. Child messages and task wakes link back to the creating turn. Restarted execution reads the committed anchor without re-exporting history. Sampling remains controlled by your tracer provider. Attach a caller’s W3C context to an input when your ingress already has one: ```rust use everruns::{CausalContext, InputMessage}; let causes = CausalContext::from_traceparents([ "00-1234567890abcdef1234567890abcdef-1234567890abcdef-01", ]); let input = InputMessage::user("Continue the analysis").with_causal_context(&causes); // session.send(input).await?; ``` Custom tools can use `ToolCallContext::causal_context()` when submitting a child input. `TaskRequest::with_causal_context` and `WakeRequest::with_causal_context` retain the same typed context through work delivery retries and completion wakes. Custom work backends must retain it with their task/wake snapshots and compare requests with `same_submission_as`, which excludes retry telemetry. Built-in child delegates and task wakes propagate it automatically. Batched and steered inputs contribute links to the reason that reads them. Retrying a stable message ID keeps its first committed causes. Carriers retain at most sixteen distinct trace/span identities. They exclude baggage, tracestate, arbitrary attributes and remote sampling decisions. Invalid entries are ignored. Old journals without carriers continue to work; disabled observers and dropped events can leave trace gaps. Custom hosts that replace the event or child-delivery adapters must forward the optional causal methods. Legacy `WakeRoutes` string receivers carry only text; their fallback runner forwards causal metadata. ## Braintrust The `braintrust` feature adds `everruns::observability::Braintrust`. `Braintrust::from_env()` returns `None` when Braintrust is disabled or no API key is set, so registration is conditional: ```rust use std::time::Duration; use everruns::Engine; use everruns::observability::Braintrust; let mut builder = Engine::builder(); if let Some(braintrust) = Braintrust::from_env() { builder = builder.observe(braintrust); } let engine = builder.build(); // ... run sessions ... let report = engine.shutdown(Duration::from_secs(10)).await; ``` It reads the `BRAINTRUST_*` variables listed on [Braintrust](https://everruns.com/docs/observability/braintrust/). Its final batch is sent before `Engine::shutdown` returns. ## Your own listeners Implement `EventListener` to receive the same events in application code, for example to write an audit log or update metrics: ```rust use everruns::approval::async_trait; use everruns::{Engine, EventFilter, EventListener, SessionEvent}; struct ToolAudit; #[async_trait] impl EventListener for ToolAudit { async fn on_event(&self, event: &SessionEvent) { println!("{} {}", event.session_id, event.event_type()); } fn filter(&self) -> EventFilter { EventFilter::event_types(["tool.started", "tool.completed"]) } fn name(&self) -> &'static str { "tool-audit" } } let engine = Engine::builder().listener(ToolAudit).build(); ``` | Method | Default | Purpose | | ---------- | -------------------- | -------------------------------------------------- | | `on_event` | required | Handle one event, after it is committed | | `filter` | `EventFilter::all()` | Restrict delivery to the listed event types | | `flush` | no-op | Write out buffered state during `Engine::shutdown` | | `name` | `"EventListener"` | Label in observer statistics | For a quick hook, `Engine::builder().on_event(|event| async move { ... })` registers an async closure that receives each `SessionEvent` by value. The event types and payloads are listed in [Events](https://everruns.com/docs/features/events/). ## Delivery and shutdown Each listener has its own bounded queue, holding up to `OBSERVER_QUEUE_CAPACITY` (8192) events and 8 MiB of serialized event data. A slow listener never delays a turn or another listener. When its queue is full, new events for that listener are dropped and counted. A listener that panics is isolated, and the panic is counted. `engine.observer_stats()` returns those counters per listener (`delivered`, `dropped`, `panics`, `flushed`, `flush_error`), with `dropped()` and `panics()` totals. `engine.shutdown(deadline)` stops accepting events, drains the queues, and calls `flush` on each listener. Listeners still running at the deadline are cancelled, and the returned `ObserverReport` has `timed_out` set. Call it before the process exits, or the last events may never reach the exporter. An OpenTelemetry exporter failure leaves `flushed=false` and supplies `flush_error`; inspect both that field and `timed_out`. `OpenTelemetry::global()` and `from_env()` use the application’s global tracer. They cannot force-flush an opaque application-owned provider, so shutdown reports that limitation. Use `with_tracer_provider` when Engine shutdown should flush it. `TelemetryGuard::tracer_provider()` exposes the convenience installer’s provider; its async `shutdown()` runs the SDK shutdown off runtime threads and reports failures. Dropping the guard retains the SDK’s blocking best-effort fallback. ```rust use std::time::Duration; use everruns::Engine; let engine = Engine::builder().build(); let report = engine.shutdown(Duration::from_secs(10)).await; if report.timed_out || report.stats.dropped() > 0 { eprintln!("observers lost events: {:?}", report.stats); } ``` ## Runnable example The [Framework observability example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/framework_observability.rs) registers both integrations and runs a turn on a simulated model, so it needs no provider key: ```bash OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \ cargo run -p everruns --features otel,braintrust \ --example framework_observability ``` ## See also * [Events and Cancellation](https://everruns.com/docs/framework/events-and-cancellation/), per-session event streams * [Deploying a Framework app](https://everruns.com/docs/framework/deployment/) * [OpenTelemetry](https://everruns.com/docs/observability/opentelemetry/) and [Braintrust](https://everruns.com/docs/observability/braintrust/), exporter details shared with the platform --- # Quickstart > Install everruns and run a real agent against a live model provider in about five minutes. Source: ## Install Add the application-facing crate with a model provider: ```bash cargo add everruns --features openai cargo add tokio --features macros,rt-multi-thread export OPENAI_API_KEY=sk-... ``` `--features openai` bundles the OpenAI driver. `everruns` does not re-export tokio, so add it for `#[tokio::main]`. Any other provider is its own crate — see [Supported providers](https://everruns.com/docs/framework/models-and-providers/#supported-providers). ## Run one turn ```rust use everruns::{Agent, Engine, OpenAI}; #[tokio::main] async fn main() -> Result<(), Box> { let agent = Agent::builder() .instructions("You are a concise assistant.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let engine = Engine::new(); let session = engine.create(agent); let turn = session.send_and_wait("Explain durable execution in one sentence.").await?; println!("{}", turn.response); Ok(()) } ``` The model id stays credential-free: `.model("…")` names the model, and `.provider(…)` supplies the driver, endpoint, and key separately. `OpenAI::from_env` reads `OPENAI_API_KEY` and redacts it from debug output. `send_and_wait` is the request/response convenience; use `send` when the application needs to stream output or steer a turn while it runs. ## Give it a tool An agent becomes useful when it can act. Annotate a function and hand it over: ```rust use everruns::{Agent, OpenAI}; #[everruns::tool] /// Look up the current stock level for a SKU. async fn stock_level(sku: String) -> Result { Ok(inventory_lookup(&sku).await) } let agent = Agent::builder() .instructions("Answer inventory questions. Use the tool rather than guessing.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .tool(stock_level()) .build()?; ``` The macro derives the JSON schema from the signature, so the model sees typed arguments and the code stays ordinary Rust. See [Tools and macros](https://everruns.com/docs/framework/agents/#tools). ## Where to go next * [Agents](https://everruns.com/docs/framework/agents/) — instructions, files, workspaces, and MCP. * [Tools and macros](https://everruns.com/docs/framework/agents/#tools) — typed function tools. * [Sessions](https://everruns.com/docs/framework/sessions/) — multi-turn state and history. * [Framework architecture](https://everruns.com/docs/framework/architecture/) — how the pieces fit. ## Testing without a provider Once you are building for real, you will want tests that do not call a model. `Model::simulated` replays a canned response through the same model/provider path: ```rust use everruns::{Agent, Engine, Model}; let agent = Agent::builder() .instructions("You are a concise assistant.") .model(Model::simulated("Everruns is ready.")) .build()?; let turn = Engine::new().create(agent).send_and_wait("Are you ready?").await?; assert_eq!(turn.response, "Everruns is ready."); ``` It is a test double, not a local model: it runs no inference, so it proves your wiring rather than any model behavior. Use it in tests and CI, not as a way to run Everruns without a provider. See [Testing and simulation](https://everruns.com/docs/framework/testing-and-simulation/). --- # Serve (experimental) > Build a hosted agent app with attribute macros, file-layout discovery and a manifest, served over a subset of the Everruns server /v1 API. Source: > **Experimental.** serve is a proof of concept. Its APIs will change and it has no compatibility promise. **serve** is an application framework on top of the Everruns Framework. You write an app as a set of annotated functions, the build declares what the app needs from its host, and the binary serves the same `/v1` session API as the Everruns server, so the SDKs, CLI and UI chat view can drive it. serve adds no runtime of its own. Agents, sessions, tools, approvals, [`ask_user`](https://everruns.com/docs/framework/ask-user/), skills and the durable event log all come from the `everruns` crate: serve is a thin layer over [`Engine`](https://everruns.com/docs/framework/architecture/). ![Serve request map: the /v1 session routes, channel webhooks, the AG-UI and A2A routes, AgentCore's /invocations and /ws, and in-process schedules all reach one serve host, which runs sessions, approvals and ask_user questions on one everruns::Engine backed by a local SQLite session log.](https://everruns.com/docs/_astro/serve-routes.C8Ygk5eV_1OflDF.svg) ```rust use serve::prelude::*; #[agent] fn analyst() -> Agent { Agent::builder() .model("anthropic/claude-sonnet-5") // resolved through the host's provider settings .instructions(md!("instructions.md")) // agent/instructions.md, hot-reloads in dev .build() } /// Run a read-only SQL query against the warehouse. #[tool(needs_approval = |a: &RunSql| scan_gb(&a.sql) > 50.0)] async fn run_sql(cx: &Cx, sql: String) -> Result { let wh = cx.connection::()?; // credentials never reach the model cx.progress("querying…").await; Ok(wh.query(&sql)?.truncate(500)) } serve::assets!(); #[tokio::main] async fn main() -> serve::Result { serve::start(App::builder().discover().build()).await } ``` For a guided start, [Tutorial: Serve an Agent and Call It from Code](https://everruns.com/docs/framework/serve-tutorial/) builds an agent, protects it with an API key, and calls it from the SDKs. ## Install ```sh cargo add everruns-serve tokio --features tokio/full cargo add --build everruns-serve-build ``` The package is `everruns-serve` and the library is imported as `serve`. Add a `build.rs` that calls `serve_build::embed()` so `agent/**` and `serve.toml` are compiled into the binary. ## Try it The examples use real models; follow each example’s credential requirements. The Bashkit workspace walkthrough below uses `OPENAI_API_KEY` and checks files across a server restart. From a checkout of the repository: ```sh cargo run -p serve-example-revenue-analyst # dev server on :3000 with a live console cargo run -p serve-example-revenue-analyst -- eval # run the evals in-process cargo run -p serve-example-revenue-analyst -- manifest ``` Then talk to it with the same requests you would send an Everruns server: ```sh ID=$(curl -s localhost:3000/v1/sessions -H 'content-type: application/json' -d '{}' | jq -r .id) curl -s localhost:3000/v1/sessions/$ID/messages -H 'content-type: application/json' \ -d '{"message":{"content":[{"type":"text","text":"What was revenue last week?"}]}}' curl -N "localhost:3000/v1/sessions/$ID/sse?after_sequence=0" ``` For `openai/` agents, set `OPENAI_API_KEY`. Optional `OPENAI_BASE_URL` selects an OpenAI-compatible Responses API endpoint; Serve removes the first `openai/` prefix before sending the model ID. This explicit URL/key pair takes precedence over OpenRouter for these agents. Otherwise, `OPENROUTER_API_KEY` routes `provider/model` IDs through OpenRouter unchanged. With serve’s `bedrock` feature, `bedrock/` models (for example `bedrock/us.anthropic.claude-sonnet-4-6`) call Amazon Bedrock on the AWS default credential chain whenever `AWS_REGION` is set. OpenAI and OpenRouter HTTP requests honor `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY` and `NO_PROXY` (including lowercase forms). Configure only a trusted forward proxy that blocks private and metadata destinations when it resolves and connects. Serve validates destination URLs before forwarding, refuses redirects, and keeps TLS verification enabled. The proxy performs the final DNS/address check; a failed proxy request does not fall back to a direct connection. | Example | Shows | | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | | [Hello app](https://github.com/everruns/everruns/tree/main/examples/serve/hello) | The smallest app: one agent, one tool, one eval. | | [Bashkit workspace](https://github.com/everruns/everruns/tree/main/examples/serve/bashkit-workspace) | Shell commands, resulting files, persistence across turns and a server restart, and session isolation. Includes a real-model walkthrough. | | [Revenue Analyst](https://github.com/everruns/everruns/tree/main/examples/serve/revenue-analyst) | A tool with approvals, a skill, Slack, a schedule, MCP and typed connections, an agent that hands work to another, evals and the Bashkit sandbox. | | [AG-UI app](https://github.com/everruns/everruns/tree/main/examples/serve/ag-ui) | An agent streamed to `@ag-ui/client` and CopilotKit, with an approval as an interrupt. | | [AgentCore app](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore) | The same kind of app packaged for Amazon Bedrock AgentCore Runtime. | | [AgentCore workspace app](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore-workspace) | An AgentCore workspace agent with a shell in the microVM and an approval. | | [A2A example](https://github.com/everruns/everruns/tree/main/examples/serve/a2a) | Two agents over A2A: a served `researcher`, and an `everruns` agent that delegates to it. | | [Voice example](https://github.com/everruns/everruns/tree/main/examples/serve/voice) | A hotel front desk people call from the browser, with a greeting and a speaking style. | ## Project layout The file layout says what each piece is: ```text revenue-analyst/ build.rs # serve_build::embed() serve.toml # name, extra secrets agent/ instructions.md # always-on prompt skills/sql-style/SKILL.md src/ main.rs # serve::assets!() + serve::start(...) agent.rs # #[agent(default)], .member("reviewer") reviewer.rs # #[agent] tools/run_sql.rs # #[tool] channels/slack.rs # #[channel] schedules/weekly.rs # #[schedule] connections/linear.rs # #[connection] evals/revenue.rs # #[eval] ``` The macros register each item at link time and `discover()` collects them; `build.rs` embeds `agent/**` and `serve.toml` into the binary. `discover()` reports every problem in the app at once, such as duplicate names, a bad cron expression or a filter naming an unknown tool. ## The pieces | Macro | On | Becomes | | -------------------------------------------------------------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `#[agent]`, `#[agent(default)]` | `fn() -> Agent` | An agent. New sessions use the default one. `Agent::builder().member("name")` lets it hand work to another agent of the app through `spawn_agent`; see [Agents that use other agents](#agents-that-use-other-agents). | | `#[tool]`, `#[tool(needs_approval)]`, `#[tool(needs_approval = \|a: &Args\| …)]` | `async fn([cx: &Cx,] args…) -> Result` | A tool named after the function, described by its doc comment. It also generates an arguments struct (`run_sql` → `RunSql`) with a JSON Schema. | | `#[channel]` | `fn() -> impl Into` (a driver such as `Slack` or `Webhook`) | `POST /v1/channels/{name}`. Each external thread maps to one session, and replies are delivered back while the turn runs. `agent = "…"` picks the agent. | | `#[schedule("0 9 * * MON")]` | `async fn(&Cx) -> Result` | A cron entry. A 5-field cron counts weekdays 0-6 from Sunday (7 is also Sunday), so `0 9 * * 1-5` runs Monday through Friday. | | `#[connection]` | `fn() -> T` or `fn() -> Result` | A value tools read with `cx.connection::()`. An `McpServer` connection is attached to every agent. | | `#[eval]` | `async fn(&mut EvalCx) -> Result` | A conversation with assertions, run in-process or against a deployment. | `Cx` is the one context type. Inside a tool it wraps the Framework’s `ToolCallContext` (session, turn and tool call ids, and `progress`) and adds typed connections, secrets and `start_session`. Approval rules are declared on the tool and enforced by the runtime. ## Commands The commands are built into the app binary, because only the linked binary knows what it registered. | Command | What it does | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `dev` (default) | Serves the API with local SQLite under `.serve/` and a live console. Models fall back to the simulator, and Markdown prompts and skills hot-reload. | | `start` | Production mode. Every model must have a provider route, and every declared secret must be set. With `--store s3://bucket/prefix` (or `SERVE_STORE`) the data lives in a bucket; see [Keep the data in a bucket](#keep-the-data-in-a-bucket). | | `manifest` | Prints the host contract as JSON: agents, models, tool schemas, skills, channels, cron entries, secrets, sandbox, evals and a build id. | | `eval [--against URL]` | Runs the evals in-process, or against a running deployment as a gate before promotion. Against an app that requires a credential, set `SERVE_CREDENTIAL` (or pass `--credential`) and every request carries it as a bearer token. | | `deploy` | Prints what a host would provision from the manifest. The one supported deployment target is [Amazon Bedrock AgentCore](https://everruns.com/docs/framework/serve-agentcore/). | Sessions that an earlier release kept in the data directory’s `everruns/` folder move to the current session store the first time they are opened, with their history and their shared workspace files. Child sessions they started through a former `#[agent(sub)]` tool move with them and are linked to their parent. A session that cannot move returns an error; it is never served as missing or empty. ## Keep the data in a bucket `start --store s3://bucket/prefix` keeps a daemon’s data in an S3 bucket, or in any S3-compatible store that supports conditional writes (R2, MinIO, GCS, Tigris, SeaweedFS). The data directory becomes a local working copy: * At boot the daemon takes the bucket’s lease, then rebuilds the data directory from the bucket. Files the bucket does not have are removed. * While it runs, every change is copied to the bucket within about a fifth of a second: databases by the pages that changed, other files (the agents’ workspace) whole. * A daemon started on another machine with the same `--store` waits for the old one’s lease to expire (15 seconds), rebuilds the data directory, and resumes the turns the old one was running. While the old daemon is still alive and renewing, the new one refuses to start. If the old one wakes up after it was replaced, its next write fails and it stops. * A clean shutdown (Ctrl+C) copies what is left and releases the lease, so the next daemon starts at once. Credentials, region and endpoint come from the standard `AWS_*` environment variables. For an S3-compatible store set `AWS_ENDPOINT`, and `AWS_ALLOW_HTTP=true` for a plain-HTTP endpoint. Keep the data directory at the same absolute path on every machine (`SERVE_DATA_DIR`), because a session’s workspace records it. A crash can lose the changes of its last fifth of a second. One daemon holds a bucket prefix at a time; give each daemon its own prefix. ## Wire API A serve app speaks a subset of the Everruns server `/v1` API, with the same request and response bodies: * `POST /v1/sessions` and `GET /v1/sessions/{id}` * `POST /v1/sessions/{id}/messages`, which also steers an active turn. Its `files` list sends files as base64 `data` or a public `url`, as on the [Agent API](https://everruns.com/docs/how-to/call-your-agent-from-code/#send-files); each is kept in the session’s `/attachments` folder * `POST /v1/sessions/{id}/cancel` * `GET /v1/sessions/{id}/sse` (resume with `since_id` or `after_sequence`) and `GET /v1/sessions/{id}/events` * `POST /v1/sessions/{id}/question-answers`, for [`ask_user`](https://everruns.com/docs/framework/ask-user/) questions * `POST /v1/sessions/{id}/tool-approvals`, to allow or reject pending tool calls It adds a few routes of its own: `GET /health`, `GET /v1/agent` (the agent card), channel webhooks, the [AG-UI](#ag-ui-and-copilotkit), [A2A](#a2a) and [voice](#voice) routes, and `POST /v1/sessions/{id}/approvals/{tool_call_id}` to approve or deny one pending tool call. Errors are `application/problem+json`. ### One agent’s API Every top-level agent also has its own base URL, `/v1/channels/{agent}`, with the same session routes rooted under it: `GET` returns the agent’s card, `POST /v1/channels/{agent}/sessions` starts a session that runs that agent, and `/v1/channels/{agent}/sessions/{id}/…` carries messages, events, cancellation, questions and approvals. A session is reachable only under its own agent’s URL. This is the API an Everruns agent exposes through an API channel, so code written against one agent works against either host. Every SDK’s `AgentClient` calls this API; see the [tutorial](https://everruns.com/docs/framework/serve-tutorial/#6-call-it-from-code). The [Agent API](https://everruns.com/docs/features/agent-api/) reference documents the routes and shapes, and [what differs on serve](https://everruns.com/docs/features/agent-api/#the-same-api-on-serve). ### Authentication By default the wire API has no authentication and no organizations: run it locally, or behind a host that authenticates requests. Configure one or more methods and every agent route requires `Authorization: Bearer `; anything else gets `401` with `WWW-Authenticate: Bearer`. The methods are the ones an Everruns API channel accepts, checked by the same verifier: * **API keys.** Static keys, compared in constant time. Set them in `SERVE_API_KEYS`, comma separated. * **OpenID Connect.** JWTs from an issuer, checked against the keys its discovery document publishes, with a required audience. * **OAuth 2.0 introspection.** Opaque tokens checked at an RFC 7662 endpoint. Token methods take the same requirements as a channel: audiences, scopes, subjects, groups, email domains and exact claim values. Methods add up from code, `serve.toml` and the environment: ```rust use serve::auth::AuthMethod; let server = serve::Server::builder(app, serve::Mode::Start) .auth([AuthMethod::oidc("https://login.example.com", ["support-agent"])]) .build()?; ``` serve.toml ```toml [[auth.methods]] mode = "oidc" provider = { type = "oidc", issuer = "https://login.example.com" } requirements = { audiences = ["support-agent"], scopes = ["agent:call"] } ``` ```sh SERVE_API_KEYS=key-for-ci,key-for-the-web-app ./revenue-analyst start ``` `serve.toml` also takes `api_keys = [...]` under `[auth]`, but it is embedded in the binary, so keep real keys in the environment. A few routes stay public so clients can find the agent: `GET /health`, each agent’s card at `GET /v1/channels/{agent}` (its `auth` list names the accepted methods), the A2A Agent Card (its `securitySchemes` and `securityRequirements` name the same methods, as an Everruns A2A endpoint’s card does) and the voice test page. Messaging channel webhooks stay public too, because Slack and webhook callers cannot send a serve credential; each channel verifies its platform’s own signature. Auth does not separate callers: every authenticated caller can reach every session. With auth on, the `dev` console’s sample requests and its approval and question hints include `-H 'authorization: Bearer '`; substitute your own key. To run evals against a protected deployment, give `eval` the credential through the environment, which keeps it out of the process list: ```sh SERVE_CREDENTIAL=key-for-ci ./revenue-analyst eval --against https://preview.example.com ``` ## AG-UI and CopilotKit With the `ag-ui` feature, every top-level agent also serves [AG-UI](https://docs.ag-ui.com) 1.0 clients such as CopilotKit and `@ag-ui/client`: ```sh cargo add everruns-serve --features ag-ui ``` The route is `POST /v1/channels/{agent}/ag-ui`, the same shape as an Everruns endpoint’s AG-UI route, so a front end moves between a local serve app and Everruns by base URL and id alone: ```ts import { HttpAgent } from "@ag-ui/client"; const agent = new HttpAgent({ url: "http://localhost:3000/v1/channels/analyst/ag-ui" }); ``` Each AG-UI `threadId` maps to one session, which survives a restart. A thread’s first run records the input’s earlier user and assistant messages as the new session’s history; after that, a run sends only the input’s last user message. Each run streams the turn as AG-UI events, with reasoning, token usage and errors visible. A tool approval or an [`ask_user`](https://everruns.com/docs/framework/ask-user/) question ends the run with an interrupt (`tool_approval` or `everruns.ask_user`), and the next run’s `resume` entries answer it and stream the rest of the turn. These are the same pending requests the approvals and `question-answers` routes answer, so either API can resolve them. Input that is not a valid `RunAgentInput` gets `400`, and an unknown agent `404`. The agent card lists each agent’s route under `ag_ui`. See [Serve AG-UI](https://everruns.com/docs/framework/ag-ui/) for the event and interrupt shapes. ## A2A With the `a2a` feature, every top-level agent also serves other agents over [A2A](https://a2a-protocol.org) 1.0 JSON-RPC: ```sh cargo add everruns-serve --features a2a ``` The endpoint is `POST /v1/channels/{agent}/a2a` and its Agent Card is `GET /v1/channels/{agent}/a2a/.well-known/agent-card.json`, the shape of an Everruns A2A endpoint. Any A2A 1.0 client works, including the official `a2a` CLI and another Everruns agent’s `a2a_agent_delegation` capability: ```sh a2a send -a http://localhost:3000/v1/channels/researcher/a2a/.well-known/agent-card.json "Tide pools" ``` Each A2A `contextId` maps to one session, which survives a restart; each task is one turn. A task streams `working`, then the final reply as a `response` artifact, then `completed` (or `failed`). `GetTask`, `ListTasks`, `CancelTask` and `SubscribeToTask` come from the A2A Rust SDK’s request handler; tasks are kept in memory, so old task ids are forgotten on restart while their context continues. Requests need the `A2A-Version: 1.0` header. A pending approval or `ask_user` question keeps the task `working` until the routes above answer it. The agent card lists each agent’s endpoint under `a2a`. [Framework A2A](https://everruns.com/docs/framework/a2a/) covers serving and calling A2A agents, and [A2A example](https://github.com/everruns/everruns/tree/main/examples/serve/a2a) shows a served agent and a second agent that delegates to it. Sessions survive a restart: the binary rebuilds each agent and resumes the session from the local store. A turn the old process left waiting on an approval or an `ask_user` question waits again: the call runs again, so the request is pending once more under the same tool call id, and answering it finishes the turn. A turn cut off while a tool executed is not re-run. A session pinned to a different build gets `409 Conflict` with an `x-serve-build` header, so a host can route it to the build that owns it. ## Voice With the `voice` feature, every top-level agent also takes browser calls through a [voice channel](https://everruns.com/docs/framework/voice/): ```sh cargo add everruns-serve --features voice ``` `POST /v1/channels/{agent}/voice/calls` takes the browser’s WebRTC offer and answers the SDP answer, the call id and the session id; `POST /v1/channels/{agent}/voice/calls/{call_id}/end` hangs up, and `GET /v1/channels/{agent}/voice` is a test page with a **Start call** button. Each utterance is a user message on an ordinary session, so a call can continue a typed conversation. The optional `[voice]` section of `serve.toml` sets the voice, greeting and speaking style for every agent. Speech uses OpenAI with `OPENAI_API_KEY`; without it, `dev` and `eval` use the offline simulator and `start` refuses calls. The agent card lists each agent’s route under `voice.endpoints`. See [Build a voice agent](https://everruns.com/docs/framework/voice/). ## Images An `[images]` section in `serve.toml` gives every agent the [Image Generation](https://everruns.com/docs/capabilities/image-generation/) tools, so an agent can draw stills, animated GIFs and SVGs: ```toml [images] model = "gpt-image-2.6-flare" # the default format = "png" # when a call picks none; "gif" animates ``` Images are saved under `/workspace/images` in the session’s files, which the section also makes writable. Keys come from the environment: `OPENAI_API_KEY` (GIF and SVG need OpenAI or Azure OpenAI), or `OPENROUTER_API_KEY` for stills. Without the section agents get no image tools. ## Agents that use other agents An agent can hand work to other agents of the same app: ```rust use serve::prelude::*; /// Answers revenue questions. #[agent(default)] fn analyst() -> Agent { Agent::builder() .model("openai/gpt-6.2-sol") .instructions(md!("instructions.md")) .member("reviewer") .build() } /// Reviews a SQL query or a revenue summary before it is shared. #[agent] fn reviewer() -> Agent { Agent::builder().model("openai/gpt-6-luna").build() } ``` The analyst gets the `spawn_agent` tool with the reviewer as a target, shown with its doc comment. Each hand-off runs as the reviewer’s own session in the analyst’s run, in the foreground or as a background task, and the answer comes back to the analyst. The reviewer is still an agent of the app, with its own sessions and routes. This is the same hand-off the [Framework](https://everruns.com/docs/framework/) and the platform use; see [Agent handoff](https://everruns.com/docs/capabilities/agent-handoff/). A member must be another agent of the app and cannot have members of its own; `discover()` reports anything else. ## Limitations * After a restart, an approval answered “always” before it is asked again is lost. * A member agent’s approvals and questions wait on the member’s own session, not on the session that handed it the work. * A2A tasks are held in memory, and A2A 0.3 clients are not served. * A deny note is not passed to the model. * There is no generic OCI build or Postgres or NATS adapter. [AgentCore](https://everruns.com/docs/framework/serve-agentcore/) is the one supported deployment target. * The server’s agent, harness, workspace and tool-result routes are not served. ## Amazon Bedrock AgentCore [`everruns-serve-agentcore`](https://docs.rs/everruns-serve-agentcore) runs a serve app on AgentCore Runtime. Replace `serve::start` with `serve_agentcore::start` in `main`. With no command, the binary then serves AgentCore’s contract on port 8080: `GET /ping` and `POST /invocations`, which takes an AG-UI `RunAgentInput` or `{"prompt": "..."}` and streams AG-UI events. serve’s own commands keep working. See [Serve on AgentCore](https://everruns.com/docs/framework/serve-agentcore/) for deployment, persistence on session storage, tools and models. ## celld [`everruns-serve-celld`](https://docs.rs/everruns-serve-celld) runs a serve app durably on [celld](https://github.com/denoland/celld), self-hosted Durable Objects. Replace `serve::start` with `serve_celld::start`; the binary then runs in a container that a Durable Object supervises, and the object keeps a snapshot and a request journal so a lost container is restored and its interrupted turn replayed. See [Serve on celld](https://everruns.com/docs/framework/serve-celld/). The full guide, wire reference and hosting contract live next to the crate in [serve reference guides](https://github.com/everruns/everruns/tree/main/crates/serve/docs). ## File-based agents Load and export the shared portable package format, including initial files and complete skill directories. See [Define agents as files](https://everruns.com/docs/how-to/define-agents-as-files/) for validation, diffs, model bindings and runnable examples. ## Agent sandbox and workspace permissions Declare capabilities with the agent in `agent.rs`: ```rust use serve::prelude::*; let agent = Agent::builder() .model("openai/gpt-6-luna") .sandbox(SandboxKind::Bashkit) .workspace_policy(everruns::WorkspacePolicy::read_only()) .build(); ``` Bashkit provides a virtual shell over the session filesystem. Selecting it does not enable writes; supply a scoped workspace policy when writes are needed. The [Bashkit workspace example](https://github.com/everruns/everruns/tree/main/examples/serve/bashkit-workspace) declares its write scope in the same builder and verifies files across turns and a server restart. Agents can select different sandboxes in one app. An explicit selection, including `SandboxKind::None`, overrides the legacy `[sandbox]` app default in `serve.toml`. The manifest lists each agent’s effective sandbox. Hosts still supply storage and MicroVM adapters; without an adapter Serve retains its Bashkit substitution. App naming, auth and deployment settings remain in `serve.toml`. --- # Serve on Amazon Bedrock AgentCore (experimental) > Deploy a serve app to Amazon Bedrock AgentCore Runtime with everruns-serve-agentcore: sessions on session storage, tools in the microVM, models via the Gateway. Source: > **Experimental.** Like [serve](https://everruns.com/docs/framework/serve/), `everruns-serve-agentcore` is a proof of concept with no compatibility promise. [Amazon Bedrock AgentCore Runtime](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agents-tools-runtime.html) runs your agent container once per session, each copy in its own microVM, and talks to it over HTTP on port 8080. `everruns-serve-agentcore` (imported as `serve_agentcore`) serves that contract around a [serve](https://everruns.com/docs/framework/serve/) app, so the same app runs under `dev` on a laptop and on AgentCore without changes. ## How it maps | AgentCore | serve | | --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | | Runtime session (`X-Amzn-Bedrock-AgentCore-Runtime-Session-Id`) | One serve session: the AG-UI thread defaults to the session id | | `POST /invocations` | An AG-UI 1.0 run of the app’s agent, streamed as server-sent events | | `GET /ws` (WebSocket) | The same runs: each text message is one invocation body, each AG-UI event one text message back | | `GET /ping` | `Healthy`, or `HealthyBusy` while a turn runs so AgentCore keeps the microVM up | | Session storage mount (`/mnt/workspace`) | serve’s SQLite session log and the agent’s workspace | | The session microVM | The sandbox: `[sandbox] kind = "microvm"` gives the agent a real shell | | OpenAI-compatible Responses API endpoint | `OPENAI_BASE_URL` and `OPENAI_API_KEY` for `openai/…` models | | Gateway MCP target | A `#[connection]` returning `McpServer` | serve’s `/v1` wire API and `/health` stay mounted next to the AgentCore routes. ## Quickstart Start from any serve app and change `main`: ```rust use serve::prelude::*; #[tokio::main] async fn main() -> serve::Result { serve_agentcore::start(App::builder().discover().build()).await } ``` ```toml [dependencies] everruns-serve = { version = "0.33", features = ["ag-ui"] } everruns-serve-agentcore = "0.33" ``` With no command (what AgentCore runs) or `agentcore`, the binary serves the AgentCore contract in serve’s `start` mode. Every other command (`dev`, `start`, `eval`, `manifest`, `deploy`) is serve’s own. Try the contract locally with the offline simulator: ```sh cargo run -p serve-example-agentcore -- agentcore --dev curl -s localhost:8080/ping # {"status":"Healthy"} curl -N localhost:8080/invocations \ -H 'content-type: application/json' \ -H 'x-amzn-bedrock-agentcore-runtime-session-id: local-session-0000000000000000000000001' \ -d '{"prompt":"Where is order A-1001?"}' # data: {"type":"RUN_STARTED",...} # ... # data: {"type":"RUN_FINISHED",...} ``` `/invocations` takes either an AG-UI `RunAgentInput` (what AgentCore’s AG-UI protocol forwards from CopilotKit or `@ag-ui/client`) or `{"prompt": "..."}` (a plain `InvokeAgentRuntime` call). Without a `threadId` and without the session header, it answers `400` with a problem document. ## Deploy AgentCore runs `linux/arm64` images on port 8080. The [agentcore example](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore) has a distroless Dockerfile; build and push from the repository root: ```sh docker buildx build --platform linux/arm64 \ -f examples/serve/agentcore/Dockerfile -t "$ECR_REPO:latest" --push . ``` Create the runtime with the AG-UI protocol, session storage and an OpenAI key for the example’s model: ```sh aws bedrock-agentcore-control create-agent-runtime \ --agent-runtime-name serve_agentcore_example \ --agent-runtime-artifact "containerConfiguration={containerUri=$ECR_REPO:latest}" \ --role-arn "$EXECUTION_ROLE_ARN" \ --network-configuration networkMode=PUBLIC \ --protocol-configuration serverProtocol=AGUI \ --filesystem-configurations '[{"sessionStorage":{"mountPath":"/mnt/workspace"}}]' \ --environment-variables "OPENAI_API_KEY=$OPENAI_API_KEY" ``` Use `serverProtocol=HTTP` instead when callers send `{"prompt": ...}`; `/invocations` accepts both either way. ## Invoke From the CLI: ```sh aws bedrock-agentcore invoke-agent-runtime \ --agent-runtime-arn "$AGENT_ARN" \ --runtime-session-id "$(uuidgen)-$(uuidgen)" \ --payload "$(echo -n '{"prompt":"Where is order A-1001?"}' | base64)" \ out.txt && cat out.txt ``` From an AG-UI client, point `HttpAgent` at the runtime’s invocations URL and send the session header, as the [workspace example’s client](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore-workspace/client) does. It also shows the approval round trip: the first run ends with an `interrupt` outcome, and the next run resumes it. ```js const agent = new HttpAgent({ url: `https://bedrock-agentcore.${region}.amazonaws.com/runtimes/${encodeURIComponent(agentArn)}/invocations?qualifier=DEFAULT`, headers: { Authorization: `Bearer ${token}`, "X-Amzn-Bedrock-AgentCore-Runtime-Session-Id": session, }, threadId: session, }); ``` Reuse a session id to return to the same microVM, conversation and workspace. Over AgentCore’s WebSocket transport, connect to the runtime’s `/ws` URL (`wss://bedrock-agentcore..amazonaws.com/runtimes//ws?qualifier=DEFAULT`) with the same session header, and send one invocation body per text message. Each AG-UI event comes back as its own text message, ending with `RUN_FINISHED` or `RUN_ERROR`; send the next body on the same socket for the next turn. The upgrade request is authenticated like any other call: sign it with SigV4 (service `bedrock-agentcore`), either in the headers or, where the client cannot set upgrade headers (a browser), as a presigned URL that also carries the session id as the `X-Amzn-Bedrock-AgentCore-Runtime-Session-Id` query parameter. The AWS SDKs have no WebSocket client, so sign the request yourself and hand it to a WebSocket library; the [agentcore-workspace example](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore-workspace) has a Python client (`client/ws.py`) that does both. ## Persistence AgentCore mounts session storage only when an invocation arrives, not while the container initializes. So `/ping` never touches storage, and the server boots on the first other request. It then picks, in order: 1. `SERVE_DATA_DIR` (or a SQLite `DATABASE_URL`) when set. The workspace is `SERVE_WORKSPACE`, else `workspace/` under the data dir. 2. `/mnt/workspace` when it is mounted: the session log goes to `/mnt/workspace/.serve`, and the agent’s workspace is `/mnt/workspace`. 3. Otherwise a temporary directory, with a warning that nothing survives the microVM stopping. With session storage, a conversation survives AgentCore stopping the microVM after its idle timeout: the next invocation with the same session id resumes it from the log. That includes a turn parked on an approval or an `ask_user` question: on the new microVM the interrupt is still open, and the client’s resume run answers it as before. Session storage is an NFS mount that refuses POSIX file locks, so the binary switches SQLite to dot-file locking (`unix-dotfile`) and rollback-journal mode for the whole process, and clears lock files a stopped microVM left behind. That is safe because one process owns a session’s storage. Session storage is per session, kept for 14 days, and reset when you deploy a new runtime version. For history across sessions and versions, point `DATABASE_URL` at storage you own, such as an S3 Files or EFS mount (both need VPC mode). ## Tools * **In-process tools.** `#[tool]` functions run in the container like anywhere else, including `needs_approval` tools: over AG-UI the run ends with a `tool_approval` interrupt and the client’s next run answers it. * **A real shell.** With `[sandbox] kind = "microvm"` in `serve.toml`, each agent gets `bash` and file tools in the session’s workspace, running directly in the container. The microVM is the isolation boundary, so there is no second sandbox inside it. Under `dev` and `eval` the same setting uses the bashkit virtual shell, so local runs stay safe. Install whatever the agent should run (git, python, a compiler) in the image, as the [workspace example](https://github.com/everruns/everruns/tree/main/examples/serve/agentcore-workspace) Dockerfile does. * **AgentCore Gateway tools.** A Gateway is an MCP server, so connect it like any other: ```rust #[connection] fn gateway() -> McpServer { McpServer::http("https://.gateway.bedrock-agentcore..amazonaws.com/mcp") .auth(Secret::named("GATEWAY_TOKEN")) } ``` `start` mode, which AgentCore runs, refuses to boot when `GATEWAY_TOKEN` is unset, so a misconfigured runtime fails its first invocation instead of running without tools. ## Models Name a Bedrock model as `bedrock/`, where the rest is a Bedrock model id or inference profile: ```rust Agent::builder() .model("bedrock/us.anthropic.claude-sonnet-4-6") ``` When `AWS_REGION` or `AWS_DEFAULT_REGION` is set, serve calls Amazon Bedrock directly with the runtime’s execution role: no keys in the image or the environment. Credentials come from the AWS default chain, which on AgentCore is the execution role’s container credentials, and refresh before they expire. The role needs `bedrock:InvokeModelWithResponseStream` on the model; for a cross-region inference profile such as `us.…`, on the profile and on the foundation model in each region it routes to. Pass the region in `--environment-variables` if the runtime does not already provide it. This route is serve’s `bedrock` feature, on by default in `everruns-serve-agentcore`; `default-features = false` drops it and the AWS SDK. For `openai/` agents, set `OPENAI_API_KEY`; optional `OPENAI_BASE_URL` selects a compatible Responses API endpoint. Serve strips only the first `openai/` prefix. An endpoint that expects `target/model` therefore needs an agent model of `openai/target/model`. An endpoint that expects `openai/gpt-6.2-sol` needs `openai/openai/gpt-6.2-sol` in the agent definition. Configure the endpoint’s credential as `OPENAI_API_KEY`. Without an explicit OpenAI URL/key pair, `OPENROUTER_API_KEY` routes `provider/model` IDs unchanged through OpenRouter. An OpenAI endpoint override does not reroute `anthropic/…` or `bedrock/…` models. ## Configuration | Variable | Meaning | | ----------------------------------- | ----------------------------------------------------------------------- | | `PORT` | Listen port, default `8080` (AgentCore requires 8080) | | `SERVE_AGENTCORE_AGENT` | The agent `/invocations` runs, default the app’s default agent | | `SERVE_DATA_DIR`, `DATABASE_URL` | Where the session log lives. Default: session storage when mounted | | `SERVE_WORKSPACE` | The agent’s workspace. Default: the session storage mount | | `OPENAI_API_KEY`, `OPENAI_BASE_URL` | Credential and optional Responses API endpoint for `openai/…` models | | `AWS_REGION`, `AWS_DEFAULT_REGION` | Region for `bedrock/` models, which run on the execution role | ## Compared with other AgentCore agents | | serve-agentcore | Strands / `bedrock-agentcore` SDK | | --------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------- | | `/ping`, `/invocations`, busy reporting | Yes | Yes | | AG-UI protocol | Yes, AG-UI 1.0 including interrupts | Yes | | `/ws` WebSocket | Yes | Yes | | MCP and A2A server protocols | Not yet | Yes | | Conversation persistence | SQLite on session storage, including a turn parked on an approval or question | File or AgentCore Memory session managers | | Long-term memory (AgentCore Memory) | Not yet | Yes | | Gateway tools | MCP connection | MCP client | | Shell and files | Built-in (`sandbox = "microvm"`) | Bring your own tools | | Code Interpreter, Browser | Not yet | Client libraries | | Identity (OAuth to third parties) | Secrets from the environment | `@requires_access_token` | | Offline evals and simulator | Yes (`eval`, `--dev`) | No | ## Limits * Approvals and `ask_user` questions park a turn without keeping the microVM busy. AgentCore may stop the microVM while one waits; with session storage the next microVM asks it again, but an “always” answer given before the restart is not remembered after it, and a request a subagent parked is lost. * `#[schedule]`s run in-process, so they only fire while a session’s microVM is up. Use EventBridge to call `InvokeAgentRuntime` on a schedule instead. * Only `bedrock/…` models use the runtime’s execution role. Other providers need the gateway or their own keys. The role must allow `bedrock:InvokeModelWithResponseStream`; a missing permission surfaces as an `AccessDeniedException` on the first turn. ## Reference * [`everruns-serve-agentcore` API reference](https://docs.rs/everruns-serve-agentcore) * [AgentCore AG-UI protocol contract](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-agui-protocol-contract.html) * [AgentCore session storage](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-persistent-filesystems.html) --- # Serve on celld (experimental) > Run Framework Engine sessions on celld with per-session Durable Object authority and replaceable native execution. Source: `everruns-serve-celld` is a source-only reference adapter, excluded from the product publish set. Use the repository example to build it. It hosts ordinary serve agents through `everruns::Engine`; the hosting plugin selects celld Durable Objects for session authority. Native containers and Wasm isolates use the shared Engine path; qualification covers the failure domains described below. Each tenant/session has its own authority object. That object commits creation, admission receipts, canonical history and execution epochs, and retains alarms for unfinished work. Native containers execute the registered Framework build. They can be replaced without restoring canonical snapshots or replaying accepted HTTP requests. ```rust #[tokio::main] async fn main() -> serve::Result { serve_celld::start(serve::App::builder().discover().build()).await } ``` Configure `AUTHORITY_URL`, `TENANT_ID`, `AUTHORITY_TOKEN` and a distinct `EXECUTOR_TOKEN` on the native host. Configure user API authentication separately with `SERVE_API_KEYS`. Startup refuses missing authority configuration. The Worker exposes the ordinary `/v1` routes and addresses authority objects by authenticated tenant and typed session identity. Keep creation and message IDs when retrying uncertain replies. Creation acknowledges the committed binding; a successful send acknowledges durable admission. Turn assignment and completion are observed separately. Disconnecting an event reader does not cancel work. `SERVE_WORKSPACE_DIR` reopens separate workspace directories by session. OS tools also need an explicit sandbox adapter through `serve_celld::Options::microvm`; remote workspace/compute services can be supplied through `Options::environment`. Commands must be isolated from the Engine process and its authority credentials. The adapter never enables an uncontained shell. Session journal durability does not preserve ephemeral container files. The local catalog/cache can be reconstructed from authority binding metadata. Channel inbox, outbox and scheduling guarantees require separately supported host services. The former `/cells/` routing and `/celld/snapshot`/`restore` protocol are removed. Existing deployments using that experimental protocol need an explicit migration; their snapshots are not the new authority format. See the [runnable example](https://github.com/everruns/everruns/tree/main/examples/serve/celld) for setup and deployment. The [isolate example](https://github.com/everruns/everruns/tree/main/examples/celld-engine) uses the same Framework Agent and Engine as Cloudflare, without a container or a celld-specific agent loop. Workerd tests and a local replicated fleet establish only their tested failure modes; production bucket consistency, alarms, build retention and workspace persistence remain deployment responsibilities. --- # Tutorial: Serve an Agent and Call It from Code > Build an agent with serve, protect it with an API key, and call it from Python, TypeScript or Rust with the SDK's AgentClient. Source: > **Experimental.** [serve](https://everruns.com/docs/framework/serve/) is a proof of concept. Its APIs will change and it has no compatibility promise. By the end of this tutorial you will have an order-status agent running on your machine as a serve app, with one tool, a real model and an API key in front of it, and a short program in Python, TypeScript or Rust that talks to it with the SDK’s `AgentClient`. This is a guided lesson, so it makes the choices for you. The [serve reference](https://everruns.com/docs/framework/serve/) covers everything it leaves out. ## What you need * Rust 1.96 or later, with `cargo`. * An OpenAI API key in `OPENAI_API_KEY`. The agent uses `openai/gpt-6.2-sol`. * For the client: Python 3.10 or later, Node.js 22 or later, or Rust. * `curl` and `jq` for the checks along the way. ## 1. Create the project ```sh cargo new order-desk cd order-desk mkdir -p agent src/tools ``` Replace the dependencies in `Cargo.toml`: ```toml [dependencies] everruns-serve = "0.48" tokio = { version = "1", features = ["full"] } [build-dependencies] everruns-serve-build = "0.48" ``` API keys need `everruns-serve` 0.48 or later. The package is `everruns-serve` and the library is imported as `serve`. Add a `build.rs` next to `Cargo.toml`, so the prompt in `agent/` and `serve.toml` are compiled into the binary: build.rs ```rust fn main() { serve_build::embed(); } ``` Name the app in `serve.toml`: ```toml name = "order-desk" ``` ## 2. Write the agent The prompt goes in `agent/instructions.md`: ```markdown You answer customer questions about their orders for an online shop. When someone asks about an order, call `order_status` with its number and answer in one or two sentences. If the tool finds no such order, say so and ask for the number again. Never guess a status. ``` The tool goes in `src/tools/order_status.rs`. The doc comments become the descriptions the model reads, and the function name becomes the tool name: ```rust use serve::prelude::*; /// Look up the status of an order by its number. #[tool] async fn order_status( cx: &Cx, /// The order number, for example 42. order: u32, ) -> Result { cx.progress(format!("looking up order {order}")).await; // A real app would query its order database here. match order { 42 => Ok("Shipped October 9 with UPS, tracking 1Z999AA10123456784, arrives October 13.".into()), 43 => Ok("Paid. Ships within two business days.".into()), _ => bail!("there is no order {order}"), } } ``` `src/tools/mod.rs` declares it: ```rust mod order_status; ``` Finally the agent and `main` in `src/main.rs`: ```rust mod tools; use serve::prelude::*; /// Answers questions about orders. #[agent] fn support() -> Agent { Agent::builder() .model("openai/gpt-6.2-sol") .instructions(md!("instructions.md")) .build() } serve::assets!(); #[tokio::main] async fn main() -> serve::Result { serve::start(serve::App::builder().discover().build()).await } ``` `#[agent]` and `#[tool]` register the function where it is defined, and `discover()` collects them, so nothing lists the tool by hand. The agent’s doc comment is its description, and its function name, `support`, is its name in URLs. ## 3. Run it ```sh export OPENAI_API_KEY=sk-... cargo run ``` The first build takes a few minutes. Then the app prints what it found: ```text serve · order-desk 0.1.0 · Dev · build b49b68ee33c43b82d ⚠ experimental proof of concept; APIs will change agent support (default) · openai/gpt-6.2-sol → openai tool order_status sandbox support · none data .serve listen http://localhost:3000 ``` `→ openai` says the model uses your OpenAI key and, if set, `OPENAI_BASE_URL`. `→ openrouter` uses `OPENROUTER_API_KEY`. An explicit OpenAI URL/key pair takes precedence for `openai/…` models. `simulator` means no real provider route is configured; see [Try it](https://everruns.com/docs/framework/serve/#try-it). `cargo run` with no command is `dev`: sessions are kept in SQLite under `.serve/`, the terminal shows each turn as it happens, and edits to `instructions.md` apply without a rebuild. ## 4. Talk to it with curl The agent’s API lives at its own base URL, `/v1/channels/support`. In a second terminal, read its card: ```sh curl -s localhost:3000/v1/channels/support ``` ```json {"name":"support","description":"Answers questions about orders.","streaming":true,"input":{"text":true,"images":true,"files":true},"auth":[],"links":{"sessions":"/v1/channels/support/sessions"}} ``` Start a session, send a message, and follow the session’s events: ```sh AGENT=localhost:3000/v1/channels/support ID=$(curl -s -X POST $AGENT/sessions -H 'content-type: application/json' -d '{}' | jq -r .id) curl -s -X POST $AGENT/sessions/$ID/messages -H 'content-type: application/json' \ -d '{"message":{"role":"user","content":[{"type":"text","text":"Where is order 42?"}]}}' curl -N "$AGENT/sessions/$ID/sse?after_sequence=0" ``` The stream shows the turn as server-sent events: `turn.started`, `tool.started` and `tool.completed` for the lookup, `output.message.delta` as the answer streams, and `turn.completed`. The stream stays open for the next turn; stop it with Ctrl+C. The first terminal shows the same turn: ```text 21:42:34 4cc33a ● new session on support 21:42:34 4cc33a ▸ Where is order 42? 21:42:36 4cc33a ⚙ order_status {"order":42} 21:42:36 4cc33a … looking up order 42 21:42:36 4cc33a ✓ order_status 21:42:38 4cc33a ◂ Order 42 shipped on October 9 via UPS and is expected to arrive October 13. Your tracking number is **1Z999AA10123456784**. ``` ## 5. Protect it with an API key Anyone who can reach port 3000 can use the agent so far. Stop the app with Ctrl+C, make a key, and start it in production mode with that key: ```sh export AGENT_KEY=$(openssl rand -hex 32) SERVE_API_KEYS=$AGENT_KEY cargo run -- start ``` `start` is production mode: a model with no route is an error instead of a simulator. `SERVE_API_KEYS` takes a comma-separated list, so each application can get its own key and you can rotate one at a time. Keep keys in the environment: `serve.toml` is compiled into the binary. The startup banner’s sample requests now carry an `authorization` header. Without a key, every session route answers `401`: ```sh curl -si -X POST localhost:3000/v1/channels/support/sessions -H 'content-type: application/json' -d '{}' ``` ```text HTTP/1.1 401 Unauthorized www-authenticate: Bearer ... {"detail":"a valid bearer credential is required","status":401,"title":"Unauthorized"} ``` With the key as a bearer token it works again: ```sh curl -s -X POST localhost:3000/v1/channels/support/sessions \ -H "authorization: Bearer $AGENT_KEY" -H 'content-type: application/json' -d '{}' ``` The card stays public so clients can find the agent, and its `auth` list now says which credential it takes: `"auth":[{"type":"agent_key"}]`. API keys are one of three methods; [Authentication](https://everruns.com/docs/framework/serve/#authentication) covers OpenID Connect and OAuth 2.0 introspection. ## 6. Call it from code Every Everruns SDK has an `AgentClient` for one agent’s API. It reads the agent’s base URL from `EVERRUNS_AGENT_URL` and the credential from `EVERRUNS_AGENT_KEY`, and sends the credential as a bearer token. In the terminal where you will run the client: ```sh export EVERRUNS_AGENT_URL=http://localhost:3000/v1/channels/support export EVERRUNS_AGENT_KEY=$AGENT_KEY # the key from step 5 ``` Pick one language. ### Python ```sh python3 -m venv .venv . .venv/bin/activate pip install "everruns-sdk>=0.3.1" ``` Save this as `ask.py`: ```python import asyncio from everruns_sdk import AgentClient async def main(): async with AgentClient() as agent: card = await agent.card() print(f"Agent: {card['name']}") session = await agent.create_session() print(await agent.run("Where is order 42?", session["id"])) print(await agent.run("And order 43?", session["id"])) asyncio.run(main()) ``` ```sh python ask.py ``` ```text Agent: support Order 42 shipped October 9 via UPS and is expected to arrive October 13. Your tracking number is **1Z999AA10123456784**. Order 43 is paid and will ship within two business days. ``` ### TypeScript ```sh npm init -y npm pkg set type=module npm install @everruns/sdk tsx ``` Save this as `ask.ts`: ```typescript import { AgentClient } from "@everruns/sdk"; const agent = new AgentClient(); const card = await agent.card(); console.log(`Agent: ${card.name}`); const session = await agent.createSession(); console.log(await agent.run("Where is order 42?", session.id)); console.log(await agent.run("And order 43?", session.id)); ``` ```sh npx tsx ask.ts ``` ### Rust ```sh cargo new ask cd ask cargo add everruns-sdk tokio --features tokio/full ``` Replace `src/main.rs`: ```rust use everruns_sdk::AgentClient; #[tokio::main] async fn main() -> Result<(), Box> { let agent = AgentClient::from_env()?; println!("Agent: {}", agent.card().await?.name); let session = agent.create_session(None, None).await?; println!("{}", agent.run("Where is order 42?", Some(&session.id)).await?); println!("{}", agent.run("And order 43?", Some(&session.id)).await?); Ok(()) } ``` ```sh cargo run ``` TypeScript and Rust print the same three lines as Python. `run` sends one message, follows the event stream until the turn ends, and returns the agent’s reply. Passing the session id keeps both questions in one conversation, which is why “And order 43?” needs no more context. Without a session id, `run` starts a new session each time. For more control, the client has the session calls directly: `create_session`, `send_message`, `stream_events`, `list_events`, `cancel`, and the answers to questions and tool approvals. ## What you built * A serve app with one agent and one tool, running on a real model. * An API key in front of the agent’s session routes, with the card left public. * A client that holds only that key and the agent’s URL. The same client code works against an agent hosted on Everruns: point `EVERRUNS_AGENT_URL` at the agent’s Agent API channel and use an agent key. See [Call your agent from code](https://everruns.com/docs/how-to/call-your-agent-from-code/) and the [Agent API](https://everruns.com/docs/features/agent-api/) reference. ## Next steps * Hold a tool call for a person’s approval with `#[tool(needs_approval)]`, and answer it with `submit_tool_approvals`. See [The pieces](https://everruns.com/docs/framework/serve/#the-pieces). * Check the agent’s behavior with `#[eval]` and `cargo run -- eval`. To run the evals against the protected app, set `SERVE_CREDENTIAL=$AGENT_KEY` and pass `--against http://localhost:3000`. * Run the app on [Amazon Bedrock AgentCore](https://everruns.com/docs/framework/serve-agentcore/), or keep its data in a bucket with [`start --store`](https://everruns.com/docs/framework/serve/#keep-the-data-in-a-bucket). --- # Application services > Compose GitHub, authenticated MCP, Slack, data services, native tools, triggers, traces, and evaluations without an Everruns server. Source: Framework applications can bind their own credentials, persistence, and retrieval services to the same operations the Server runs. The new APIs on this page are available from the repository’s `main` branch until the next crate release; run the linked examples from a source checkout or use the Git dependency shown in the [GitHub guide](https://everruns.com/docs/framework/github/). Enable only the Cargo features you use. Credentials stay in application-owned values and never enter an agent’s serialized configuration. | Priority | Feature | Framework API | Feature flag / guide | | -------- | ----------------------------------------------------------------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | 1 | GitHub operations and rotating application credentials | `.capability(GitHub)` | `github`; [GitHub tools](https://everruns.com/docs/framework/github/) | | 1 | MCP bearer credentials, OAuth login, refresh and rotation | `.mcp_auth(McpAuth)` | `mcp-auth`; [MCP authentication](https://everruns.com/docs/framework/mcp-authentication/) | | 1 | Slack reactions, message edits, user lookup and external file uploads | `.slack_actions(SlackActions)` | `slack-actions`; [Slack](https://everruns.com/docs/framework/slack-services/) | | 1 | Slack agent pane, streaming, status/title, approvals, progress, history and participation | `Channel::slack(Slack)` | `channels`; [Slack](https://everruns.com/docs/framework/slack-services/) | | 2 | Event filters, changes-only checks, durable dedupe/outbox and session routing | `triggers::Dispatcher` | `triggers`; [Event triggers](https://everruns.com/docs/framework/event-triggers/) | | 2 | Named agent coordination | `.member(Agent)` | [Coordinator agents](https://everruns.com/docs/framework/coordination/) | | 2 | Cursor, Browserless and image generation | `.cursor(...)`, `.browserless(...)`, `images` | `cursor`, `browserless`, `images`; [Native services](https://everruns.com/docs/framework/native-services/) | | 3 | Persistent, session-scoped SQL | `.sql_database(SqlDatabase)` | `sql-database`; [Data services](https://everruns.com/docs/framework/data-services/) | | 3 | Named read-only or writable directory memory | `.memory(MemoryDirectory)` | [Data services](https://everruns.com/docs/framework/data-services/) | | 3 | Application knowledge bases and citable retrieval indexes | `.knowledge(Knowledge)` | `knowledge`; [Data services](https://everruns.com/docs/framework/data-services/) | | 3 | E2B sandbox create/list/execute/files/pause/resume/delete | `.e2b(...)` | `e2b`; [Native services](https://everruns.com/docs/framework/native-services/) | | 4 | Shared event-to-turn/step projection | `trace::Trace::project` | [Evaluation and traces](https://everruns.com/docs/framework/evaluation-and-traces/) | | 4 | Shared scorers, bounded evaluation runs and online scoring | `evaluation::Evaluator`, `OnlineScoring` | `evaluation`; [Evaluation and traces](https://everruns.com/docs/framework/evaluation-and-traces/) | These are application execution features. Cloud account selection, integration management screens, managed indexing/sync workers, deployment scheduling, and Server reporting APIs remain host responsibilities. The default Framework dependency graph stays offline. Each guide links a complete example that uses a real model. For model-free verification, acceptance tests use scripted models against real Engine sessions and mock vendor endpoints: ```bash cargo test -p everruns --features github,mcp-auth,sql-database,knowledge,slack-actions,channels,triggers,evaluation --test service_parity --test github_framework ``` ## Shared implementations and host boundaries Server and Framework execute the shared operations below. Each host supplies its own identity, storage and lifecycle adapters; sharing an operation does not make the two hosts’ persistence or permission models interchangeable. | Priority | Feature | Shared implementation | Host-owned behavior | | -------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | GitHub | `everruns-integrations::github`: the same schemas and complete vendor operations | Server resolves permitted connections; applications provide per-operation credentials and write opt-ins | | 1 | MCP transport and OAuth | `everruns-core::mcp`: discovery, registration, PKCE, authorization parameters, callback validation, code exchange and refresh | Hosts select requested scopes and registration policy. Server owns browser callbacks, organization-scoped egress, encrypted records and replica locks; applications own callback delivery and token stores | | 1 | Slack actions and delivery surfaces | `everruns-integrations::slack_channel`: API operations, uploads, Markdown, streams and agent-pane surfaces | Hosted routing, durable delivery recovery and authorization; application conversation bindings, participation and consent policies | | 2 | Triggers | `everruns-core::triggers` and `triggers_github`: filters, changes-only comparisons, templates and GitHub normalization | Server persists hosted deliveries and routing; Framework’s dispatcher owns its SQLite outbox and admission recovery | | 2 | Named members | Core’s delegate and session execution paths | Applications register named agents; Server resolves agent records and permissions | | 2–3 | Cursor, Browserless, images and E2B | Existing native capabilities, vendor clients and tool operations | Credentials, network policy, sandbox ownership and cleanup | | 3 | Session SQL | `everruns::database::runtime`: executor, authorizer, validation and limits; Server reexports these modules | Local directory storage versus hosted page/object storage, tenant checks and transactions | | 3 | Memory | Core file-store and mount interfaces and the existing memory tools | Local named directories versus hosted storage, sharing and access policy | | 3 | Knowledge | Existing Knowledge tools, retrieval contracts and citation types | Application lexical/custom retrievers versus managed ingestion, indexing and hosted retrieval; automatic retrieval persistence | | 4 | Traces | `everruns-core::session_trace`: one event-to-turn/step projection | Framework validates a complete event prefix; Server maintains its persisted incremental projection | | 4 | Evaluation | Core scoring predicates, file-content grading, citation evidence, judge prompts and output parsing | Evidence loading, unavailable-evidence policy, model selection, billing, result storage and worker scheduling; application-specific graders | OAuth supports both public-client registration and confidential-client registration through the same protocol implementation. Discovery retains Server’s protected-resource root fallback and issuer-path metadata fallbacks. Shared client errors retain recognized OAuth classifications without provider bodies, token values or transport URLs. Inbound Everruns `/oauth/*` authorization-server endpoints serve a different role from this outbound MCP client. The [cross-host OAuth contract tests](https://github.com/everruns/everruns/blob/main/crates/server/src/api/user_connections/tests/shared_oauth_contract.rs) exercise embedded and browser login against the same endpoints. [Refresh integration tests](https://github.com/everruns/everruns/blob/main/crates/server/src/storage/connection_resolver/tests/shared_protocol.rs) verify that Server’s two-replica lock still persists a rotation before releasing callers. These tests complement the Engine-level acceptance suites above. --- # Session Controls > Run slash commands, hot-install capabilities, switch model, reasoning effort and MCP servers, and query session context on a live Engine session. Source: A host that drives a `Session`, such as a terminal UI, a print mode or an ACP server, needs more than send and wait. It runs slash commands, installs extensions while the session is live, switches model or reasoning effort, adds MCP servers, and reviews what the agent did. `Session` exposes those operations directly. None of them hands out a runtime, starts a turn or writes to the conversation history. Stability: experimental. ## Commands Capabilities contribute slash commands. Every frontend of one session sees the same list, and `run_command` dispatches against it with the context the next model call will use: the same provider, message filters, capability instructions and filesystem. ```rust use everruns::{Agent, Engine, OpenAI}; let agent = Agent::builder() .instructions("You maintain this repository.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .capability("btw") .build()?; let session = Engine::new().create(agent); for command in session.commands().await? { println!("/{} {}", command.name, command.description); } // `/btw` answers a side question without interrupting the main task. let output = session.run_command("/btw", Some("which files changed?")).await?; println!("{}", output.message); ``` `run_command` runs out of band: no message is appended and no turn starts. It is safe to call from a tool of the running turn, which is how a model-facing `run_command` tool reaches the same registry. An unknown command returns `SessionControlError::Rejected`. ## Hot-installed capabilities Registering code and activating it are separate steps. `Engine::register_capability` makes a trusted capability (written against the Core `Capability` contract from [host services](https://everruns.com/docs/framework/host-services/)) resolvable for every session the Engine runs. It changes no model surface until a session activates it. ```rust use everruns::hosting::services::Capability; use everruns::{CapabilityRef, Engine, Session, SessionControlError}; engine.register_capability(extension)?; session .activate_capability(CapabilityRef::new("my_extension")) .await?; // Its tools, instructions and commands are part of the next model call. session.run_command("/my-command", None).await?; session.deactivate_capability("my_extension").await?; // Absent from the next model call; its commands are gone too. ``` Activation validates the configuration and dependencies before anything changes, and repeating it is a no-op. A capability the Agent declares cannot be deactivated at the session layer. Registration never overrides: an id or alias the Framework catalog or another registration already serves is rejected. The activation is the session’s persisted capability overlay. Capability code is never persisted, so after a restart register it again before resuming sessions that use it, as you re-attach the Agent. ## Model, reasoning effort and MCP servers ```rust use everruns::{McpServer, Model, ReasoningEffort, Session}; session.set_model("gpt-6-luna").await?; session.set_reasoning_effort(Some(ReasoningEffort::High)).await?; session .add_mcp_server(McpServer::http("docs", "https://mcp.example.com/mcp")) .await?; // Unavailable selections fail and leave everything as it was. assert!(session.set_model(Model::on("not-configured", "x")).await.is_err()); ``` A plain model id selects a model of the Agent’s provider; `Model::on` names a provider the Agent or its host provider services already serve. A model value that bundles its own provider is refused: a live selection never changes credentials or provider code, and nothing falls back to another provider. Changes apply from the next model call, including later calls of a turn that is running. `reset_model` returns to the Agent’s model, and `refresh_mcp` rediscovers MCP tools after a credential refresh. To show per-server MCP status, reconnect a single server, or follow readiness changes, see [MCP status, reconnect and doctor](https://everruns.com/docs/framework/host-services/#mcp-status-reconnect-and-doctor). ## Queries for host review `command_context` returns the same projection as `inspect` (model, filtered messages, tools, instructions and commands) but answers during an active turn too. `complete_out_of_band` runs a tool-less completion over that context with the session’s provider and selections, for checks such as reviewing whether a task was finished: ```rust let verdict = session .complete_out_of_band(Some("Answer yes or no."), "Was the request completed?") .await?; ``` When the host owns the review policy and its token budget, `complete` sends a request the host curates instead. The provider receives exactly its system prompts and messages, with no tools. The session adds none of its own instructions or conversation, and no message filters or model views apply. What the session does contribute is what it owns: the provider and model its next model call would use (live selections included), the Agent’s call settings, attachment resolution and provider errors. ```rust use std::time::Duration; use everruns::{CompletionRequest, ReasoningEffort}; let verdict = session .complete( CompletionRequest::new() .system_prompt("Decide whether the task is complete. Answer yes or no.") .user(evidence) .purpose("completion_review") .reasoning_effort(ReasoningEffort::Low) .timeout(Duration::from_secs(60)), ) .await?; ``` To review with filtered session context, read `command_context` and reduce it into your own bounded messages. A request carries 1 to 64 user or assistant messages and up to 8 system prompts. System, tool-call and tool-result messages are rejected. * **Metadata.** `purpose` and `metadata` add provider metadata: up to 8 entries, keys of lowercase ASCII letters, digits, `_`, `-` or `.`, and values up to 256 bytes. Keys the session stamps from its own authority (`session_id`, `harness_id`, `turn_id`, `exec_id`, `org_id`, `agent_id`, `model_id`) are rejected, not overwritten. When the host also sets request metadata for every model call, the request’s keys win over the host’s, and `session_id` always comes from the session. * **Reasoning effort.** Without `reasoning_effort` the request uses the session’s current selection. A per-request effort applies to that request only, and an effort the model’s curated profile cannot honor is rejected. * **No fallback.** A selected provider the session can no longer serve fails the request instead of reaching another provider. * **Timeouts.** `timeout` ends the request with `SessionControlError::TimedOut`. Dropping the future abandons it as well. `complete_out_of_band` remains the convenience form: it is a request whose system prompts are the session’s instructions plus yours, and whose messages are the filtered conversation plus your prompt. None of these append to history, start a turn, take execution ownership or resume interrupted work, and all of them are answered during an active turn, including from a tool of that turn. `history()` remains the bounded way to page the canonical conversation. ## Where the state lives | Setting | Coherent hosting (default Engine, `EngineBuilder::hosting`) | Custom backends and local profiles | | -------------------------- | ----------------------------------------------------------- | -------------------------------------------- | | Capability overlay | Committed through the hosting authority; durable | Persisted in the session record | | Model and reasoning effort | Committed through the hosting authority; durable | Held by the Engine; re-apply after a restart | | MCP servers | Held by the Engine; apply from the next activation | Held by the Engine; re-apply after a restart | | Registered capability code | Held by the Engine; register again after a restart | Same | MCP definitions can carry credentials, so they are never committed to session state. On coherent hosting, commands run on the query runtime that inspection uses; a change a command makes to the capability overlay is committed, and other session-record writes are not retained. ## Related * [Host services](https://everruns.com/docs/framework/host-services/) for capabilities and providers every session of an Agent starts with. * [Session hosting](https://everruns.com/docs/framework/hosting/) for durable session authority under Engine. ## Inspect compaction `session.inspect().await?.compaction` exposes the effective strategy, soft working-history token budget, proactive setting, history-retrieval availability and saved-checkpoint metadata. It reads existing state without generating a summary, starting a turn or exposing checkpoint contents. The public `CompactionStatus`, `ContextCheckpoint` and `CompactionEventSummary` types are available from `everruns`. The built-in harness presets include compaction. With the default `builtins` feature, sessions without an explicit harness also get managed continuity; agent capability configuration can override its defaults. Custom harnesses choose their own capabilities and must include `compaction` when desired. Restart recovery requires persistent hosting: `MemoryHosting` checkpoints last only as long as its memory store. See [compaction](https://everruns.com/docs/advanced/compaction/) for native-provider compatibility, portable continuity, costs and configuration. --- # Sessions > Keep conversation history across turns, read and resume Framework sessions, and choose how session state persists. Source: An `Agent` is immutable reusable behavior. An `Engine` owns session identity, history, and runtime state. A `Session` is an engine-bound live conversation. ```rust use everruns::{Agent, Engine, OpenAI}; let agent = Agent::builder() .instructions("Remember the conversation.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let engine = Engine::new(); let session = engine.create(agent); let first = session.send_and_wait("My project is Atlas.").await?; let second = session.send_and_wait("Continue with that project.").await?; assert!(first.success); assert!(second.success); ``` `Session` is always a live conversation. `send` accepts a message without waiting for a response. If a turn is active, the message steers that turn; if the previous turn has already finished, it starts a follow-up turn. Admission returns before the executor assigns a turn. The receipt reports `Pending`; query its authoritative assignment to learn whether it started or steered a turn: ```rust let session = engine.create(agent); let initial = session.send("Plan my trip.").await?; let latest = session.send("Prefer trains.").await?; let result = latest.wait().await?; if let Some(assignment) = latest.assignment().await? { println!("turn: {}, steered: {}", assignment.turn_id, assignment.steered); } ``` Waiting on the latest receipt works before or after assignment. `send_and_wait` (also available as the shorter `run` alias) is request/response convenience over the same live session, not a separate mode. The first asynchronous operation materializes the in-process host. Later turns reuse it and send accumulated history through the same context-assembly path. Two sessions opened on one engine have different opaque IDs and isolated histories. The engine retains each immutable Agent snapshot, so a session keeps working after the original Agent handle is dropped. Volatile resume is engine-scoped: another `Engine` rejects the id rather than guessing its configuration. A local profile can be attached to another Engine with the trusted Agent snapshot; live Engines for the same profile share its backend bundle. Conversation isolation does not imply filesystem isolation. The concise `engine.create(agent)` path permanently selects the Agent’s default head before its first inspection or turn; call `session.start().await` to make that selection observable earlier. To fix a session to an isolated project view, bind an [`Environment`](https://everruns.com/docs/framework/workspaces-and-environments/) before execution. A session can never switch heads after it starts. `Session::inspect` returns the context assembled for the next model call. Use it for application assertions and debugging rather than reaching into runtime records or backend stores. Keep `Session::session_id()` when the application may need to reopen a conversation. [History and resume](#history-and-resume) below covers typed resume, bounded transcript pages, and cursor snapshots, and [Persistence](#persistence) covers engine-lifetime memory and the crash-durable local profile. See [Workspaces and Environments](https://everruns.com/docs/framework/workspaces-and-environments/) for exact-head resume, isolation, sharing, and lifecycle, and [Events and cancellation](https://everruns.com/docs/framework/events-and-cancellation/) to observe a turn in flight. ### Send files `Session::attach` keeps a file in the session’s own files at `/attachments//`, read-only, and returns the part that sends it with a message. The model gets images and PDFs as content and other files by name and path, and can open any of them again with `read_file`, including after compaction. A zip is unpacked into a read-only folder next to it. ```rust,no_run use everruns::InputMessage; let report = session .attach("report.pdf", Some("application/pdf"), std::fs::read("report.pdf")?) .await?; let mut message = InputMessage::user("Summarize the attached report."); message.content.push(report); session.run(message).await?; ``` The bytes decide the file’s type, whatever the caller declared. Embedded channels keep the files people send (a Slack file, for example) the same way. ## History and resume Every Framework session has a typed `SessionId`. Keep that value when an application may need to reopen the conversation: ```rust use everruns::{Agent, Engine, OpenAI, SessionId}; let agent = Agent::builder() .instructions("Remember the conversation.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .build()?; let engine = Engine::new(); let session = engine.create(agent); let session_id: SessionId = session.session_id(); session.send_and_wait("My project is Atlas.").await?; drop(session); let resumed = engine.resume(session_id).await?; resumed.send_and_wait("Continue with that project.").await?; ``` `resume` verifies the ID against the engine’s session catalog. It does not infer identity from a non-empty transcript: a valid session can have no messages, and stray events do not create a resumable session. An unknown ID returns a typed not-found error. The resumed session uses the immutable Agent snapshot attached to that engine; it never reconstructs behavior from events. ### Read bounded history `Session::history` creates an owned query. Calling `page` returns at most 100 messages by default in canonical event-sequence order, oldest first: ```rust let page = session.history().page().await?; for message in &page.messages { println!("{:?}: {}", message.role, message.text()); } ``` Set a smaller or larger page size with `limit`. The maximum is 256 messages; an excessive value returns `HistoryError::InvalidLimit` with the allowed maximum. A page never claims to contain the entire transcript. Continue from its opaque cursor: ```rust let first = session.history().limit(25)?.page().await?; if let Some(cursor) = first.next_cursor { let second = session.history().limit(25)?.after(cursor)?.page().await?; // `second` continues the same stable snapshot. } ``` `HistoryCursor` is opaque, session-bound, and safe to store as a string with `Display` and restore with `FromStr`. A cursor fixes the snapshot’s high-water mark: events appended after the first page do not appear midway through that page walk. Start a new query to see them. Passing a malformed, cross-session, expired, or incompatible cursor returns a distinct typed history error. History projection also applies a bounded raw-event replay safety limit; an unusually lifecycle-heavy snapshot that exceeds it returns `HistoryError::HistoryTooLarge` instead of performing an unbounded scan. For callers that intentionally walk the whole snapshot, `pages` is a lazy convenience that still reads one bounded page at a time: ```rust let mut pages = session.history().limit(50)?.pages(); while let Some(page) = pages.next_page().await? { for message in page.messages { println!("{}", message.text()); } } ``` After the final page, `next_page` remains fused and returns `None`. It does not re-read the backend or produce repeated empty terminal pages. ## Persistence Framework history is a read-only projection of canonical events. Normal execution has one write path, the engine’s event log, so a resumed session and a running session cannot disagree about the conversation. | Deployment | Conversation state | Recovery boundary | Use when | | --------------------------- | ----------------------------------------------- | ------------------------------------------ | -------------------------------------------- | | `Engine::new()` | Volatile coherent memory authority | One process | Embedding, tests and short-lived tools | | Engine with `SqliteHosting` | Transactional local session authority | The configured disk plus native supervisor | Durable single-node Framework services | | Engine with `ObjectHosting` | Per-session object authority | Platform storage and alarms | Native or Wasm execution on Cloudflare/celld | | Legacy `LocalConfig` | Local canonical events and catalog | One trusted application process | Existing local-store integrations | | Everruns Platform | PostgreSQL canonical events and actor authority | Distributed server and workers | Horizontal workers and remote clients | ### Default: engine-lifetime memory By default, `Engine` owns a volatile session catalog and event log. It retains the immutable Agent snapshot associated with each session and requires no database, server, network connection, credential, or filesystem access. Dropping a `Session` does not immediately discard its committed history. Reopen it by passing its typed `SessionId` to the engine that created it. A separate engine cannot infer the session’s Agent configuration, and process exit loses volatile history. This default fits tests, command-line tools, short-lived workers, and applications that deliberately own a higher-level record elsewhere. ### Coherent native and Durable Object hosting For durable admission and independently recoverable execution, choose a coherent session authority on Engine. Native `hosting::SqliteHosting` owns one transactional database; the [session hosting SPI](https://everruns.com/docs/framework/hosting/) supports external authorities for native or Wasm Engine placement. A source-only Durable Object reference demonstrates the extension. Register immutable `AgentBuild` identities and retained Environment factories before creating sessions. Start the native supervisor or configure platform alarms so recovery does not depend on a client reconnecting. See [Durable Object hosting](https://everruns.com/docs/framework/durable-objects/). `create_registered` and `send_with_id` accept caller-retained identities for retry after uncertain responses. Admissions, approvals, client results and cancellation are authority commands; the journal and wakeup obligation commit together. Resources such as workspace files and compute still require explicit host services. ### Checkpoints and rewind `Engine::checkpoints(session_id)` (experimental) returns a handle for conversation checkpoints on a branch-aware authority such as `SqliteHosting`. A host can build checkpoint commands, a restore picker or a conversational restore tool on it alone: ```rust use everruns::checkpoints::RestoreMode; use everruns::hosting::{AgentBuild, SqliteHosting}; use everruns::{Agent, Engine, Model, SessionId}; let build = AgentBuild::new("assistant", "v1"); let agent = Agent::builder() .instructions("Be concise.") .model(Model::simulated("Done.")) .build()?; let engine = Engine::builder() .hosting(SqliteHosting::open("authority.db")?) .register(build.clone(), agent) .build(); let id = SessionId::new(); let session = engine.create_registered(id, &build).await?; let checkpoints = engine.checkpoints(id); session.send_and_wait("Plan the migration.").await?; let saved = checkpoints.create_labeled("plan agreed").await?; session.send_and_wait("Now rewrite it in Go.").await?; // Show the user what a restore would undo, then restore. let preview = checkpoints.preview(saved.id).await?; for turn in &preview.dropped { println!("undo: {}", turn.input.as_deref().unwrap_or("")); } let restored = checkpoints.restore(saved.id, RestoreMode::Branch).await?; // The undone turns stay addressable: redo brings them back. if let Some(undone) = &restored.undone { checkpoints.redo(undone).await?; } ``` * `create` marks the end of the last finished turn, so a tool can call it while its own turn runs. `list`, `get` and `preview` read without changing anything, and `delete` forgets a checkpoint. * A `CheckpointSummary` carries its `id`, `turn_index`, the `message_id` that started its last turn, `created_at`, an optional label and a display-safe `preview`. `on_active_branch` is `false` for a checkpoint left on an undone branch; restoring it brings those turns back. * `RestorePreview` lists the turns that leave context (`dropped`) and the turns a restore brings back from another branch (`restored`), with exact totals. Text excerpts are short and have control and bidirectional characters removed, so they are safe to show in a terminal or a single UI line. * `RestoreMode::Branch` keeps the undone turns as a branch named in `RestoreOutcome::undone`, which `redo` reselects. Restoring to a point a retained branch already holds reselects that branch instead of adding one. * `RestoreMode::Truncate` drops the current branch from the branch catalog, and checkpoints that only it held are no longer listed. For a checkpoint on the current branch that is just the undone turns. For a checkpoint with `on_active_branch == false` it is the whole current branch, including the turns before the point where the two branches split. * A session keeps at most 1024 branches, the active one included. Once a `Branch` restore would exceed that, it fails; delete retained branches you no longer need with `delete_branch`, or restore with `Truncate`. `delete_branch` never deletes the active branch, and checkpoints only the deleted branch held are no longer listed. A session also keeps at most 1024 checkpoints, not counting ones no branch reaches; `delete` frees room. Nothing is deleted from the journal: a restore changes which branch is the conversation, so context, history and new input follow it while raw exports keep every event. Checkpoints and branches survive a restart and `Engine::resume`. Restore and redo need a quiescent session, with no running turn, pending wait or unconsumed input; otherwise they fail without changing anything, so a restore tool should apply its choice after its turn ends. The default volatile engine refuses checkpoints. Checkpoints cover the conversation only: workspace files are not rewound. ### Local: crash-durable events This is the legacy local-storage lane. New durable applications should use `SqliteHosting` and its supervisor, described above. The feature-gated `LocalConfig` adds a crash-durable event log under the configured application data directory. It also supplies a trusted real-disk workspace plus SQLite-backed task and schedule state: ```rust use everruns::{Agent, Engine, LocalConfig, OpenAI}; let local = LocalConfig::new(".everruns-data").workspace("./workspace"); let agent = Agent::builder() .instructions("Work inside the configured workspace.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .local(local) .build()?; let engine = Engine::new(); let session = engine.create(agent); ``` Enable it with `cargo add everruns --features local`. Select both directories from trusted application configuration. After a restart, rebuild the Agent from trusted application configuration, attach it to a new engine, and resume the committed session by ID. For a session created with an explicit Harness, also deserialize its portable definition and call `Engine::attach_with_harness`; `Engine::attach` remains the no-Harness path. The local profile is designed for one embedded process at a time. Coordinate process ownership before handing the directory to another application process. Within one process, every live Engine configured with the same local data directory shares one backend bundle, so concurrent Engine values cannot build divergent JSONL indexes or SQLite handles for that profile. The event-log file format and host backends are not Framework APIs. Do not edit the log or build application writes around its representation. Use `Session::history` for bounded reads and `Engine::resume` to continue a session; see [History and resume](#history-and-resume) for the complete lifecycle. Applications remain responsible for filesystem permissions, backups, retention, and selecting a data directory that is not controlled by model or request input. New local state files are created owner-only on Unix, but applications must still protect copied files and backups. Message content is application data and may be sensitive even though provider credentials are not written there by Framework configuration. ### Resume after a process restart Enable `local` and configure a trusted application data directory when sessions must survive a new Agent or process: ```rust use everruns::{Agent, Engine, LocalConfig, OpenAI}; let build_agent = || -> Result> { Ok(Agent::builder() .instructions("Remember the conversation.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .local(LocalConfig::new(".everruns-data")) .build()?) }; let first_engine = Engine::new(); let session = first_engine.create(build_agent()?); session.start().await?; let session_id = session.session_id(); // In a later process, rebuild trusted behavior before resuming persisted state. let restarted_engine = Engine::new(); restarted_engine.attach(session_id, build_agent()?).await?; let resumed = restarted_engine.resume(session_id).await?; ``` The local profile stores a durable session catalog and crash-durable canonical event log alongside its workspace, task, and schedule state. After restarting, build another Agent with the same trusted data directory, call `engine.attach(session_id, agent)`, then `engine.resume(session_id)`. Attachment rejects IDs absent from that Agent’s configured local catalog. A new session is made durable by its first async operation (`run`, `inspect`, or a history page read); merely allocating a synchronous handle does not commit it. A running turn, or one waiting on a person (a tool approval or an `ask_user` question), lives inside the process, so a process exit leaves it in the log without an end. After resuming, `resumed.interrupted_turn()` reports such a turn and the tool calls it had not finished, and `resumed.resume_interrupted_turn()` continues it in the same turn. A call runs again only when that is safe: it waited on a person, which asks the approver or `ask_user` responder again under the same tool call id, its tool is marked `FunctionTool::idempotent()`, or it never started (the log has no `tool.started` for it). Any other unfinished call is recorded as interrupted (listed in `not_rerun`), so the model learns its outcome is unknown and nothing runs twice. A turn cut off between steps reasons again. For a session created with an explicit Harness, persist its serialized portable definition, deserialize it after restart, and call `engine.attach_with_harness(session_id, agent, harness)` instead. Harness deserialization validates the definition and generates a new process-local runtime identity. The local profile is for one embedded process at a time. Do not write or edit its files as application data: messages are a read-only projection of committed events, and the storage formats are not Framework APIs. History does not contain model credentials or application secrets unless an application deliberately includes them in message content or event metadata. Choose and protect the local data directory accordingly. ### Canonical host persistence Durable conversation truth belongs to canonical events; history and context are projections of that record. Advanced hosts use `EventLog` and `EventHistory` from `everruns-core` (`host` feature), including `JsonlEventLog` when a local append-only event log is appropriate. Framework applications continue sessions with `Engine::resume` and traverse bounded event-derived pages from `Session::history`. A host that needs its own storage implements the public `EventLog`/`EventReader` SPI and supplies it through `HostBackends::with_event_log`; see [Implementing a custom event log](https://everruns.com/docs/framework/events-and-cancellation/#implementing-a-custom-event-log). `JsonlEventLog` bounds startup recovery before indexing: the default accepts at most 128 MiB and 1,000,000 canonical events. Oversize logs fail to open with a typed recovery-limit error instead of allocating or scanning without bound. Do not design new application persistence around a legacy storage representation. ### Moving legacy sessions to `SqliteHosting` `Engine::cut_over_legacy_session` (experimental) moves an existing `LocalConfig` or custom-backend session into `SqliteHosting` with the same `SessionId`. The import is lossless and atomic: the full event history keeps its event ids and sequences, and admissions, parked client calls and approvals, compaction checkpoints, session values, finished task records and the workspace binding come along. Retrying after a crash or a lost reply returns the same result. A cutover to a different build fails. `cut_over_legacy_session_with_metadata` also records binding metadata the legacy lane never stored, such as the `parent_session_id` of a child session. ```rust let profile = SqliteHosting::open("authority.db")?; let engine = Engine::builder().hosting(profile.clone()).build(); engine.register_agent_with_environment( build.clone(), agent.clone(), None, "imported-workspace", ImportedWorkspaceFactory::new(agent, profile), )?; let report = engine.cut_over_legacy_session(session_id, &legacy_agent, &build).await?; let session = engine.resume(session_id).await?; ``` The legacy lane is frozen first. After that, `Engine::attach` with the legacy Agent returns `ResumeError::CutOver` and no legacy turn starts, in any process that shares the store. The JSONL file is never written again; when a tool still needs JSONL, rebuild it from the authority with `SqliteHosting::rebuild_jsonl_export(session_id, path)`, which writes the same line format atomically and can be repeated at any time. If a cutover fails before the authority commits, the session stays frozen until you retry or call `Engine::abandon_legacy_cutover`. The session must be quiescent. A cutover refuses, without writing anything to the authority, while a legacy turn is running or while the session has a task still running under its legacy owner. Schedules move with the session and fire from `SqliteHosting::supervise`; the legacy lane no longer delivers them. Tasks whose owner is gone move with the session; the authority re-attaches or settles them on the next activation without re-running uncertain work. Session secrets move only when the authority has secret keys (`SqliteHosting::with_secret_keys`, the same `key_id:base64_key` format as `SECRETS_ENCRYPTION_KEY`); they are stored encrypted and the cutover refuses without keys. It also refuses when the target build cannot reopen the session’s workspace head. Use `ImportedWorkspaceFactory` so the authority reopens that head after restarts. The Agent’s in-memory default workspace is not durable on either lane, so its files are not migrated; the report sets `volatile_workspace`. ### Rewinding and branching history `SqliteHosting` keeps every event forever and tracks which events form the conversation (experimental). A branch is a list of event-sequence ranges made of whole turns. The model context, `Session::history` and `Session::events_after` follow the active branch. `SqliteHosting::export_jsonl` still returns every event with its original id and sequence. New input extends the active branch, so sequences on a branch can have gaps. ```rust let lineage = engine.history_lineage(session_id).await?; // Rewind to just after sequence 12; the old branch stays retained. let ranges = lineage.rewind_ranges(12).expect("12 is on the active branch"); engine .select_history_branch(session_id, command_id, lineage.revision, "undo-1", Some(ranges)) .await?; // Redo: reselect the retained branch by name. engine .select_history_branch(session_id, redo_id, lineage.revision + 1, "main", None) .await?; ``` A selection is accepted only when the session is quiescent: no execution, no open turn, no pending tool call or approval, and no unconsumed input. It also checks the revision you read. Anything else is refused, never queued or discarded. Retrying the same command id returns the first reply. Compaction checkpoints follow the branch too. A checkpoint can be used only on a branch that shares the history it summarized, so a summary of an abandoned branch never shows up on another branch. Checkpoints of a shared earlier part stay usable. `MemoryHosting` and other authorities without branch support refuse these calls. To import saved history that already has branches, convert your own timeline format into a `ConversationLineage` and pass it with `Engine::cut_over_legacy_session_with_options(.., CutoverOptions::new().lineage(lineage))`. The whole journal is imported. The lineage names the active branch, the retained branches and each branch’s compaction checkpoints. A retry must pass the same lineage. ### One process runs a session at a time The following lease behavior describes the legacy `LocalConfig` actor lane, not coherent Engine hosting or Platform workers. In that lane, each session runs as an actor: its event log is its state, and a turn holds the session’s lease while it runs. The lease is taken when a turn starts, renewed while it runs (including while it waits on an approval or a question), and given up when the turn ends. With the local profile the leases live in the profile’s SQLite database, so if two processes share a data directory, a turn started in one while the other runs the same session fails with an error saying the session runs in another process. Between turns no process holds a session, so either one can pick it up. A lease lasts 10 seconds unless renewed. After a crash, application-driven recovery can acquire the session once that lease expires; this is not a ten-second end-to-end recovery guarantee or an independent restart supervisor. Coherent Engine hosting instead uses authority-fenced execution epochs and independent activation. A parked approval, question or client-side tool does not retain execution. See [session hosting](https://everruns.com/docs/framework/hosting/). ### Platform: distributed durable execution The Everruns Platform uses the same `everruns-core` (`engine` feature) turn state machine and turn loop as the Framework. The server stores each input and marks the session runnable; a worker leases the session over gRPC and runs the whole turn, and PostgreSQL stores the canonical events. If a worker disappears mid-turn, its lease expires and a later worker resumes the turn from the session’s event log. The Platform is a deployment boundary, not another configuration mode on `everruns::Engine`. Remote applications use the Platform API or an SDK; product hosts compose the lower-level durable crates. See [Framework Architecture](https://everruns.com/docs/framework/architecture/) for the layer map. --- # Slack services > Use Slack-native action tools and advanced signed Slack channels in standalone Framework applications. Source: Slack action tools and Slack channels are independently optional. Action tools let an agent act in a conversation your application authorizes. Channels receive signed Slack events and deliver agent replies. ## Native tools Enable `slack-actions`. Bind credentials and the authorized destination outside serialized agent configuration: ```rust use everruns::{Agent, Model, slack::SlackActions}; let actions = SlackActions::new( std::env::var("SLACK_BOT_TOKEN")?, "C012345", "1730000000.000001", ); let agent = Agent::builder() .instructions("Use Slack tools to help in the authorized conversation.") .model(Model::new("gpt-6.2-sol", everruns::OpenAI::from_env()?)) .slack_actions(actions) .build()?; ``` This enables `slack_add_reaction`, `slack_update_message`, `slack_lookup_user`, and `slack_upload_file`. The operations and tool schemas are shared with Server. Uploads use Slack’s external upload reservation, byte transfer, and completion flow; the retired `files.upload` method is not used. Channel arguments cannot redirect the bound credential. Uploads must use the configured thread. Message edits/reactions in a thread verify target membership; user lookup returns only display identity fields. Upload URLs are constrained to Slack or an explicitly configured trusted API origin, redirects are refused, and signed upload URLs do not enter error messages. The [native actions example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/slack_actions.rs) reacts to a thread root and uploads a file through a real agent. Set `OPENAI_API_KEY`, `SLACK_BOT_TOKEN`, `SLACK_CHANNEL_ID` and `SLACK_THREAD_TS`: ```bash cargo run -p everruns --features openai,slack-actions --example slack_actions ``` ## Advanced channels Enable `channels` and configure a Slack app with event subscriptions, a signing secret, and the scopes required by the enabled operations. The driver verifies Slack’s signature over the original body and enforces its five-minute replay window. Empty explicitly configured signing secrets are rejected. ```rust use everruns::channels::{Channel, Channels, Slack, SlackAdmission}; let slack = Slack::new( std::env::var("SLACK_BOT_TOKEN")?, std::env::var("SLACK_SIGNING_SECRET")?, ) .workspace(std::env::var("SLACK_TEAM_ID")?) .agent_pane() .task_progress() .approval_buttons() .thread_history(30) .participation(SlackAdmission::Direct); let channels = Channels::builder(engine) .channel("support", Channel::slack(slack), agent) .build(); ``` Use `Channel::slack` to select delivery per turn: the Slack agent pane streams replies and shows status/title; ordinary channels receive completed messages. Streaming supports close, cancellation cleanup and replacement when a guardrail retracts text. Task progress edits one message in place. Approval buttons answer `request_approval` questions in the conversation. They are one-use opaque cards, expire after fifteen minutes, are superseded by new conversation input or another card, and may be answered only by the requesting Slack user in the original channel. They do not grant arbitrary durable tool-approval authority. Outstanding cards are process-local; after a restart, the person can answer in the thread or the application can request another card. Thread history seeds a new session without starting extra turns, excludes the triggering message/bots, and has a hard bound of 100 messages. Participation runs before session creation: use `SlackAdmission::Direct`, implement `Participation`, or use `SlackRelevance::new(decisions)` for an explicitly selected decision model. Relevance times out after two seconds and declines on errors; direct messages and mentions pass. Pane stop/title/context events route to the existing conversation. Latest title/view state is available through `Channels::conversation_state`; view data accompanies the next user input as untrusted reference data. Supply `ConversationStore` for durable metadata, `ChannelStore` for durable bindings/pending delivery, and a durable Engine profile for sessions. The defaults are process-local. Mount `Channels::handle` on your HTTP framework, preserving the raw body and signature headers. Keep intake within Slack’s three-second acknowledgement deadline. `Slack::from_env` permits unsigned requests when no signing secret exists for local experiments; production applications should use the explicit constructor and require both secrets. The [complete HTTP example](https://github.com/everruns/everruns/blob/main/crates/everruns/examples/slack_channel.rs) mounts both Events API and Interactivity on `/slack`, requires workspace pinning and verifies raw request signatures: ```bash cargo run -p everruns --features openai,channels --example slack_channel ``` Set the four variables named at the top of that source file and configure Slack’s URLs through an HTTPS ingress. The example uses process-local session/binding state. For a durable Engine, pre-register its `AgentBuild` and use `ChannelsBuilder::channel_registered`; re-register that exact build after restart together with durable channel/conversation stores. --- # Testing and Simulation > Test Framework applications deterministically without network access or provider credentials. Source: `Model::simulated` is the default testing tool. It uses the normal provider resolution and execution path while returning a fixed response locally. Its implementation comes from the publishable, production-safe `everruns-llmsim` crate; the Framework does not depend on test-support code. ```rust use everruns::{Agent, Engine, Model}; let agent = Agent::builder() .instructions("Return the configured result.") .model(Model::simulated("approved")) .build()?; let session = Engine::new().create(agent); let turn = session.send_and_wait("Review this.").await?; assert!(turn.success); assert_eq!(turn.response, "approved"); ``` Useful test layers are: 1. Build-time validation tests for agent, tool, model, MCP, and compaction configuration. 2. Offline session tests with `Model::simulated`. 3. Context assertions through `Session::inspect`. 4. Event/cancellation tests through the public session API. 5. A small opt-in live-provider suite for protocol integration. Keep normal tests credential-free and deterministic. Do not make a live model’s wording or tool choice a unit-test oracle. Temporary directories should own workspace and local-state tests so they do not read or modify developer data. The runnable programs in [Framework examples](https://everruns.com/docs/framework/examples/) are also compiled in CI using only the public `everruns` facade. ## Scripted and low-level simulation Use `Model::simulated_with_config` when an application test needs multiple assistant turns, deterministic tool calls, an injected provider error, or request capture: ```rust use everruns::{Agent, LlmSimConfig, Model}; use everruns_llmsim::{SimToolCall, SimTurn}; let simulation = LlmSimConfig::scripted(vec![ SimTurn::ToolCalls(vec![SimToolCall { name: "lookup".into(), arguments: serde_json::json!({"id": 7}), id: Some("call_lookup".into()), }]), SimTurn::Assistant("approved".into()), ]); let agent = Agent::builder() .instructions("Use lookup, then report the result.") .model(Model::simulated_with_config(simulation)) .build()?; ``` Advanced hosts depend on `everruns-llmsim` with its `host` feature for `LlmSimRuntimeExt`. The `.llm_sim(...)` method registers the provider without changing model selection; `.llm_sim_as_default(...)` explicitly selects it when no default was already configured. Use `everruns-test-support` only for testing/demo helpers such as its in-memory agentic loop, writable fixtures, test doubles, and fake capabilities. --- # Upgrade Notes > Breaking changes for Framework users, release by release, with the code changes each upgrade needs. Source: This page collects the changes a Framework application must make when it upgrades across a release that moved or renamed public Rust APIs. Releases that need no code changes are not listed. For every release, see the [changelog](https://github.com/everruns/everruns/blob/main/CHANGELOG.md). ## 0.49 ### `local()` and `backends()` are deprecated `AgentBuilder::local` and `AgentBuilder::backends` select the older facade lane, which recovers less after a crash than Engine hosting. They still work and now emit a deprecation warning. New code builds the Engine with a hosting profile instead, and existing sessions move over with [`Engine::cut_over_legacy_session`](https://everruns.com/docs/framework/sessions/#moving-legacy-sessions-to-sqlitehosting): ```diff let agent = Agent::builder().local(LocalConfig::new(dir)) /* … */ .build()?; let engine = Engine::new(); let agent = Agent::builder() /* … */ .build()?; let engine = Engine::builder().hosting(SqliteHosting::open(dir.join("authority.db"))?).build(); ``` AG-UI threads move with `AgUiThreads::hosted`, and coordination threads run as child sessions of the hosting authority. Sessions written by an older lane still move one by one with `Engine::cut_over_legacy_session`. ### Custom task registries honor recovery guards `SessionTaskUpdate` adds `expected_recovery`. Code constructing updates with `..Default::default()` needs no change; complete struct literals add `expected_recovery: None`. A custom `SessionTaskRegistry::update` must compare a supplied guard against the persisted task under the same lock or transaction as the write. On a mismatch, return `Ok(None)` before applying the update. Returning the unchanged task as a successful update would let a stale recoverer publish a completion notification. The local SQLite and server PostgreSQL registries enforce this contract. ### MCP calls take an idempotency key `McpClient::call`, `McpClient::call_as_tool_result` and `McpTransport::call_tool` take a new last argument, the call’s idempotency key, which goes in the request’s `_meta` as `io.everruns/idempotency-key`. Pass `None` to send none. A custom `McpTransport` adds the parameter; a tool reads the current call’s key from `ToolContext::idempotency_key`. ```diff client.call(&connection, "search", args).await?; client.call(&connection, "search", args, None).await?; ``` ## 0.48 ### Durable turns option removed The experimental `durable` feature of the `everruns` crate and `everruns::durable::Backend` are removed, with no replacement flag. Every `Engine` session now runs as an actor that holds the session’s lease while a turn runs, and durability comes from the session’s event log: with `LocalConfig` a turn a restart cut off resumes from the log. Drop `durable` from the crate’s features and the `.backend(durable::Backend::...)` call from `Engine::builder()`. See [Sessions](https://everruns.com/docs/framework/sessions/#one-process-runs-a-session-at-a-time). ```toml # Before everruns = { version = "0.47", features = ["durable"] } # After everruns = { version = "0.48", features = ["local"] } ``` ### Durable crates * `everruns-durable` no longer has a `workflows` feature: `Workflow`, `Activity`, `WorkflowExecutor`, timers and child workflows are removed. The task queue, event log, schedules, retries, dead letter queue and circuit breakers stay. Remove `features = ["workflows"]`. * `everruns-durable-engine` is retired with no new home. Its published versions remain usable. ### Harness tree The built-in harnesses form a tree by use case instead of one chain. Base now carries the system essentials, Conversation and Worker both build on it, and Bashkit Worker builds on Worker. See [Harnesses](https://everruns.com/docs/features/harnesses/). * `Harness::worker()` no longer includes a shell. It is the worker kit (files, project instructions, skills, long context, budgeting, subagents, tasks) with no compute. Use `Harness::bashkit_worker()` for the previous files-and-bash behavior. * `Harness::worker_base()` is deprecated and returns `Harness::bashkit_worker()`. * `Harness::base()` is no longer empty: it adds compaction, error disclosure, tool-call repair, loop detection and parallel tool calls. Build a custom harness with `Harness::builder` for an empty foundation. * Agent packages may name `bashkit-worker`; `worker-base` still loads and binds Bashkit Worker. Hosted organizations are migrated once on upgrade. The old Worker included a shell, so everything bound to it moves to the worker that matches its compute: * Agents with a container or managed Agent sandbox policy move to Sandbox Worker and keep their policy. * Every other Agent, the org default, custom child harnesses, triggers and sessions move to Bashkit Worker. A Bashkit-only Agent sandbox policy is cleared because Bashkit Worker fixes it. * Existing sessions keep the compute they started with. * `worker-base` stays as a deprecated row with its old capabilities, so custom harnesses that name it keep working. ## 0.46 ### Integration-catalog package retired `everruns-integrations-catalog` published its final deprecated forwarding release at 0.45.0. Its source is removed from the workspace and there is no 0.46.0 release of that package. When upgrading platform dependencies to 0.46, use `everruns-capabilities::integrations_catalog` with `hosted-integration-catalog`, as described below. Hosted registration and integration features are preserved; previously published shim versions remain usable and are not yanked. ## 0.45 ### Hosted integration registration moves into capabilities Replace the `everruns-integrations-catalog` dependency with `everruns-capabilities` and enable its `hosted-integration-catalog` feature. Change imports from `everruns_integrations_catalog::` to `everruns_capabilities::integrations_catalog::`. The public registration API, integration set, registration order, and deployment-grade feature gates are preserved. The old package shipped its deprecated forwarding release at 0.45.0 before source retirement in 0.46. Its published versions remain usable. Framework applications using `everruns` and its selected integration features need no changes. The hosted integration set remains opt-in. ## 0.42 ### SQLite connections move to `everruns::sqlite` The embedded SQLite wrapper moved out of `everruns-durable` into the `everruns` facade, so the `local` feature no longer compiles the durable engine. `everruns-durable`’s `sqlite` feature and `sqlite` module are removed. Replace `everruns-durable = { features = ["sqlite"] }` with `everruns = { features = ["local"] }` and `everruns_durable::sqlite` with `everruns::sqlite`; the functions and types are unchanged. `everruns_durable::UpdateField` still names `UpdateField`; the `everruns_durable::update_field` module is no longer public. ## 0.41 ### Embedded SQLite ownership `everruns::local::SqliteDb` retains its shared callback API, including callbacks typed as `&rusqlite::Connection`, `rusqlite::params!`, and `OptionalExtension`. Connections now open through `everruns-durable`’s optional `sqlite` feature. Framework applications keep the same local stores and schemas; no data migration is needed. Hosts opening independent handles should use `everruns_durable::sqlite::open` or `open_in_memory` (`everruns::sqlite` from 0.42). To add application tables to the Framework database, retain `SqliteDb` and use its `with_conn` callbacks. ### Retired forwarding crates The five forwarding packages `everruns-engine`, `everruns-host`, `everruns-builtins`, `everruns-mcp`, and `everruns-ag-ui` are removed from source after their single deprecated 0.40 release. Keep the canonical core imports and feature selections described below; published deprecated versions remain usable. ## 0.40 ### Consolidated execution modules The planned release moves the execution, host, built-in, MCP, AG-UI, and A2A modules into `everruns-core`. Its default features remain portable; opt into the modules your host uses. The five former packages will ship one deprecated forwarding release before removal in the next platform release. Existing behavior and protocol formats will be preserved. This step is gated on the complete 0.39 publication, including its deprecated `everruns-platform` shim, before that shim is removed. | Deprecated crate prefix | Canonical import | Core feature | | ----------------------- | --------------------------- | ------------ | | `everruns_engine::` | `everruns_core::engine::` | `engine` | | `everruns_host::` | `everruns_core::host::` | `host` | | `everruns_builtins::` | `everruns_core::builtins::` | `builtins` | | `everruns_mcp::` | `everruns_core::mcp::` | `mcp` | | `everruns_ag_ui::` | `everruns_core::ag_ui::` | `ag-ui` | After upgrading, select concrete integrations on `everruns`, and use `everruns::batteries::runtime_capability_registry()` and `everruns::batteries::runtime_egress_service()` for the matching composition. The core host module accepts explicitly injected services. A custom shell hook host supplies a `BashHookDispatcherFactory` through `HostBackends`; the facade injects the Bashkit factory when its `bashkit` feature is enabled. Utility-model bootstrap configuration will live in `everruns::utility_llm` behind `utility-llm`. A2A protocol and outbound client APIs will be available from `everruns_core::a2a` with the `a2a` feature. ## 0.39 ### Hosted capabilities and server records Replace the `everruns-platform` dependency with `everruns-capabilities` and keep the hosted capability features your application uses. Change hosted registry and runtime-store imports from `everruns_platform::` to `everruns_capabilities::`. The platform name receives one deprecated forwarding release before removal. Library hosts use `everruns_core::AgentDefinition`, `HarnessDefinition`, and `ExecutionSession` for execution configuration and session views. Persisted agent, harness, session, model, provider, skill, and other management records are owned by the server and are no longer exposed by a library crate. Store adapters resolve authorization, versions, inheritance, and lifecycle before returning portable values. Extensions continue to implement the service interfaces in `everruns-contracts`. HTTP API paths and serialized responses are unchanged by this ownership move; HTTP clients need no migration. ## 0.38 ### Runtime store migration Custom hosts implementing `everruns_platform::PlatformStore` now return `everruns_core::AgentDefinition`, an inheritance-resolved `everruns_core::HarnessDefinition`, `everruns_core::ExecutionSession`, and `everruns_contracts::typed_id::SessionParticipantId`. Resolve authorization, record projection, harness inheritance, and lifecycle validation in the host adapter. The raw `get_harness_chain` method is removed. `PlatformStoreSubagentDelegate` remains available to attach this runtime store to a `ToolContext`; the stock `SubagentCapability` needs no host-specific override. `LocalPlatformStore` directly forwards its runner’s portable session views. The deprecated provider, capability, and model-profile shims are removed after their final release. Depend on `everruns-contracts`; replace `everruns_provider::` with `everruns_contracts::`, `everruns_capability::` with `everruns_contracts::capability::`, and `everruns_model_profiles::` with `everruns_contracts::model_profile_data::`. ### Harness levels `Harness::base()`, `Harness::conversation()`, `Harness::worker()` and `Harness::bashkit_worker()` share the same capability definitions as the hosted harnesses. `Harness::generic()` is deprecated; it keeps its previous capabilities instead of silently switching to Worker. Choose the smallest foundation and add agent-specific capabilities. Enable the required host integrations separately. Existing `engine.create(agent)` behavior is unchanged when no harness is bound. On the hosted platform, upgrades pin existing agents that inherited a Generic org default to Generic before selecting Conversation as the new default. Explicit bindings, sessions and custom defaults remain unchanged. The upgrade runs once; a later explicit choice to use Generic as the org default is respected. If a custom harness already uses a new built-in name, it keeps its ID, display name, configuration and bindings. Its addressable name becomes `-custom`, with a numeric suffix if needed. Update API or CLI calls that select that custom harness by name; calls using its ID keep working. ## 0.18 0.18 narrows `everruns-core` to the neutral execution kernel. Types that were persisted control-plane records, hosted service contracts, product composition or concrete integrations moved to the crate that owns them. The behaviour, the wire formats and the stored schema are unchanged, only the import paths. ### Retain an Engine for sessions The Framework exposes a concrete application execution owner. The 0.18 API removes `agent.session()` and `agent.resume(id)`; applications retain the engine that owns session identity and resume authority: ```diff let session = agent.session(); use everruns::Engine; let engine = Engine::new(); let session = engine.create(agent); let id = session.session_id(); let resumed = agent.resume(id).await?; let resumed = engine.resume(id).await?; ``` `Engine` volatile resume is deliberately process-local and engine-scoped. The old `InMemoryEngine` name remains a type alias. Engine retains the immutable Agent snapshot and exact Environment/WorkspaceHead; it does not serialize a Scale-compatible Agent definition. For a locally persisted session after process restart, rebuild the Agent from trusted application configuration, call `engine.attach(id, agent).await?`, and then `engine.resume(id).await?`. Attachment verifies the persisted session catalog before accepting the behavior snapshot; credentials and closures are never serialized. This affects you if your Rust code imports from `everruns_core` directly. If you use the `everruns` facade, most of this is invisible: the facade re-exports what applications need, and where a moved type is part of that surface it is re-exported from its new home under the same name. ### The quickest path Most migrations are a find-and-replace of a crate prefix. Compile, read the unresolved-import errors, and look each symbol up in the tables below. ```bash cargo build 2>&1 | grep -E "unresolved import|no .* in" ``` Add whichever crates the table points you at: ```toml everruns-platform = "0.18" # persisted records, hosted service contracts everruns-host = "0.18" # execution composition and host wiring everruns-provider = "0.18" # provider SPI, typed IDs, sqlx impls everruns-capability = "0.18" # capability identity/configuration contract everruns-mcp = "0.18" # MCP adapter and the OAuth protocol client everruns-llmsim = "0.18" # deterministic production-safe simulator ``` The earlier `everruns-session-services` preview package was consolidated into `everruns-host`; use `everruns_host::session_services` for its namespaced API. ### Composition The single biggest change for embedders. `PlatformDefinition` no longer exists. | 0.17 | 0.18 | | ------------------------------------------------------ | --------------------------------------------------- | | `everruns_core::PlatformDefinition` | `everruns_host::HostComposition` | | `everruns_core::PlatformDefinitionBuilder` | `everruns_host::HostCompositionBuilder` | | `everruns_server::oss_platform_definition()` | `everruns_server::oss_host_composition()` | | `everruns_server::oss_platform_definition_for_grade()` | `everruns_server::oss_host_composition_for_grade()` | | `everruns_worker::default_platform_definition()` | `everruns_worker::default_host_composition()` | | `ServerAppBuilder::platform_definition(..)` | `ServerAppBuilder::host_composition(..)` | | `WorkerAppBuilder::platform_definition(..)` | `WorkerAppBuilder::host_composition(..)` | ```diff use everruns_core::PlatformDefinition; use everruns_host::HostComposition; let platform = PlatformDefinition::builder() let composition = HostComposition::builder() .capability_registry(capabilities) .driver_registry(drivers) .build(); ServerAppBuilder::new().platform_definition(platform) ServerAppBuilder::new().host_composition(composition) ``` The type is otherwise identical, same fields, same builder methods. It moved to the layer that executes a turn, because selecting a deployment’s capabilities and drivers is composition rather than kernel configuration. #### Input/Reason/Act execution kernel Concrete phase execution moved out of `everruns-core`. Import phase algorithms and their I/O values from `everruns-engine`; keep neutral effect contracts in core. | 0.17 | 0.18 | | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------- | | `everruns_core::atoms::{InputAtom, InputAtomInput, InputAtomResult}` | `everruns_engine::{InputAtom, InputAtomInput, InputAtomResult}` | | `everruns_core::atoms::{ReasonAtom, ReasonInput, ReasonResult}` | `everruns_engine::{ReasonAtom, ReasonInput, ReasonResult}` | | `everruns_core::atoms::{ActAtom, ActInput, ActResult, ToolCallResult}` | `everruns_engine::{ActAtom, ActInput, ActResult, ToolCallResult}` | | `everruns_core::atoms::AtomContext` | `everruns_core::ExecutionContext` (also re-exported by `everruns-engine`) | | `everruns_core::atoms::{PreToolUseHook, PostToolExecHook, PreToolUseDecision}` | `everruns_core::tool_hooks::*` | The generic `Atom` trait was removed; it had no production dynamic-dispatch use. Call the concrete executor’s inherent async `execute` method. There is no core compatibility module in 0.18. Serialized phase payloads retain the same fields, so durable records remain readable even though the Rust ownership path changed. #### Turn context and command completion Store-backed turn preparation now belongs to `everruns-host`. Core keeps the secret-free execution snapshot, pure context transformations, and narrow effects used by custom hosts. | 0.17 | 0.18 | | ---------------------------------------------------- | ---------------------------------------------------- | | `everruns_core::assemble_turn_context` | `everruns_host::assemble_turn_context` | | `everruns_core::inspect_turn_context` | `everruns_host::inspect_turn_context` | | `everruns_core::load_execution_snapshot` | `everruns_host::load_execution_snapshot` | | `everruns_core::load_execution_snapshot_for_session` | `everruns_host::load_execution_snapshot_for_session` | | `everruns_core::StoreCommandHost` | `everruns_host::StoreCommandHost` | `everruns_engine::ReasonAtom::new` no longer accepts harness, agent, session, and provider stores or a driver registry. Construct an `everruns_host::StoreTurnContextResolver` from those host services, then pass that resolver plus the narrow message, capability, and event effects to the atom. Hosts that already loaded a `ResolvedExecutionSnapshot` should call `everruns_host::assemble_turn_context_from_snapshot` and execute the atom with the resulting `AssembledTurnContext`; this avoids a second store load. For a fully custom host, implement the neutral `everruns_core::TurnContextResolver`, or provide already-resolved `ResolvedTurnContextInput` to `everruns_core::assemble_resolved_turn_context`. That input contains a secret-free model/provider identity and an opaque ready driver; provider keys and endpoints are never serializable kernel values. `CommandTurnContext` now exposes `session_id` directly instead of an `ExecutionSession`. Commands retain the same filtered messages, effective prompt, locale, model, streaming, and error-decision behavior without receiving a session record. ### Persisted records These are database and API records. Execution consumes a portable projection of each; the stored row is control-plane state. | 0.17 (`everruns_core::`) | 0.18 | | ----------------------------------------------------------------------------------------- | -------------------------------- | | `Agent`, `AgentVersion`, `AgentStatus`, `AgentVersionChangeKind` | `everruns_platform::` | | `Harness`, `HarnessStatus`, `BuiltInHarnessDefinition`, `BuiltInHarnessRole` | `everruns_platform::` | | `Session`, `SessionStatus`, `SessionSource`, `SessionActivity`, `SessionParticipant` | `everruns_platform::` | | `Workspace`, `WorkspaceStatus` | `everruns_platform::workspace::` | | `Eval`, `EvalCase`, `EvalRun`, `EvalCaseResult`, `EvalRunDataset`, `EvalTarget`, `Scorer` | `everruns_platform::` | | `Observer`, `ObserverMatch`, `LlmJudgeConfig`, `TraceScore` | `everruns_platform::` | | `FeatureFlags`, `FeatureFlagMap`, `FeatureFlagDefinition` | `everruns_platform::` | | `Budget`, `LedgerEntry` | `everruns_platform::` | If you were reading a stored record to run a turn, you probably want the portable projection instead, `AgentDefinition`, `HarnessDefinition` and `ExecutionSession` all stay in `everruns_core`, produced at the platform loading boundary by `Agent::execution_definition`, `Harness::execution_definition` and `Session::execution_session`. ### Hosted service contracts | 0.17 (`everruns_core::`) | 0.18 | | ------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | | `session_sqldb::*`, `SessionSqlDbStore`, `DatabaseInfo`, `SqlQueryResult`, `SqlExecuteResult`, `TableSchema`, `ColumnSchema`, `SessionSqlDbError` | `everruns_platform::session_sqldb::` | | `traits::SessionMutator` | `everruns_host::SessionMutator` (also re-exported by platform) | | `session_sandbox::*`, config, state, instance, exec/file payloads, `SessionSandboxProvider`, `SessionSandboxProviderPlugin` | `everruns_platform::session_sandbox::` | | `Connector`, `ConnectorRegistry`, `ConnectorPlugin` | `everruns_platform::connector::` | | `EmailSender`, `EmailMessage`, `SystemEmailConfig`, `ResendEmailSender` | `everruns_platform::email::` | | `OAuthClient`, `TokenSet`, `PkcePair` | `everruns_mcp::oauth::protocol::` | Neutral per-turn contracts remain in `everruns-core`, but the catch-all `everruns_core::traits` module is gone. Import from the owning concern instead, for example `everruns_core::tool_context::ToolContext`, `everruns_core::session_files::SessionFileSystem`, or `everruns_core::provider_resolution::ProviderStore`. The deployment-owned `SessionFileSystemFactory` and its context now come from `everruns-host`. Two of these also changed how a capability *reaches* the service. `sqldb_store` and `session_mutator` are no longer fields on `ToolContext`; they resolve from the type-keyed extension bag: ```diff let Some(store) = &context.sqldb_store else { ... }; let Some(store) = context.extensions.get::() else { ... }; let store = &store.0; ``` If you implement a custom host, install them the way `everruns-host` does: ```rust extensions.insert(Arc::new(SessionSqlDbStoreExt(store))); extensions.insert(Arc::new(SessionMutatorExt(mutator))); ``` ### Capabilities and implementations | what | 0.18 home | | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | | Knowledge Bases and Indexes, Memories, delegation, subagents, background and scheduled work, user hooks, citations, model scouting, platform management | `everruns_platform::capabilities::` | | Session info and session storage | `everruns_host::session_services::capabilities::` (also re-exported by platform) | | Session SQL database and session sandbox | `everruns_platform::capabilities::` | | `spawn_background` and its runtime, event sink, admission permits, reattach | `everruns_platform::background_run::` | | Portable built-ins, human intent, infinity context, skills, UI prompts, compaction, tool search | `everruns_builtins::` | | `everruns_openui::{PromptOptions, default_library, generate_prompt}` | `everruns_builtins::openui::{PromptOptions, default_library, generate_prompt}` | | `everruns_a2ui::{PromptOptions, default_catalog, generate_prompt}` | `everruns_builtins::a2ui::{PromptOptions, default_catalog, generate_prompt}` | | OpenRouter workspace, model scout, and provider-executed server tools | `everruns_integrations_openrouter::` | | Filesystem, shell, web fetch, Lua | `everruns_integrations_*` | | MCP adapter | `everruns_mcp::` | | In-process HTTP egress transport | `everruns_host::DirectEgressService` | | Telemetry init, exporter event listeners, `CompositeEventListener` | `everruns_host::observability::` (feature `observability`) | | `llmsim` driver, configs, scripted turns, registry helpers, host-builder extension | `everruns_llmsim::` | | in-memory agentic loop, writable test doubles, fixture capabilities | `everruns_test_support::` | | `everruns_core::in_memory::{InMemoryAgentStore, InMemoryHarnessStore, InMemorySessionStore, InMemoryProviderStore}` | `everruns_host::{InMemoryAgentStore, InMemoryHarnessStore, InMemorySessionStore, InMemoryProviderStore}` | | `everruns_core::in_memory::{InMemoryMessageRetriever, InMemoryEventEmitter}` | `everruns_test_support::{InMemoryMessageRetriever, InMemoryEventEmitter}` for isolated deterministic tests | Product presets compose these explicitly. Core registries are now empty by default: use `everruns_host::runtime_capability_registry()` for the Framework preset or `everruns_platform::capabilities::hosted_capability_registry()` for the hosted product catalog. Hosted conversation history has no writable message-store replacement. Append canonical events through `everruns_host::EventLog` / `HostEventEmitter` and read messages through `EventHistory`. This avoids message/event dual writes and keeps resume and replay behavior identical across in-memory and durable hosts. `everruns-test-support` continues to re-export its 0.17 simulator paths during the 0.18 migration, so existing test suites can upgrade without an immediate import rewrite. Treat that as a migration bridge: production code, new tests, and low-level hosts should depend on `everruns-llmsim` directly. The application-facing `everruns::Model::simulated` and `Model::simulated_with_config` APIs are unchanged. ### Features | 0.17 | 0.18 | | -------------------------------------- | ------------------------------------------------------------------------- | | `everruns-core/sqlx` | removed, use `everruns-provider` with `features = ["sqlx"]` | | `everruns-core/embedded-platform-docs` | removed, it gated nothing; use `everruns-platform/embedded-platform-docs` | | `everruns-platform/sqlx` | removed, it forwarded to core’s and nothing enabled it | | `everruns-core/llm-tests` | removed, use the `everruns-llm-tests` package for live provider tests | `everruns-core` now has an empty default feature set. OpenAPI derives remain available only with `features = ["openapi"]`; structural outlines remain available only with `features = ["tree-sitter-outlines"]`. Neither subtree is present in a default core build. Concrete provider protocol and utility-model implementations also moved to their effectful owners: | 0.17 (`everruns_core::`) | 0.18 | | ------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- | | `OpenAIProtocolChatDriver`, `openai_protocol` | `everruns_contracts::` | | `OpenResponsesProtocolChatDriver`, `openresponses_protocol` | `everruns_contracts::` | | `driver_helpers`, `stream_reconnect` | `everruns_contracts::` | | `OpenAiUtilityLlmService` (now `ProviderUtilityLlmService`), `SystemUtilityLlmConfig`, `UTILITY_OPENAI_API_KEY_ENV` | `everruns_host::` with `features = ["utility-llm"]` | Core no longer initializes Rustls. Provider HTTP clients install the workspace crypto provider when they are first constructed, while server, worker, and CLI startup owners install it eagerly. Custom binaries that combine TLS stacks can depend on `everruns-provider` with `features = ["tls-aws-lc-rs"]` and call `everruns_contracts::install_default_crypto_provider()` once during startup; the call is idempotent and safe under concurrent initialization. ### Provider and typed-ID imports Provider-owned modules are no longer compatibility-exported by `everruns-core`. Low-level consumers must add `everruns-provider` directly. This keeps credentials and concrete driver assembly out of the neutral kernel and makes the dependency owner visible in `Cargo.toml`. There are two common compiler-error shapes: 1. **The module moved to another crate.** Add that crate and change the prefix. 2. **The module stayed public, but its root convenience re-export was removed.** Keep the dependency and qualify the symbol through its module. The second case produces the misleading-looking `no X in the root` error. It does not necessarily mean the type moved. These replacements are deliberately literal so they can be applied with ordinary search-and-replace: | before | after | | ---------------------------------- | ---------------------------------------------------- | | `everruns_core::ProviderStore` | `everruns_core::provider_resolution::ProviderStore` | | `everruns_core::SessionStore` | `everruns_core::execution_loading::SessionStore` | | `everruns_core::MessageRetriever` | `everruns_core::message_retriever::MessageRetriever` | | `everruns_core::SessionFileSystem` | `everruns_core::session_files::SessionFileSystem` | | 0.17 core path | 0.18 direct path | | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | | `everruns_core::driver_registry::*` | `everruns_contracts::driver_registry::*` | | `everruns_core::model::*` | `everruns_contracts::model::*` | | `everruns_core::model_profiles::*` | `everruns_contracts::model_profiles::*` | | `everruns_core::model_spec::ModelSpec` | `everruns_contracts::model_spec::ModelSpec` | | `everruns_core::provider::*` | `everruns_contracts::provider::*` | | `everruns_core::runtime_provider::*` | `everruns_contracts::runtime_provider::*` | | `everruns_core::typed_id::*` | `everruns_contracts::typed_id::*` | | `everruns_core::error::*` | `everruns_contracts::error::*` | | `everruns_core::tool_types::*` | `everruns_contracts::tool_types::*` | | `everruns_core::capability_types::{CapabilityId, CapabilityRef, CapabilityError}` | `everruns_contracts::capability::{CapabilityId, CapabilityRef, CapabilityError}` | | `everruns_core::AgentCapabilityConfig` | `everruns_contracts::capability::CapabilityRef` | | core plugin capability ID/validation helpers | the same symbol in `everruns_capability` | | `everruns_core::ExecutionPhase` or `message::ExecutionPhase` | `everruns_contracts::execution_phase::ExecutionPhase` | | `everruns_core::ToolResultImage` or `tools::ToolResultImage` | `everruns_contracts::tool_types::ToolResultImage` | | other root-level provider symbols | the same root symbol in `everruns_provider` | The credential-bearing `everruns_core::ResolvedModel` is removed. Store and transport boundaries now resolve two separate values: * `ModelSpec`, safe to serialize and pass through the kernel; and * a host-owned runtime `Provider` (or internal `ProviderConfig`) containing endpoint and authentication state. `ProviderStore::get_model_spec` and `get_default_model_spec` return only the first value. Hosts obtain provider configuration separately and join it only while constructing a non-serializable driver/provider execution value. `ProviderStore::get_provider_config` no longer has a default implementation. Every custom host must state where credentials live: return its resolved `ProviderConfig`, or explicitly return `None` when the provider was registered directly in the host registry or is selected but not configured. A missing credential no longer prevents turn-context/command assembly; the constructed driver rejects the first model/list/compact operation locally, before network I/O. This keeps recovery commands reachable without turning an empty token into an outbound authorization header. ### Simulator and compaction naming `LlmSimRuntimeExt::llm_sim` now only registers/replaces the simulator provider. It never changes the selected model. Existing compact test setups that relied on implicit selection should use the explicit name: ```diff builder.llm_sim(config) builder.llm_sim_as_default(config) ``` When a builder already calls `default_model(...)`, keep `.llm_sim(config)`; the selected model is preserved regardless of method order. `everruns_builtins::CompactionConfig` is the sole application-facing policy builder. The expanded implementation value previously available as `everruns_builtins::compaction::CompactionConfig` is now `everruns_builtins::compaction::RuntimeCompactionConfig` (also re-exported at the crate root). This makes an unqualified `CompactionConfig` unambiguous. The public core test/backend conveniences are gone as well: | removed core value | replacement | | ----------------------------------- | ------------------------------------------------------------------- | | `EchoTool`, `FailingTool` | define the small test `Tool` locally, or use test-support executors | | `InMemoryCompactionCheckpointStore` | `everruns_host::InMemoryCompactionCheckpointStore` | ### What deliberately did not move Worth knowing so you do not go looking: * **`SessionTask`, `TaskMessage`** and the task registry stay in `everruns_core`. They are turn-execution vocabulary, `wake_queue` decides mid-turn wakes from a task’s wake policy, and they appear in the canonical `task.created` / `task.updated` / `task.message.*` event payloads. * **`SessionSchedule`, `SessionScheduleStore`** stay. A portable built-in (`usage_limit_auto_continue`) schedules an auto-resume after a provider usage limit, and it sits below platform in the dependency graph. * Schedule quota and minimum-interval environment variables are no longer read by core. Local/server adapters resolve deployment policy and call the parameterized core validation helpers. * **`SessionResourceRegistry`** stays. `resource_ownership` and the portable skills capabilities consume it. * **`SessionFileSystem`, `SessionStorageStore`** and the other neutral store contracts stay. Core owns the contract; hosts own the backend. The rule these follow: whether something belongs in the kernel is decided by whether a portable execution path consumes it during a turn, not by whether it is persisted. All four above are persisted, and all four are essential for execution. ### Getting unstuck If a symbol is not in these tables, import it from the crate that defines it; `everruns-core` no longer acts as a compatibility facade for provider-owned APIs. Framework applications can continue to prefer the higher-level `everruns` facade. The crate-level docs on `everruns-core` record where each remaining family lives and why. --- # Build a Voice Agent > Let people talk to a Framework agent and hear its answers, in your own Rust host or as a serve app, with the same agent, tools and transcript as text. Source: A voice agent is an ordinary agent with a **voice channel** in front of it. The channel listens to the caller, turns each finished utterance into a user message, and speaks the agent’s answer as it streams. The agent itself does not change: the same session can take typed messages and calls, and both land in one transcript. > **Status:** experimental. `everruns::voice` carries no compatibility promise yet, and the speech provider is OpenAI. ## Two modes Your agent, with its own model and tools, writes every answer in both modes. They differ in what listens and speaks, and in where the audio goes. | Mode | Listens and speaks | Audio path | Use it for | | ----------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------- | | **Delegated** (default) | A speech-to-speech model, `gpt-realtime-2` | Browser to OpenAI over WebRTC; never through your server | Browser calls with the lowest latency | | **Cascaded** | A speech-to-text model (`gpt-4o-transcribe`) and a text-to-speech model (`gpt-4o-mini-tts`) | Through your process as PCM | Native and mobile apps, your own transport, picking each model separately | ## How a delegated call works 1. The browser sends a WebRTC offer with its microphone to your server. Your server places the call with OpenAI and returns the SDP answer, so the API key never reaches the browser. Audio then flows directly between the browser and OpenAI. 2. When the caller stops talking, the transcript becomes a user message with `metadata.source = "voice"`. If a turn is already running, the message steers it, as a typed message would. 3. The answer is spoken sentence by sentence while it streams. A filler line (“One moment.”) covers a turn that stays silent for a while. 4. When the caller talks over the answer, speech stops at once. With `Interruption::Cancel` the running turn is cancelled too. This is the same voice loop the Everruns server runs for its [voice channels](https://everruns.com/docs/features/voice/), so a call behaves the same in your host, in serve, and on the platform. ## In your own host ```bash cargo add everruns --features voice,openai ``` ```rust use everruns::providers::openai::OpenAI; use everruns::voice::{Interruption, VoiceChannel}; let voice = VoiceChannel::delegated(OpenAI::from_env()?.realtime()) .voice("marin") .greeting("Hi, you are talking to an AI assistant. How can I help?") .speaking_style("Warm and unhurried.") .interruption(Interruption::Steer); // `offer_sdp` is the browser's WebRTC offer, posted to your route. let call = voice.accept_webrtc(&session, &offer_sdp).await?; let answer_sdp = call.answer_sdp().to_string(); // send back to the browser // Follow the call: what the caller said, what was spoken, interruptions. let mut events = call.events(); // Hang up from the server, or wait for the caller to. let summary = call.end().await?; ``` `VoiceCall` keeps the call running until you end it, the caller hangs up, or the value is dropped. Keep it somewhere your hang-up route can find it, keyed by `call.call_id()`. Use `accept_webrtc_as` to pass OpenAI a stable, privacy-preserving end-user id for abuse monitoring. Write the agent’s instructions for speech: short sentences, no markdown, lists or links, numbers said the way a person says them. Delivery (pace, tone) belongs in `speaking_style`; business rules stay in the agent. ### Cascaded calls A cascaded channel takes audio through your process: 16-bit little-endian PCM, mono, 24 kHz, in both directions. You connect it to the caller over any transport, such as a WebSocket or a native app’s audio stream. ```rust use everruns::providers::openai::OpenAI; use everruns::voice::{AudioOut, VoiceChannel}; let voice = VoiceChannel::cascaded(OpenAI::from_env()?.speech()) .speech_model("gpt-4o-mini-tts") .voice("marin") .greeting("Hi, you are talking to an AI assistant. How can I help?"); let (call, mut audio) = voice.accept_audio(&session).await?; // Caller audio in: forward microphone frames as they arrive. audio.input.send(frame).await?; // Speech out, already paced at playback speed. while let Some(out) = audio.output.recv().await { match out { AudioOut::Audio(pcm) => play(pcm), // after what came before AudioOut::Clear => drop_unplayed(), // the caller talked over it } } ``` Dropping `audio.input` hangs up, as does `call.end()`. Each spoken sentence is synthesized while the previous one plays, and when the caller talks over the answer you get `AudioOut::Clear` at once. The call’s events and summary work as in delegated mode. ### Settings | Builder | Default | What it does | | --------------------- | ------------------- | ------------------------------------------------------------------------------- | | `model` | `gpt-realtime-2` | Speech-to-speech model of a delegated channel. | | `transcription_model` | `gpt-4o-transcribe` | Speech-to-text model for the caller, in both modes. | | `speech_model` | `gpt-4o-mini-tts` | Text-to-speech model of a cascaded channel. | | `voice` | `marin` | Provider voice. | | `language` | none | Language hint for transcribing the caller, such as `en`. | | `greeting` | none | Spoken when the call connects. Say that the caller is talking to an AI. | | `turn_detection` | `ServerVad` | `ServerVad` ends a turn on a pause; `SemanticVad` waits for a finished thought. | | `interruption` | `Steer` | Talking over the answer: `Steer` keeps the turn running, `Cancel` stops it. | | `filler` | `One moment.` | Line spoken while the agent works. | | `filler_after` | 1.5 s | Silence before the filler. Zero turns fillers off. | | `speaking_style` | none | Delivery instructions for the speech model. | `VoiceChannel::with_config` takes a whole `VoiceChannelConfig`, the same type a platform voice channel stores, so settings move between the two unchanged. ### Test without a microphone `Realtime::simulated()` places calls on an offline simulator. Play the caller with `SimulatedCall`: ```rust use everruns::voice::{Realtime, SimulatedCall, VoiceChannel}; let call = VoiceChannel::delegated(Realtime::simulated()) .greeting("Hello!") .accept_webrtc(&session, "v=0") .await?; let caller = SimulatedCall::find(call.call_id()).unwrap(); caller.say("What is the weather?"); let spoken = caller.wait_spoken(2, std::time::Duration::from_secs(5)).await; assert_eq!(spoken[0], "Hello!"); ``` Pair it with [`Model::simulated`](https://everruns.com/docs/framework/testing-and-simulation/) and a whole call runs in a unit test. For cascaded channels, `Speech::simulated()` treats audio as text: each input frame is one utterance, and each output frame is the UTF-8 text that was spoken. ## As a serve app With the `voice` feature, every top-level agent of a [serve](https://everruns.com/docs/framework/serve/) app takes calls: ```bash cargo add everruns-serve --features voice ``` ```toml # serve.toml: how calls sound. Every field is optional. [voice] voice = "marin" greeting = "Hi, you are talking to an AI assistant. How can I help?" speaking_style = "Warm and unhurried." ``` Run `OPENAI_API_KEY=… cargo run -- dev` and open `http://127.0.0.1:3000/v1/channels/{agent}/voice` for a test page with a **Start call** button. Your own page uses two routes: | Method | Route | Use | | ------ | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- | | `POST` | `/v1/channels/{agent}/voice/calls` | Place a call: `{ "sdp": "…", "session_id"?: "…", "conversation"?: "…" }`. Answers `session_id`, `call_id` and `answer_sdp`. | | `POST` | `/v1/channels/{agent}/voice/calls/{call_id}/end` | Hang up. Answers what the call did. | ```js const pc = new RTCPeerConnection(); const mic = await navigator.mediaDevices.getUserMedia({ audio: true }); pc.addTrack(mic.getTracks()[0]); const audio = new Audio(); audio.autoplay = true; pc.ontrack = (event) => (audio.srcObject = event.streams[0]); pc.createDataChannel("oai-events"); const offer = await pc.createOffer(); await pc.setLocalDescription(offer); const response = await fetch("/v1/channels/concierge/voice/calls", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ sdp: offer.sdp }), }); const { session_id, call_id, answer_sdp } = await response.json(); await pc.setRemoteDescription({ type: "answer", sdp: answer_sdp }); ``` Each call lands in a session of the agent’s voice channel: * With `conversation`, your key for the caller (a user id, a phone number), every call with that key continues the same session. * With `session_id`, the call continues that session, such as a chat that started typed. * With neither, the call starts a new session. Follow the transcript at `/v1/sessions/{session_id}/sse` like any session. ### Cascaded calls over a WebSocket Set `mode = "cascaded"` and the app takes calls over a WebSocket instead: ```toml [voice] mode = "cascaded" speech_model = "gpt-4o-mini-tts" voice = "marin" ``` Connect to `ws://…/v1/channels/{agent}/voice/ws` (add `?conversation=…` or `?session_id=…` to continue a conversation). Binary messages carry 16-bit PCM, mono, 24 kHz, both ways. Text messages are JSON from serve: | `type` | When | | ------------- | ------------------------------------------------------------- | | `call` | First message: `session_id`, `call_id`, `sample_rate`. | | `clear` | The caller talked over the answer; drop audio not played yet. | | `transcript` | A finished line: `role` (`user` or `assistant`) and `text`. | | `interrupted` | Speech was cut off; `heard` is what was spoken. | | `ended` | The call finished, with its counts. | Closing the socket hangs up. The test page uses this transport when the app is cascaded. An app takes one transport: the other route answers `400` and names the right one. Without `OPENAI_API_KEY`, `dev` and `eval` place calls on the offline simulator (`model = "sim"` forces it, or `speech_model = "sim"` for a cascaded app), and `start` refuses calls. Calls are unauthenticated like every serve route, so put a public app behind your own auth. The [voice example](https://github.com/everruns/everruns/tree/main/examples/serve/voice) is a hotel front desk you can call. ## On the platform The Everruns server has [voice channels](https://everruns.com/docs/features/voice/) with the same settings, behind the Voice feature flag. A Framework agent [moved to the platform](https://everruns.com/docs/framework/moving-to-platform/) keeps its voice settings by storing the same `VoiceChannelConfig` on a `voice` channel. The server runs delegated channels only, so a cascaded agent switches to delegated when it moves. ## Not yet * Native mode, where the speech model answers by itself and calls the agent’s tools. * Phone numbers. --- # Workspaces and Environments > Isolate writable project heads, bind them to sessions, reopen them safely, and configure safe read and write scopes. Source: An `Agent` describes behavior. A `Session` owns conversation continuity. An `Environment` fixes the execution resources for that session, beginning with one `WorkspaceHead`. A `Workspace` is logical project lineage, not a directory alias. Each `WorkspaceHead` is a stable, backend-owned mutable view of that lineage. All heads present the same portable `/workspace` namespace even when a backend implements them as Git worktrees, remote volumes, or another storage system. ## Isolated local Git heads Enable the `local` feature to use the public Git-worktree backend: ```rust use std::sync::Arc; use everruns::{ Agent, Engine, LocalGitWorkspace, OpenAI, Workspace, WorkspacePolicy, }; let backend = Arc::new(LocalGitWorkspace::new(state)?); let workspace = Workspace::open(backend, repository.to_string_lossy()).await?; let head = workspace .head("feature") .from_revision("main") .create() .await?; let agent = Agent::builder() .instructions("Work in the selected project head.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .workspace_policy(WorkspacePolicy::read_write()) .build()?; let engine = Engine::new(); let session = engine.create(agent).workspace(head).start().await?; assert!(session.workspace_head().is_some()); ``` Head creation is isolated by default. The Framework rejects binding the same isolated head to a second session. Opt into a shared mutable head with `workspace.head("shared").shared().create()`. A shared real-disk head does not become isolated: Framework compare-and-set writes report stale-content conflicts within the host process, and backend status reports Git conflict and dirty metadata. Coordinate other writers at the application or backend layer. Use `head.fork("name").await` to create an isolated head from the current checkpoint. `checkpoint`, `status`, `archive`, and `destroy` are explicit lifecycle operations. Dropping a head, session, agent, workspace, or backend never deletes a worktree or branch. The local backend’s explicit `destroy` removes the worktree and retains its Git branch. Archive blocks later reopen; it does not revoke a filesystem handle already owned by a running session. ## Exact resume `start()` persists the backend’s credential-free opaque binding before the session can execute. `Engine::resume` asks the recorded backend to reopen that exact workspace and head. It returns a structured `ResumeError` when the backend is missing, the head is unavailable, the binding is corrupt, or the backend returns a different identity. It never substitutes an empty or different head. After a process restart, the Agent attached to a durable session created from an explicit backend must register that backend with `AgentBuilder::workspace_backend`. The backend used by a live Environment is remembered automatically in that Agent snapshot. The default memory backend and `AgentBuilder::workspace(path)` shorthand backend are registered by the Framework itself. ## Compatibility window Use `WorkspaceBackend`, `WorkspaceBackendId`, `LocalGitWorkspace`, `AgentBuilder::workspace_backend`, and `WorkspaceHead::backend` in new code. The provider-named types and methods remain as deprecated forwarding aliases. Existing error matches keep their behavior during the deprecation window. Framework and built-in backend paths continue to emit `BuildError::DuplicateWorkspaceProvider`, `ResumeError::WorkspaceProviderUnavailable`, `SessionEnvironmentError::ProviderConflict`, `WorkspaceError::ProviderUnavailable`, and `WorkspaceError::Provider`. Their replacements are `BuildError::DuplicateWorkspaceBackend`, `ResumeError::WorkspaceBackendUnavailable`, `SessionEnvironmentError::BackendConflict`, `WorkspaceError::BackendUnavailable`, and `WorkspaceError::Backend`. Match both names while migrating. New `WorkspaceBackend` implementations should return the backend-named `WorkspaceError` variants. Persisted `WorkspaceBinding::provider_id`, SQLite columns, and existing `workspace-provider` state directory names do not change in this migration. ## Workspace, roots, policy, and sandbox These concepts are deliberately separate: | Concept | Meaning | | ----------------- | ----------------------------------------------------------------------- | | `Workspace` | Logical project or lineage | | `WorkspaceHead` | One reopenable mutable view selected for a session | | `/workspace` | Stable model-visible path presented by the head filesystem | | Additional roots | Extra named mounts in `WorkspaceRootSet`; never heads or lineage | | `WorkspacePolicy` | Portable read/write authorization composed over the selected filesystem | | Sandbox | A process/compute isolation boundary; not provided by path policy alone | The Environment carries its head plus an open type-keyed extension boundary for future compute or network resources. Backends implement the async `WorkspaceBackend` trait directly; there is no backend enum or vendor switch. Every head supplies the existing `SessionFileSystem`, so file tools, seeded files, containment checks, mounts, and `WorkspacePolicy` remain one stack. When a compute resource needs the selected filesystem, attach it with `EnvironmentBuilder::workspace_extension`; its constructor receives the exact head that Framework file tools will use. For a simple application, `engine.create(agent)` is the concise path. Its first `send` or `inspect` selects the default head automatically; optional `session.start().await` selects it earlier without running a turn. `AgentBuilder::workspace(path)` is shorthand for one explicitly shared local directory across that Agent’s sessions; it does not create isolated heads. The shorthand is still a first-class shared head and its exact canonical path binding is persisted for resume. Choose an Environment when isolation, forking, or backend-specific lifecycle matters. ## Workspace security `WorkspacePolicy` is the portable security boundary for files visible to an in-process agent. Applications configure it through `everruns`; they do not need `RealDiskFileStore`, `HostBackends`, or a runtime-owned blocklist. ```rust use everruns::{Agent, OpenAI, WorkspacePolicy}; fn build(root: &std::path::Path) -> Result> { let policy = WorkspacePolicy::builder() .allow_read("/") .allow_write("generated") .deny_write("generated/locked") .allow_hidden(".github") .build()?; Ok(Agent::builder() .instructions("Work only inside the configured workspace.") .provider(OpenAI::from_env()?) .model("gpt-6.2-sol") .workspace(root) .workspace_policy(policy) .build()?) } ``` ### Defaults `WorkspacePolicy::default()` and `WorkspacePolicy::read_only()` use the same secure baseline: | Operation | Default | | ---------------------------------------- | ------- | | Read ordinary workspace files | Allowed | | Write, create, or delete | Denied | | Read or write hidden paths | Denied | | Read or write common credential paths | Denied | | Read framework-managed `.agents` content | Allowed | | Recursively delete a directory | Denied | `WorkspacePolicy::read_write()` is an explicit opt-in to ordinary writes. It does not expose additional hidden or sensitive paths and does not enable recursive deletion. It also keeps common dependency and build directories such as `node_modules` and `target` non-writable at every depth. For narrower access, start with `WorkspacePolicy::builder()`, which has no readable or writable scopes until you add them. A custom builder does not inherit the default `.agents` exception. If the agent needs workspace-provided instructions or skills, add both a readable scope and a narrow `allow_hidden(".agents")` opt-in. Protected path names are defense in depth, not content-based secret scanning. Keep credentials outside the mounted workspace, add explicit deny scopes for application-specific secret locations, and never place credentials in `.agents` content. ### Matching and precedence Scopes are literal path prefixes, not globs, and compare ASCII letters without case sensitivity so a deny cannot be bypassed on a case-insensitive backend. `generated` therefore includes `generated/report.md` but not `other/generated/report.md`. * A deny scope always wins over an allow scope. * `deny_write_component` rejects an exact directory or file name at every depth; `deny_write` rejects one rooted path subtree. * Hidden paths need `allow_hidden` for the narrow path that should be visible. * Common credential paths need the stronger `allow_sensitive` opt-in, which also permits hidden components inside that specific scope. * `compose` is restrictive: every composed policy must allow the operation. A library can add constraints without accidentally broadening the application’s policy. Trusted starter files are installed before model access is enforced. This lets an application seed a read-only file even under a non-writable policy. Later reads, writes, and deletes of that file still go through the policy, so seeding a hidden file does not automatically expose it. ### Paths and containment Policy paths live in one portable workspace namespace. These spellings identify the same file: ```text src/lib.rs /src/lib.rs /workspace/src/lib.rs ``` Traversal (`..`), NUL bytes, and backslash-separated paths fail closed. Host absolute paths are backend-specific and are not portable policy scopes; use `/workspace/...` in application configuration and model instructions. The policy layer controls visibility and mutation. The selected filesystem backend remains responsible for mapping workspace paths to storage. The local host backend canonicalizes its root, keeps resolved paths contained, and rejects symlinks in existing path components before every operation. An absolute path outside the configured root cannot expose that host file. The policy governs capabilities that use Everruns’ session filesystem. A custom tool that calls `std::fs`, launches a shell, or uses another storage API does not pass through this boundary. Apply equivalent restrictions to those tools or run them in a sandbox. ### Symlinks and races The built-in local backend rejects a symlink introduced after the workspace was configured because it rechecks components on every operation. This blocks normal traversal and symlink-swap attempts between operations. It is not an OS sandbox. A malicious process running as the same operating system user can race a final path check and filesystem syscall. If local processes are mutually untrusted, use an isolated sandbox/filesystem backend or operating-system isolation. Do not use `WorkspacePolicy` as a substitute for that process boundary. ### Backend extension The in-process host applies the policy after resolving the platform’s filesystem factory. In-memory, local-disk, database, and custom backends all receive the same policy checks. Backend authors still own containment, symlink-safe I/O, quotas, durability, and atomic update guarantees for their storage system. Directory listings and grep are enforced at the same boundary as direct reads. Denied files are not opened by policy grep, and denied names, match counts, and byte totals are not returned. Recursive deletes inspect descendants through the backend before deletion, so opting into recursion does not override a deny or protected descendant. Backends with mutable external state must still treat that preflight-to-delete window as a race boundary. `WorkspaceRootSet` additional roots are named mounts inside one selected head; they are not independent heads and carry no fork/reopen lifecycle. Likewise, `WorkspacePolicy` is path authorization, not a compute sandbox. The sections above describe the identity and lifecycle model.