Physical Architecture
The physical components that make up an Everruns deployment, control plane, workers, PostgreSQL, NATS, Valkey, and the reverse proxy, and how they fit together.
The getting-started architecture page describes the logical shape of Everruns: a control plane, a worker tier, and a shared database. This page goes one level deeper and describes the physical components an operator actually deploys, what each one is for, when it is optional, and how data flows between them.
Components at a glance
Section titled “Components at a glance”| Component | Role | Required | Default port |
|---|---|---|---|
| Reverse proxy | TLS termination and route fan-out for /api, /mcp, /.well-known/*, UI |
Yes (or equivalent ingress) | 443 |
| Control plane (server) | REST API, SSE event streams, gRPC server for workers, owns all state | Yes | 9301 (HTTP), 9001 (gRPC) |
| Worker pool | Stateless executors of the agentic loop: each leases a session and runs its whole turn | Yes | – (outbound only) |
| PostgreSQL 17 | Durable storage for agents, sessions, events, durable task queue | Yes | 5432 |
| NATS | Live session events across control-plane instances, task notifications | Optional for one control-plane instance, required for more | 4222 |
| Valkey | Distributed sliding-window rate limiting across control-plane instances | Optional | 6379 |
| Management UI | Operator interface for agent and provider configuration | Optional | – (served by proxy) |
| Tenant egress proxy | Gives tenant outbound traffic its own address, per-org limits and usage counts | Optional | 4750 (proxy), 4751 (health) |
Workers never talk to PostgreSQL, NATS, or Valkey directly. Every read and write goes through the control plane’s gRPC service on port 9001. This is what lets workers run with no database credentials, no encryption keys, and no awareness of the data tier.
PostgreSQL, the only required stateful component
Section titled “PostgreSQL, the only required stateful component”PostgreSQL holds the Platform’s authoritative metadata and session history. With the default STORAGE_BLOB_BACKEND=db, file and image bytes also live there. With STORAGE_BLOB_BACKEND=s3, bytes live in the configured object store, so recovery needs both the database and that store. Preserve the deployment’s encryption keys separately; a database backup alone cannot recover encrypted secrets. See Object storage.
What lives in PostgreSQL:
- Agents, sessions, messages, and durable events
- The session leases workers run turns under, and the durable task queue for background work (both claimed via
SKIP LOCKED) - Encrypted LLM provider credentials, MCP server registrations, capability config
- Per-session virtual filesystems, knowledge bases, and the event log used for SSE replay
Operational requirements:
- PostgreSQL 17. UUIDv7 is implemented via a custom SQL function; PG 18’s native
uuidv7()will be adopted once it is widely available on managed services. - Direct connection for
LISTEN/NOTIFY. Pooled or proxied endpoints (PgBouncer, Neon-pooler, RDS Proxy) interleave notification frames with query traffic. SetDATABASE_URLto the pooled endpoint for normal queries andDATABASE_UNPOOLED_URLto a direct session-scoped endpoint for listeners. Startup fails fast if the configured listener URL looks pooled. - Pool sizing. With
EXPECTED_INSTANCES=Nset, each instance sizes its pool so thatpool × instancesstays under 80% ofPG_MAX_CONNECTIONS. - Migrations. Auto-applied on server startup via embedded sqlx migrations, protected by a PostgreSQL advisory lock so multiple control-plane instances can boot together without racing.
See docs/sre/environment-variables.md for the full list of database-related variables.
NATS, required for more than one control-plane instance
Section titled “NATS, required for more than one control-plane instance”Live session events (what a chat streams over SSE) are pushed from the instance that published them to the instances serving the clients. NATS carries that push between instances. How a control-plane instance delivers them depends on its setup, and /health reports the choice as event_delivery:
| Setup | event_delivery |
Behavior |
|---|---|---|
NATS_URL set and reachable |
nats |
Events reach every instance. Delta events (output.message.delta, reason.thinking.delta, tool.output.delta, and voice transcript deltas) skip PostgreSQL and use Core NATS pub/sub without retained replay. Durable events still persist to PG, so SSE reconnection via since_id keeps working. |
One instance, no NATS_URL |
in_memory |
Events are pushed inside the one process. Every event, deltas included, is also written to PG. Correct for a single control-plane instance only. |
NATS_URL awaiting startup recovery, or EXPECTED_INSTANCES above 1 with no NATS_URL |
polling |
Each open stream polls PostgreSQL for new events (100 ms while busy, backing off to 1 s while quiet). Every event is written to PG. Streams stay correct across instances, at the cost of extra queries and latency. /health reports status: degraded, and the system doctor shows a warning. |
Run more than one control-plane instance only with NATS, and set EXPECTED_INSTANCES so an instance started without NATS falls back to polling instead of in-memory delivery. The system doctor flags a problem when several instances run and any of them uses in-memory delivery, because clients on one instance would miss events published through another.
If NATS_URL is configured but startup connection attempts fail, the server retries
in the background every 30 seconds (each attempt is bounded to 20 seconds), while
continuing to serve through PostgreSQL polling. Once connected, it cycles existing
polling SSE streams with a disconnecting notice. Clients reconnect with their
last durable cursor, subscribe before replay, and deduplicate overlapping durable
events. /health and the system doctor then report nats instead of polling.
Deployments without NATS_URL remain on their selected fallback.
Live session delivery uses Core NATS pub/sub, not a retained JetStream event stream. Durable events are replayed from PostgreSQL and carry a persisted sequence/cursor. Events delivered ephemerally through NATS do not carry a durable sequence and must not advance the reconnect cursor. Token deltas remain best-effort, including during reconnects; completed messages remain in durable session history.
NATS also carries task notifications: task.available.{activity_type} subjects replace PG NOTIFY for worker wakeup, dropping notification latency from about 30 ms to about 1 ms. Only the control plane connects to NATS; workers talk to the server via gRPC.
Valkey, optional distributed rate limiting
Section titled “Valkey, optional distributed rate limiting”Valkey is a Redis-compatible key-value store (a Linux Foundation fork of Redis). Everruns uses it for exactly one thing: sliding-window rate limiting that is coordinated across control-plane instances.
When VALKEY_URL is not set, rate limiting falls back to an in-memory governor, accurate per-instance, but with N instances behind a load balancer a single IP can consume up to N× the intended budget. Set VALKEY_URL when you run more than one control-plane instance and need a shared budget.
Connection details:
- Accepts
redis://,rediss://(TLS),valkey://,valkeys://(TLS) schemes - Uses atomic Lua scripts for sliding-window counters
- Fail-open: if Valkey is unreachable, the rate limiter allows the request rather than rejecting traffic on a side-channel outage
- Only the control plane connects to Valkey; workers do not need access
Worker pool, no shared state
Section titled “Worker pool, no shared state”Workers are the most operationally boring component in the deployment. They have:
- No database connection
- No encryption key
- No NATS or Valkey access
- No durable local state
For agent turns, workers lease a session from the control plane over gRPC and drive its whole reason–act loop, appending events through the control plane. If a worker dies, its session lease expires (10 seconds by default) and another worker reconstructs the unfinished turn from the event log. Fenced writes reject a displaced holder; external requests already in flight still require the tool’s recovery policy and idempotency. Background activities use a separate durable task queue and stale-task reaper. Add workers for throughput; remove them to save cost.
See Worker authentication for the WORKER_GRPC_AUTH_TOKEN and optional mTLS setup that secures this internal channel.
Reverse proxy contract
Section titled “Reverse proxy contract”A reverse proxy (or platform ingress that enforces the same routes) is mandatory in production:
| Route | Destination | Notes |
|---|---|---|
/api/* |
Control plane | Disable proxy buffering for SSE |
/mcp |
Control plane | Do not rewrite under /api |
/.well-known/* |
Control plane | OAuth discovery; do not rewrite |
/health |
Control plane | Health check target |
| Everything else | UI | If UI is deployed; otherwise 404 |
TLS terminates at the proxy. Worker gRPC traffic stays on the private network, never expose port 9001 publicly. See local/Caddyfile and infra/docker-compose/docker-compose.yaml for working configurations.
Outbound traffic
Section titled “Outbound traffic”The control plane and workers both make outbound calls, and Everruns splits them in two:
- Platform traffic is what the deployment itself owns: LLM providers, the utility model, system email, sandbox provider APIs. It always leaves directly, from the deployment’s own address.
- Tenant traffic is what an agent, a capability, an MCP server or an integration chooses: web fetches, tool calls to third-party APIs, remote MCP servers. Every such request goes through one boundary, the egress service, which applies the policy below and writes an audit line with its org, session and decision (
everruns::egress::auditlog target).
Egress policy
Section titled “Egress policy”EVERRUNS_EGRESS_POLICY decides which public hosts tenant traffic may reach:
| Mode | Behavior |
|---|---|
open (default) |
Any public host. |
curated-writes |
GET and HEAD reach any public host; writes, MCP and integration traffic need the built-in allowlist. |
curated-all |
Every request needs the allowlist. |
In both curated modes a built-in deny list is checked first, URLs are capped at 2048 characters and IP-literal hosts are refused. An organization can be granted extra hosts or open egress for its own traffic; see Network Access Control. Private, loopback and cloud metadata addresses are refused in every mode, with DNS pinned so a lookup cannot be swapped between the check and the connection.
Tenant egress proxy
Section titled “Tenant egress proxy”Set EVERRUNS_EGRESS_PROXY_URL and EVERRUNS_EGRESS_PROXY_SECRET on the control plane and workers, and tenant traffic leaves through everruns-egress-proxy instead of directly. Platform traffic never uses it. You get two things:
- Its own outbound address. Give the proxy host a separate public IP. If a tenant’s traffic gets that address blocked somewhere, model calls, email and billing keep working, and the address can be replaced without touching the rest of the deployment.
- Per-org limits and usage. Each request carries a five-minute token, signed with the shared secret, that names its org and session. The proxy verifies the token, resolves the destination again and refuses non-public addresses and SMTP ports, applies per-org caps, and counts bytes and requests per org and session each minute.
The binary ships in the server image. Limits are off unless you set them, and usage goes to the log (everruns::egress::usage target). A control endpoint (EGRESS_PROXY_CONTROL_URL) can supply limits per org and receive usage instead. A proxy URL with a missing or short secret fails closed: tenant requests error rather than leave directly. Variables: Tenant egress proxy.
# Proxy hostEVERRUNS_EGRESS_PROXY_SECRET=<32+ random bytes>EGRESS_PROXY_ORG_RPM=600 # optional capsEGRESS_PROXY_ORG_BYTES_PER_DAY=5000000000everruns-egress-proxy
# Control plane and workersEVERRUNS_EGRESS_PROXY_URL=http://egress-proxy:4750EVERRUNS_EGRESS_PROXY_SECRET=<same value>Development modes
Section titled “Development modes”The same binaries collapse into smaller deployments for local work:
DEV_MODE=true(single process). Nothing to install or run beside the binary: the server starts a temporary PostgreSQL of its own and deletes it on exit. Execution runs in-process inside the server binary; the gRPC server is disabled. Data is lost on restart. Useful for UI iteration and API development.just start-all(full local). Brings up PostgreSQL, Valkey, and NATS as local processes (no Docker required) and starts the server + worker against them. Mirrors production wiring on a single machine.- Docker Compose. The production-shaped topology in one machine; see Docker Compose.
Multi-instance deployment
Section titled “Multi-instance deployment”Multiple control-plane instances can run behind a load balancer with no session affinity:
| Concern | How it stays correct |
|---|---|
| Database connections | EXPECTED_INSTANCES=N divides the pool so pool × instances ≤ 80% of PG_MAX_CONNECTIONS |
| SSE delivery | NATS subjects fan out to every instance (required for more than one instance; without it, instances poll PostgreSQL and report degraded); reconnects are idempotent |
| Turn and task claiming | SKIP LOCKED on session leases and the durable task queue partitions work naturally; a session’s fenced lease keeps one worker on it |
| Migrations | PostgreSQL advisory lock prevents concurrent runs |
| Rate limits | Valkey-backed sliding-window counters are shared; in-memory falls back to per-instance |
Workers do not require coordination, add as many as you need, in as many regions as you need, as long as they can reach the control-plane gRPC port.
Further reading
Section titled “Further reading”- Architecture (Getting Started), the logical model
- Environment Variables, every knob and its default
- Network Access Control, per-agent and per-org outbound rules
- Docker Compose, a production-shaped local setup
- Custom backends, low-level execution-host composition