Request Signing
How Everruns signs outbound HTTP requests with Ed25519 signatures per RFC 9421, enabling target servers to verify bot identity
When an AI agent fetches web content, the receiving server has no way to distinguish it from an anonymous scraper. Request signing solves this by attaching a cryptographic signature to every outbound HTTP request, letting target servers verify who is making the request and choose to grant or deny access based on that identity.
Everruns implements Web Bot Auth, the IETF working-group profile of RFC 9421 HTTP Message Signatures, using Ed25519 signatures.
Background
Section titled “Background”The Problem
Section titled “The Problem”Traditional bot identification relies on User-Agent strings, which are trivially spoofed. IP-based allow lists are brittle and don’t scale. There is no standard way for a web bot to prove its identity to a server.
HTTP Message Signatures (RFC 9421)
Section titled “HTTP Message Signatures (RFC 9421)”RFC 9421 defines a general mechanism for signing HTTP messages. A sender selects components of the request (method, authority, specific headers) and signs them with a private key. The signature and a description of what was signed are transmitted as structured headers:
Signature: sig=:base64-encoded-signature:Signature-Input: sig=("@authority");created=1735689600;expires=1735689900; keyid="JWK-thumbprint";alg="ed25519";nonce="random"; tag="web-bot-auth"The receiving server reconstructs the same signature base from the request, fetches the sender’s public key, and verifies the signature. Replay protection requires the receiver to enforce the created/expires window and reject reused nonces within that window. A signature and random nonce alone do not prevent replay.
Web Bot Auth
Section titled “Web Bot Auth”The Web Bot Auth draft builds on RFC 9421 specifically for bot identification:
- Algorithm: Ed25519 (fast, small keys, no parameter choices)
- Covered components:
@authority(the target domain) at minimum - Key identity: JWK Thumbprint (SHA-256 hash of the canonical public key representation)
- Signature tag:
"web-bot-auth", distinguishes bot-auth signatures from other uses of RFC 9421 - Discovery: the
Signature-Agentheader names the https URL where the bot’s public keys can be found
Key Discovery
Section titled “Key Discovery”The same draft defines how target servers find a bot’s public keys:
GET <signature-agent-origin>/.well-known/http-message-signatures-directoryThis returns a JSON Web Key Set (JWKS) containing the bot’s Ed25519 public keys. The target server uses the kid field to match the key against the keyid in the incoming signature.
How It Works in Everruns
Section titled “How It Works in Everruns”Request signing is a server-wide feature owned by the egress layer, the part of Everruns every outbound tenant request passes through. When it is configured, requests made by the web_fetch tool and by curl in the Bashkit shell are signed. Each redirect hop is signed for its own host. Other outbound traffic, such as LLM providers, MCP servers and sandbox networking, is not signed.
Everruns follows the IETF working-group draft draft-ietf-webbotauth-httpsig-protocol and stays compatible with Cloudflare’s verifier.
Signing (outbound)
Section titled “Signing (outbound)”- A tool builds a request and asks the egress layer for platform signing.
- The egress layer applies the deployment’s network policy. Refused requests are never signed.
- For an
https://request, it adds three headers:Signature-Agent: the deployment’s origin as a quoted string, for example"https://app.example.com".Signature-Input: covers@method,@authority,@pathandsignature-agent, withcreated,expires(5 minutes by default),keyid,alg="ed25519", a randomnonceandtag="web-bot-auth".Signature: the Ed25519 signature, encoded as standard base64.
- Plain
http://requests are sent unsigned, because a signature on an unencrypted connection can be replayed by anyone on the path. A request that already carries its ownSignatureheader keeps it.
Key directory (inbound)
Section titled “Key directory (inbound)”Everruns serves its public keys at /.well-known/http-message-signatures-directory. This endpoint:
- is public and needs no authentication
- returns a JWKS with media type
application/http-message-signatures-directory+json - signs the response once per key, covering
@authority;reqandContent-Digest, withtag="http-message-signatures-directory", so the key set cannot be re-served under another host - is cacheable for a day and allows cross-origin reads
Example response:
HTTP/1.1 200 OKContent-Type: application/http-message-signatures-directory+jsonCache-Control: public, max-age=86400Content-Digest: sha-256=:...:Signature-Input: binding0=("@authority";req "content-digest");created=...;expires=...;keyid="...";tag="http-message-signatures-directory"Signature: binding0=:...:
{"keys":[{"kty":"OKP","crv":"Ed25519","kid":"<JWK thumbprint>","x":"<public key>","use":"sig"}]}What target servers see
Section titled “What target servers see”| Header | Purpose |
|---|---|
Signature |
The Ed25519 signature over the covered components |
Signature-Input |
What was signed: components, timestamps, key ID, algorithm, nonce, tag |
Signature-Agent |
Quoted https origin whose well-known directory holds the key |
Target servers that support Web Bot Auth can:
- Extract the
keyidfromSignature-Input - Fetch the directory from the
Signature-Agentorigin’s well-known path - Verify the signature
- Apply access policies based on the verified identity
Servers that don’t support it simply ignore the extra headers.
Verifying Signatures (Server Side)
Section titled “Verifying Signatures (Server Side)”If you operate a server that receives requests from Everruns agents, here’s how to verify them.
Verification steps
Section titled “Verification steps”- Check the tag: parse
Signature-Inputand confirmtag="web-bot-auth". Ignore signatures with other tags. - Check timestamps: reject if
createdis in the future orexpiresis in the past. A 5-minute clock skew tolerance is reasonable. - Fetch the public key: take the
Signature-Agentorigin (strip the surrounding quotes) and fetch<origin>/.well-known/http-message-signatures-directory. Find the key matching thekeyidfromSignature-Input. Cache the JWKS (keys rotate infrequently). - Reconstruct the signature base: build the canonical representation per RFC 9421 Section 2.5 using the covered components listed in
Signature-Input. - Verify: use Ed25519 to verify the signature against the reconstructed base and the fetched public key.
Python example
Section titled “Python example”import base64import hashlibimport timeimport httpxfrom cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
def verify_bot_auth(request) -> bool: """Verify a web-bot-auth signature on an incoming request."""
# 1. Parse Signature-Input header sig_input = request.headers.get("signature-input", "") if 'tag="web-bot-auth"' not in sig_input: return False # not a bot-auth signature
# Extract parameters from sig_input # sig=("@authority" "signature-agent");created=...;expires=...;keyid="...";... params = parse_signature_params(sig_input)
# 2. Check timestamps now = int(time.time()) if params["created"] > now + 300 or params["expires"] < now: return False # expired or future-dated
# 3. Fetch public key from the Signature-Agent origin ("https://...") agent_origin = request.headers.get("signature-agent", "").strip('"') if not agent_origin.startswith("https://"): return False jwks_url = f"{agent_origin}/.well-known/http-message-signatures-directory" jwks = httpx.get(jwks_url).json() key_data = next(k for k in jwks["keys"] if k.get("kid") == params["keyid"]) public_key_bytes = base64.urlsafe_b64decode(key_data["x"] + "==") public_key = Ed25519PublicKey.from_public_bytes(public_key_bytes)
# 4. Reconstruct signature base (RFC 9421 Section 2.5) # Covered components are listed in parentheses in Signature-Input sig_base = build_signature_base(request, params)
# 5. Verify signature = base64.b64decode( request.headers["signature"].split(":")[1] # sig=:base64: ) try: public_key.verify(signature, sig_base.encode()) return True except Exception: return FalseNode.js example
Section titled “Node.js example”import { createPublicKey, verify } from "node:crypto";
async function verifyBotAuth(request) { const sigInput = request.headers["signature-input"] || ""; if (!sigInput.includes('tag="web-bot-auth"')) return false;
const params = parseSignatureParams(sigInput);
// Check timestamps (5-minute tolerance) const now = Math.floor(Date.now() / 1000); if (params.created > now + 300 || params.expires < now) return false;
// Fetch public key const origin = (request.headers["signature-agent"] || "").replace(/^"|"$/g, ""); if (!origin.startsWith("https://")) return false; const res = await fetch( `${origin}/.well-known/http-message-signatures-directory` ); const jwks = await res.json(); const jwk = jwks.keys.find((k) => k.kid === params.keyid);
const key = createPublicKey({ key: jwk, format: "jwk" });
// Reconstruct signature base and verify const sigBase = buildSignatureBase(request, params); const signature = Buffer.from( request.headers["signature"].split(":")[1], "base64" );
return verify(null, Buffer.from(sigBase), key, signature);}Note: The
parseSignatureParamsandbuildSignatureBasehelpers follow the structured fields parsing rules from RFC 8941 and the signature base construction from RFC 9421 Section 2.5. Libraries like httpbis-message-signatures (Python) and @httpbis/message-signatures (Node.js) handle both.
Configuration
Section titled “Configuration”Request signing is configured with environment variables. Set the same values on the server and on every worker: workers sign, and the server publishes the directory.
Environment variables
Section titled “Environment variables”| Variable | Required | Default | Description |
|---|---|---|---|
BOT_AUTH_SIGNING_KEY_SEED |
yes | – | Base64url-encoded 32-byte Ed25519 seed. Unset disables signing and serves an empty directory. |
BOT_AUTH_SIGNATURE_AGENT_URL |
no | PUBLIC_APP_URL |
https origin that serves the key directory, sent as Signature-Agent. Signing stays off when neither is set. |
BOT_AUTH_PUBLISHED_KEY_SEEDS |
no | – | Comma-separated seeds listed in the directory but not used to sign, for key rotation |
BOT_AUTH_VALIDITY_SECS |
no | 300 |
Request signature lifetime in seconds |
Generate a signing key
Section titled “Generate a signing key”python3 -c "import os, base64; print(base64.urlsafe_b64encode(os.urandom(32)).rstrip(b'=').decode())"Keep the seed in your secret store. Anyone holding it can sign as your deployment.
Enable signing
Section titled “Enable signing”export BOT_AUTH_SIGNING_KEY_SEED="your-base64url-seed-here"export BOT_AUTH_SIGNATURE_AGENT_URL="https://agents.yourcompany.com"The origin must route /.well-known/http-message-signatures-directory to the Everruns server, outside any bot challenge, so verifiers can always fetch it.
Verify it’s working
Section titled “Verify it’s working”curl -si https://agents.yourcompany.com/.well-known/http-message-signatures-directoryThe response should list one key and carry Signature and Signature-Input headers.
Register with Cloudflare
Section titled “Register with Cloudflare”Sites behind Cloudflare treat signed requests as a verified bot only after registration. In the Cloudflare dashboard, open the Bot Submission Form, choose Request Signature as the verification method, and enter your directory URL. See Cloudflare’s Web Bot Auth guide.
Rotate the key
Section titled “Rotate the key”- Generate a new seed and add it to
BOT_AUTH_PUBLISHED_KEY_SEEDS. Deploy, then wait at least a day so verifiers refresh their cached directory. - Swap the seeds: the new seed becomes
BOT_AUTH_SIGNING_KEY_SEED, the old one moves toBOT_AUTH_PUBLISHED_KEY_SEEDS. Deploy. - After another day, remove the old seed.
If a seed leaks, skip the waiting: replace it and remove the old one right away. Verifiers stop accepting the old key once their cached directory expires.
Standards
Section titled “Standards”| Standard | Role |
|---|---|
| RFC 9421, HTTP Message Signatures | Core signing mechanism, how to sign and verify HTTP requests |
| RFC 8941, Structured Field Values | Encoding format for Signature and Signature-Input headers |
| RFC 7638, JWK Thumbprint | How the keyid is computed from the public key |
| RFC 7517, JSON Web Key (JWK) | Format of the keys in the JWKS response |
| RFC 8037, Ed25519 in JOSE | Ed25519 key representation in JWK format |
| draft-ietf-webbotauth-httpsig-protocol | Web Bot Auth: the bot profile of RFC 9421, Signature-Agent, and the key directory. Replaces the earlier draft-meunier architecture and directory drafts |