Web Bot Auth: Signature-Agent, key directories, limits

A website verifies a bot by fetching the key the bot points to

A Web Bot Auth request carries an RFC 9421 signature and a header that says where the signer's public key lives. The website does not need a prior relationship with the bot. It reads the URL, fetches a key set from it, finds the key named in the signature, and checks the signature.

The IETF working group's charter, read on 18 October 2026, says the group "will standardize methods for cryptographically authenticating automated clients and providing additional information about their operators to Web sites." The group's protocol document is draft-ietf-webbotauth-httpsig-protocol-00, dated 1 September 2026, intended status Standards Track, authored by Thibault Meunier (Cloudflare) and Sandor Major (Google). It expires on 5 March 2027.

The draft states its own standing in its boilerplate: it is inappropriate to cite Internet-Drafts other than as "work in progress." Everything below is a snapshot of that draft and of one implementer's documentation, both read on 18 October 2026.

For the RFC 9421 mechanics (signature base, covered components, Content-Digest), see RFC 9421 HTTP message signatures for AI agents. This entry covers what Web Bot Auth adds on top.

A signed request carries three headers

Section 5.2.5 of the draft gives the request. Line wrapping follows RFC 8792.

GET /path/to/resource HTTP/1.1
Host: origin.example.com
Signature: sig=abc==
Signature-Input: sig=("@authority" "signature-agent";key="sig");\
                 created=1700000000;\
                 expires=1700011111;\
                 keyid="ba3e64==";\
                 tag="web-bot-auth"
Signature-Agent: sig="https://signer.example.com"

What section 5.2 requires of the signature, as the draft words it:

• At least one of @authority or @target-uri is covered.
• created and expires are present. "It is RECOMMENDED that expiry be no more than 24 hours."
• tag "MUST be web-bot-auth".
• keyid "MUST be a base64url JWK SHA-256 Thumbprint".
• The Signature-Agent header is itself a covered component.

Two omissions matter for anyone who carries money or credentials. The draft does not require a body digest: "Most automated traffic is GET, and a mandatory digest would force every Agent to buffer request bodies it would otherwise stream." An agent that needs one "MUST send and cover Content-Digest". And it forbids shared secrets: "Implementations MUST NOT use shared HMAC", because "Shared secrets break non-repudiation and make auditing difficult."

Signature-Agent is where the two documents disagree

The draft defines Signature-Agent as "a Dictionary Structured Header" whose members are strings holding https URIs. It then addresses the older form directly:

Earlier versions of this protocol defined Signature-Agent as a bare String, and deployments still send it (Appendix E.1.2). A verifier MAY accept that form … Signers MUST send the dictionary form.

Cloudflare's Web Bot Auth page, shown as "Last updated Oct 8, 2026", describes what it accepts differently: "Cloudflare implements the Signature-Agent format from draft-meunier-http-message-signatures-directory-03, where the header value is a structured string such as "https://signature-agent.test"." It then lists what makes verification fail, including "The message uses the dictionary form from later drafts, such as sig2="https://signature-agent.test"."

Side by side, with the covered component each form implies:

The draft notes the forms are easy to tell apart: a String Item begins with a double quote character, and a dictionary member key does not. A signer that targets a specific verifier should follow that verifier's current documentation and test it. Cloudflare publishes an endpoint for this: according to its page, crawltest.com/cdn-cgi/web-bot-auth returns 401 if the message is well formed but the key is unknown, 200 if the key is known and the message verifies, and 400 otherwise.

That is a statement about two documents on one date, not about which is right. The draft is newer and carries the working-group name. The implementation is the one that decides whether a request gets through today.

The key directory is a JWK Set at a well-known path

Section 5.5 defines three discovery types for the URL in Signature-Agent, selected by a type parameter ("When type is absent, its value is directory."):

• directory: resolve the HTTP Message Signatures Directory at the well-known URI.
• jwks_uri: "Resolve the member value as a direct JWK Set URI."
• cimd: "Resolve the member value as a Client ID Metadata Document" URI.

All three "resolve to a JSON Web Key Set (JWKS)". Section 5.5.1 requires HTTPS and, at the well-known path, the media type application/http-message-signatures-directory+json. The draft's example exchange:

GET /.well-known/http-message-signatures-directory HTTP/1.1
Host: example.com
Accept: application/http-message-signatures-directory+json

HTTP/1.1 200 OK
Content-Type: application/http-message-signatures-directory+json
Cache-Control: max-age=86400
{
  "keys": [{
    "kty": "OKP",
    "crv": "Ed25519",
    "kid": "NFcWBst6DXG-N35nHdzMrioWntdzNZghQSkjHNMMSjw",
    "x": "JrQLj5P_89iXES9-vFgrIy29clF9CC_oPPsw3c5D0bs",
    "use": "sig",
    "nbf": 1712793600,
    "exp": 1715385600
  }]
}

The origin learns the URL from the request, so it resolves keys after the request arrives, not before. Per section 5.2.5, later requests verify from cache.

Cloudflare's page adds its own requirements for the directory: it must be served from /.well-known/http-message-signatures-directory over HTTPS, signed with the http-message-signatures-directory tag, and the page says "Cloudflare supports Ed25519 key algorithm." To be recognised as a verified bot, the operator submits the directory URL through a Bot Submission Form in the dashboard, with "Request Signature" as the verification method.

What a verifier has to get right

Three rules in the draft are easy to miss, and each one closes a specific attack.

Look keys up by URL and key together. Section 5.4: "Key lookup MUST be keyed on the (URL, key) pair, not on the key alone." The draft explains why. A verifier indexing by keyid alone "will verify a request as coming from one URL that provides a key it learned from another", and the party whose URL was claimed "cannot detect or stop this".

Cache the directory, but bound the cache. Verifiers "SHOULD cache the directory contents and refresh upon expiration", and an appendix says they "should not fetch the directory for every request". Failed lookups can be cached too, but "Negative cache entries should expire after no more than five minutes."

Rotate by overlap. Operators "SHOULD rotate keys by publishing the old and the new key together, then removing the old one."

Neither document gives you replay or revocation

Replay: the draft's section 5.2.3 points to RFC 9421 section 7.2.2 and says "This document defines no additional nonce requirements." An appendix adds "Whether a verifier requires and enforces nonces is deployment policy." Cloudflare's page recommends a nonce and then states "there is currently no nonce validation, nor does Cloudflare guard against replay attacks using a database of seen nonces." Its advice is a short expires, and "a minute is often sufficient." The draft's ceiling for a recommended expiry is 24 hours. A captured request is therefore replayable for as long as its expires allows, on a verifier that keeps no nonce store.

Revocation: section 5.5.2 says directory removal "is not a revocation mechanism, and this document does not define any". Section 6.3 spells out the consequence. Removal "takes effect at each verifier on its next refresh, so the key can keep verifying for as long as Section 5.5.2 allows", and the draft names signature lifetimes (Section 5.2) as the lever that acts faster. With the example directory's Cache-Control: max-age=86400, a verifier that cached the key just before its removal could keep accepting it for a day.

For a bot operator the lesson is the same as for any signed credential: the key is the identity, and the signature lifetime is the blast-radius setting. Pick expires in seconds, not hours. For a website operator: decide whether you need a nonce store, because the protocol will not decide it for you.

Cloudflare's page lists further limits that apply to its implementation, such as unsupported components (@query-params, @status) and a warning to sign Content-Digest if "there is zero risk of a message being altered on the way to Cloudflare."

Where Mandare fits, and what it does not claim

Mandare uses parts of this stack and does not implement the profile. Checked against the public repository at commit 327442e (3 October 2026), and applying to @mandarelabs/cli v0.1.0, the version npm view reported on 18 October 2026:

• In passport mode, agent requests to Mandare's gateway carry RFC 9421 signatures made and checked with the web-bot-auth npm package (pinned at 0.1.3 in packages/passport/package.json), with its web-bot-auth tag. The signature must cover @method, @path, @authority, content-digest and signature-agent, and the nonce is single use. The concepts page describes this as the Passport.
• Signature-Agent there is the agent's did:key as a single quoted string, and keyid is the SHA-256 hex of the raw public key. The draft requires a dictionary of https URIs and a JWK thumbprint. The source file's header comment says "Interoperability with Web Bot Auth verifiers is NOT claimed", and none has been tested.
• Mandare publishes a JWK Set key directory, built with mandare directory, modelled on the Web Bot Auth directory drafts. It carries door keys for checking ledger signatures. Doors do not resolve agent keys through it. docs/KEY-DIRECTORY.md lists the differences field by field, including that it writes "alg": "EdDSA" where the draft restricts alg to the RFC 9421 registry.

The two problems also point in different directions. Web Bot Auth lets a website recognise a bot. Mandare's door sits in front of an operator's own agents and refuses calls past a signed limit. A signed agent identity is an input to that refusal, but proving an agent's name to a third-party site is not what the door does. The agent revocation command, mandare kill <agent-did>, is described in the CLI reference, and the effect of a stolen key under proof of possession is in stolen AI agent API key: what proof of possession fixes.