Fail closed or fail open: what an AI gateway should do

A failed check is a decision you make before it fails

An agent gateway sits between an agent and something that costs money: an LLM provider, a card, a paid API. Before it forwards a request it asks other systems questions. Is there budget left? Does policy allow this? Can the decision be written down? Is a human required?

Every one of those systems can be unreachable. The gateway then has two choices, and both are a policy:

• Fail open. Treat the failed check as a pass and forward the request. The agent keeps working, and spend goes unchecked until the dependency returns.
• Fail closed. Treat the failed check as a refusal. The agent gets an error, and no money moves.

Neither is free. Fail open buys availability and pays for it in the case a limit exists to prevent: a looping agent during an outage. Fail closed buys the limit and pays in stopped work. The mistake is not choosing one. The mistake is never choosing, and shipping whatever the software defaulted to.

Three systems with documented defaults show how different those are.

Envoy rejects by default and lets you flip it per filter

Envoy's external authorization filter asks a separate service whether to admit a request. Its API reference, read on 15 October 2026, documents the failure setting directly. With failure_mode_allow set to true, the filter "will accept the client request" even if communication with the authorization service has failed or it "has returned an HTTP 5xx error". With false, it "will reject client requests and return Forbidden" in the same cases. The same text then says "Defaults to false."

# Fragment of an envoy.filters.http.ext_authz filter configuration
failure_mode_allow: false    # the documented default: reject when the check fails

Two neighbouring fields are worth copying as ideas. status_on_error sets the status returned "when the authorization server returns an error or cannot be reached", with a default of 403. And when failure_mode_allow and failure_mode_allow_header_add are both true, Envoy adds x-envoy-auth-failure-mode-allowed: true to the request headers. A request that skipped its check carries a label saying so. Anything downstream can count those requests or refuse them.

Stripe Issuing gives you two seconds and a fallback you configure

Stripe Issuing asks the card issuer's own endpoint to approve or decline each authorization in real time. Stripe's documentation on real-time authorizations, read on 15 October 2026, states the budget: "If Stripe doesn't receive your approve or decline response within 2 seconds, the Authorization is automatically approved or declined based on your timeout settings", or Autopilot settings if configured.

So the failure behaviour is a setting in your Stripe dashboard, not a property of your code. A webhook that is down, slow or answering with a bad API version does not produce an error page. It produces whichever decision you configured.

The page also carries a warning that applies to any fail-open setting. Autopilot, listed as public preview on the page, "provides fallback options that allow you to continue making real-time authorization decisions when your systems are down", and the monitoring section says: "Don't rely on approved to gauge integration health because Autopilot can approve on your behalf during an error condition." The page gives the signal to watch: request_history.reason set to webhook_error or webhook_timeout. An authorization with approved: true and webhook_error means the money moved without your check.

The lesson generalises. If you let a check fail open, you need a second signal that tells you it did. A dashboard that only shows approvals will look healthy during the outage.

LiteLLM is open in documented cases and closed behind a flag

LiteLLM's budgets and rate limits documentation, read on 15 October 2026, describes both directions. Without a database, "litellm_settings.max_budget fails open there rather than erroring", and "the global budget check is skipped and requests keep being served past the limit". Key, team and user budgets are not enforced either, because "virtual keys cannot be resolved without a database".

The closing switch is a setting under general_settings in config.yaml:

general_settings:
  fail_closed_budget_enforcement: true

With it enabled, "every budgeted request validates spend against the authoritative database before being admitted", and a request whose spend cannot be verified "is rejected with a 503 instead of being admitted on an unverifiable budget". The docs say it is off by default and recommend leaving it off to "keep healthy under-budget traffic entirely off the database". That is the trade stated plainly: the closed mode validates spend against the database on every budgeted request.

The mechanics of LiteLLM's key budgets are covered in LiteLLM budget per key. A gateway-side spend limit from another vendor, and what it does when it cannot count, is in Cloudflare AI Gateway spend limits.

Which way each dependency on a spend path should fail

The three systems above cover one kind of dependency each. An agent spend path usually has more. The table is a recommendation, not a quotation: the reasoning is that a dependency should fail open only if its absence cannot cause money to move that the limit would have stopped.

Two rows deserve a note. The audit-write row is closed because spend with no record cannot be reconstructed later, and a record written before the action can still refuse it when the write fails. And the provider-timeout row is the quiet one: a request that timed out may still have been billed. Treating it as zero reopens budget headroom that was in fact spent.

Fail closed has a price, so make the price visible

A gateway that fails closed turns a dependency outage into an agent outage. Three habits keep that tolerable.

1. Scope what closes. Close the spend path, not the whole service. Reads, health checks and the ability to stop an agent should not depend on the thing that is down.
2. Return a distinct, countable error. A 503 with a stable code per cause is an alert source. Envoy's header and Stripe's webhook_timeout reason are the same idea from the other side.
3. Watch for silent outages. A refusal caused by an unavailable ledger cannot be written to that ledger. The signal has to come from outside it: a health endpoint, a metric, a log line.

None of this needs a particular product. It needs the question asked per dependency, and the answer written next to the config.

Where Mandare fits, and the limits of the current release

Mandare's gateway (the "door") is written to fail closed on the spend path. This section applies to the CLI at v0.1.0, which npm view @mandarelabs/cli version reported on 15 October 2026, checked against commit 327442e of the public repository.

What the code and docs say, with the file:

• The documentation index lists "Ledger unavailable → no action" and "Witness dead → high-value actions refuse (and the kill switch still works)". In packages/gateway/src/server.ts, a failure to append the reservation entry returns 503 with the message ledger unavailable — refusing to act (fail-closed).
• packages/gateway/CLAUDE.md lists the closed conditions: no mandate, no credential, no FX rate or a stale projection return 503, and an unpriced model on a direct provider is denied, with the reason given as "no price = no metering = no spend".
• Provider failures settle conservatively. The red-team test provider-failure.test.ts hangs the provider, expects a 502, and asserts the settled cost equals the reserved estimate and is greater than zero.
• If the result entry cannot be written after a call executed, the gateway sets a halted flag and answers every later request with 503. Its source comment calls "executed-but-unrecorded" the one state it refuses to continue from, and the error text tells the operator to restart after investigating. GET /healthz reports halted and spend_path_open.
• Witness gating applies to actions above the mandate's approval threshold. The environment reference gives MANDARE_WITNESS_ACK_TIMEOUT_MS a default of 1500, and "no verified ack ⇒ the action is refused", returned as 503 WITNESS_UNAVAILABLE.

The limits, next to the claims they limit:

• Witness gating covers high-value actions only. Below the threshold, calls proceed while the witness is down. The threat model states the cost: between witness ticks (default 1 second) there is a sub-second window where truncation before the first witnessing is possible. Detecting deleted audit log entries shows what a witness does catch.
• A halted gateway is a stopped gateway. The flag lives in the gateway process and clearing it is a restart, so an unattended fleet needs an operator or a supervisor watching /healthz.
• The card rail is tested against a Stripe mock that signs webhooks the way Stripe does. The live Stripe test-mode run has not happened. docs/CARD-RAIL.md also says that webhook-timeout declines "happen outside the door and are not ledger entries", and that fail-closed on a timeout depends on the operator setting the dashboard default to decline. That setting lives in the operator's Stripe dashboard, as quoted above.
• The fail-closed behaviour is exercised by red-team tests written by the project and run in CI. Two AI-assisted review passes have been done; there is no external audit yet.
• The kill switch is local and offline, with "No cloud round-trip to jam" in the words of the documentation index. That is a claim about mandare kill and the door, not about your agent's own process, which may keep running while the door refuses it.

For the shape of the whole model, the concepts page lists each door and what it checks.