← Butler

Building Resilient HTTP Clients: Timeouts, Deadlines, Retries, and Circuit Breakers

Four mechanisms that sound simple in isolation and interact in ways that are easy to get subtly wrong — timeouts that don't compose, retries that make an outage worse, circuit breakers with no shared state.

Timeouts vs. deadlines — not the same thing

A per-stage timeout (connect timeout, read timeout) bounds one phase of one attempt. A deadline is a total wall-clock budget for the entire call — every retry attempt and every redirect hop counted against the same clock, started once and never reset. Conflating the two is a common bug: a generous per-attempt read timeout combined with several retries can add up to a caller waiting far longer than they ever agreed to, because nothing was tracking the total. The fix is mechanical once you see it: compute each attempt's actual bound as min(per_attempt_timeout, deadline.remaining), so a per-attempt timeout can never add extra time on top of the total budget, no matter how many retries or redirects happen inside it.

Retries: not every failure deserves one

The reflex "just retry on failure" is right often enough to be dangerous the rest of the time. Two distinctions matter:

• Idempotency. GET/HEAD/OPTIONS are always safe to retry — nothing that reached the server can be confirmed either way, and retrying doesn't make a read any less safe. POST is a write with no idempotency guarantee; auto-retrying it on a 5xx risks a double-charge, a duplicate order, a second email sent — so it shouldn't happen without the caller explicitly opting in per-request.
• Permanent vs. transient failure. Not everything that raises an exception is transient. A connection reset or a timeout might genuinely succeed on the next attempt. A TLS certificate that fails verification — self-signed, expired, wrong hostname — will not become valid a few hundred milliseconds later; retrying it just delays surfacing a real problem behind a generic "retries exhausted" error instead of the actual certificate failure. Getting this distinction wrong in either direction is a real bug, not a style choice: retry too eagerly and you waste time (or worse, mask a security failure); retry too conservatively and a genuinely transient blip becomes a hard failure the caller didn't need to see.

Backoff matters too, separately from whether to retry at all: retrying immediately after a failure just adds more load to a server that's already struggling. Exponential backoff spaces retries out geometrically; jitter (scaling each computed delay by a random factor rather than using it exactly) exists specifically so that many clients failing at the same moment — a common real-world case — don't all retry in lockstep and hit the recovering server with a synchronized thundering herd.

Circuit breakers: stop calling a service that's already down

Retries and backoff help one caller ride out a blip. They don't help when a downstream service is genuinely down for an extended period — every caller still tries, still waits out its timeout, still retries, all against a service that isn't coming back in the next few seconds. A circuit breaker tracks failures per scope (typically per host) and, once a failure threshold is crossed, trips open: every call fails immediately with a clear "circuit open" error instead of waiting out a doomed timeout. After a recovery window, it allows exactly one probe request through — a success closes the circuit again, a failure reopens it. The three states (CLOSED → OPEN → HALF_OPEN) are the standard shape for this; what varies between implementations is scope (per-host vs. one shared breaker per client) and what counts as a "failure" (a raised transport error, a 5xx response, or both).

How Butler puts these together

A Resilience::Deadline is created once per call and threaded through every middleware and every retry/redirect attempt inside it. Per-attempt timeouts are always additionally capped by the deadline's remaining budget. Retries follow the idempotency rule above exactly — GET/HEAD/ OPTIONS retried automatically, POST never auto-retried on a 5xx, PUT/DELETE only when explicitly marked idempotent: true — with exponential backoff and equal-jitter, and Retry-After honored ahead of the computed delay when a server sends one. Certificate verification failures are raised as their own error type specifically so the retry policy can exclude them while still retrying other, genuinely transient TLS failures — the permanent-vs- transient distinction above, enforced in code rather than left as a judgment call at the call site. The circuit breaker is a CLOSED → OPEN → HALF_OPEN → CLOSED/OPEN state machine, scoped per-host by default.

Further reading

• Deadline vs. per-stage timeouts in Butler's architecture docs.
• Retries and circuit breaker configuration in the README.
• Choosing a Ruby HTTP client — how this compares across Butler, Faraday, Excon, HTTParty, and Net::HTTP.
• Back to Butler.