Exponential Backoff and Jitter

1. Decide whether a retry is appropriate

A retry is another request, not a free recovery step. It can help when the failure is temporary, such as a connection interruption, a timeout, HTTP 408, HTTP 429, or some server errors. The exact retryable responses remain provider and operation specific.

The operation must also be safe to replay. A read is often idempotent, but a write can create a duplicate when the server completed the work and the response was lost. Use provider-supported idempotency keys, resource versions, conditional requests, or application-level deduplication where appropriate. Do not retry authentication, validation, permission, or other permanent failures without evidence that the condition changed.

• Classify the response or transport failure before choosing a retry.
• Check whether the operation is idempotent or protected against duplicate execution.
• Confirm that enough of the caller's deadline remains for another useful attempt.
• Follow the provider's documented retryable statuses and timing instructions.

2. Count attempts and retries consistently

An attempt is any request sent to the dependency. A retry is an attempt made after an earlier attempt fails. Under that definition, four maximum attempts means one original request plus no more than three retries.

Libraries do not all use the same configuration names. A max retries setting of four can mean five total attempts, while a max attempts setting of four can mean four total attempts. Verify the actual library contract and use both numbers in logs and dashboards.

total_attempts = 1 original attempt + retry_attempts

Example policy:
maximum_attempts = 4
maximum_retries  = 3

3. Start with capped exponential backoff

Exponential backoff increases the nominal delay after each failure. A multiplier of two doubles the pause for each later retry. A maximum delay stops individual pauses from growing without bound.

In the formula below, n is one for the first retry after the original request. The cap limits the nominal delay, not the complete operation. Request duration, queue time, and every earlier pause still consume the caller's total deadline.

nominal_delay(n) = min(
  maximum_delay,
  initial_delay x multiplier^(n - 1)
)

4. Understand why retries collide

Imagine hundreds of clients losing access to the same service at nearly the same time. If each client waits exactly 250 milliseconds, then 500 milliseconds, then 1 second, they wake in coordinated waves. Exponential growth reduces how often the waves arrive, but deterministic timing does not break their synchronization.

Those waves can overload a recovering service, cause another round of failures, and create a retry storm. Reaching the maximum delay can make the pattern worse because every later retry uses the same capped pause. Jitter spreads the work across time by choosing a random delay independently for each retry.

5. Full Jitter uses the widest range

Full Jitter chooses a delay from zero through the current nominal backoff. If the nominal delay is 2 seconds, one client might wait 180 milliseconds and another might wait 1.7 seconds. The average of a uniform range is half the nominal delay, but an individual run can land anywhere in the range.

AWS's published contention simulation found that adding jitter substantially reduced work compared with deterministic exponential backoff in its scenario. Throttle's retry strategy simulation likewise shows how randomization changes one defined fixed-window workload. Neither result makes Full Jitter universally best, but both demonstrate why timing distribution matters.

nominal = capped exponential delay
sleep = random_uniform(0, nominal)

expected sleep = nominal / 2

6. Equal Jitter preserves a minimum pause

Equal Jitter keeps half of the nominal delay and randomizes the other half. For a 2-second nominal delay, the actual range is 1 through 2 seconds. This preserves a larger minimum pause than Full Jitter while still spreading client wake-up times.

The expected delay is three quarters of the nominal value. That extra waiting can be useful when a service needs recovery time, but it also consumes more of the caller's deadline. Choose the policy from the failure mode and provider guidance rather than the formula's name.

nominal = capped exponential delay
sleep = nominal / 2 + random_uniform(0, nominal / 2)

expected sleep = nominal x 3 / 4

7. Other jitter policies use different tradeoffs

A percentage policy varies around the nominal delay, such as plus or minus 20 percent. gRPC documents this kind of randomized backoff in its retry example. It provides a narrower spread than Full Jitter and can exceed the nominal value unless another cap is applied.

Decorrelated Jitter makes the next range depend on the previous randomized delay. Implementations vary, so document the exact formula, bounds, and random-number behavior. The important property is that clients do not all calculate the same wake-up time after a shared failure.

percentage jitter example:
sleep = random_uniform(nominal x 0.8, nominal x 1.2)

Apply an explicit maximum when the upper range must remain capped.

8. Honor Retry-After and server pushback

RFC 9110 defines Retry-After as either a number of seconds or an HTTP date. When a provider sends a valid value, treat it as the earliest permitted retry time. A local policy can wait longer, but it should not wake earlier.

Read a new value from each response instead of assuming the first value applies forever. If the instructed wait cannot fit inside the remaining operation deadline, stop and return or defer the work instead of issuing a late request.

server_floor = parsed Retry-After, when valid
local_delay = jitter(capped exponential delay)
actual_wait = max(server_floor, local_delay)

if actual_wait + request_budget > remaining_deadline:
  do not retry

9. Bound the complete retry policy

A maximum delay controls one pause. It does not limit how many requests are sent or how long the complete operation lasts. Production policies need several independent bounds so retries stop when recovery is no longer useful.

Retries can also multiply across service layers. If five layers each perform three attempts, a deeply failing dependency can receive far more traffic than the original call count suggests. Prefer retrying at one deliberate layer, and use a shared retry budget or token bucket when many calls can fail together.

• Maximum attempts: limits requests sent for one logical operation.
• Maximum delay: caps one calculated backoff pause.
• Total deadline: includes request execution, waits, queueing, and all attempts.
• Retry budget: limits aggregate retry traffic across many operations.
• Concurrency bound: prevents delayed work from waking into an unbounded request wave.

10. Work through a six-retry schedule

Suppose the initial delay is 250 milliseconds, the multiplier is two, the maximum delay is 30 seconds, and the policy allows six retries after the original request. The nominal waits are 250, 500, 1,000, 2,000, 4,000, and 8,000 milliseconds. They total 15.75 seconds before request execution time is added.

With Full Jitter, each wait ranges from zero through its nominal value. The expected total is 7.875 seconds, but one real run can be shorter or longer. With Equal Jitter, each wait ranges from half through all of its nominal value, producing an expected total of 11.8125 seconds. Expected values describe many samples, not one guaranteed schedule.

Retry   Nominal   Full Jitter   Equal Jitter
1       250 ms    0-250 ms      125-250 ms
2       500 ms    0-500 ms      250-500 ms
3         1 s     0-1 s         0.5-1 s
4         2 s     0-2 s           1-2 s
5         4 s     0-4 s           2-4 s
6         8 s     0-8 s           4-8 s

Nominal wait total: 15.75 s
Full Jitter expected total: 7.875 s
Equal Jitter expected total: 11.8125 s

11. Measure retries as added work

A successful logical call can hide several failed attempts. Count original calls, total attempts, successes after retry, final failures, retry delays, deadline exhaustion, and server pushback separately. The ratio of total attempts to original calls reveals retry amplification.

Keep provider, operation, status or transport error, attempt number, chosen delay, and remaining deadline in structured telemetry without recording secrets. Alert on sustained amplification, exhausted retry budgets, and growing delayed queues rather than treating every isolated retry as an outage.

retry_amplification = total_attempts / original_calls

1.00 = no retries
1.25 = one extra attempt for every four original calls, on average

Backoff and jitter checklist

• List the exact transient errors and status codes eligible for retry.
• Protect replayed writes with idempotency or conditional controls.
• Define whether the configured limit counts retries or all attempts.
• Choose an initial delay, multiplier, maximum delay, and jitter formula.
• Honor valid Retry-After and provider-specific pushback.
• Cap attempts and complete elapsed time, not only individual delays.
• Retry at one deliberate layer and limit aggregate retry traffic.
• Bound concurrency when delayed work resumes.
• Record attempts, waits, eventual success, final failure, and amplification.
• Test failure handling with deterministic random seeds before production.