API Rate Limits Explained

1. A rate limit is a policy, not one number

A statement such as 600 requests per minute names a quota and a time period. It does not, by itself, explain when the period begins, whether unused capacity carries forward, how quickly capacity returns, whether short bursts are accepted, or which consumer and endpoints share the counter.

Treat the complete policy as the contract. Provider documentation might publish only part of it, and additional protections can remain dynamic or undisclosed. A successful response or positive remaining value therefore does not promise that the next request will be accepted.

policy = {
  quota_unit, scope, window_or_refill,
  sustained_rate, burst_capacity, request_cost,
  exceeded_behavior, response_fields
}

2. Why APIs limit traffic

Rate limits protect finite resources and keep one consumer from taking an unfair share. They can reduce overload risk, bound cost, protect a downstream database or provider, control expensive operations, and slow abusive or accidental traffic.

A limiter is only one reliability control. Servers still need bounded work, capacity planning, overload protection, and safe degradation. A request rate can remain constant while its cost rises because payloads grow, caches miss, queries become more expensive, or dependencies slow down.

• Reliability: keep accepted work within sustainable service capacity.
• Fairness: partition capacity among users, tenants, credentials, projects, or IP addresses.
• Cost: constrain billable operations, tokens, bytes, recipients, or compute-heavy endpoints.
• Security and abuse response: reduce automated misuse and accidental request loops.
• Product policy: distinguish tiers, allocations, or permitted integration behavior.

3. Identify the parts of a quota policy

The quota unit might be one request, but it can also be a token, cost point, byte, recipient, row, or weighted operation. The scope identifies what shares the quota. The time rule defines when capacity resets or refills. A burst rule controls how quickly available capacity may be spent.

Also record what happens when the policy is exceeded. A provider can reject immediately, delay work, return a response with retry guidance, lower a dynamic service limit, or close a connection under extreme load. HTTP does not prescribe one counting algorithm.

• Quota: the amount of counted work.
• Partition or scope: the consumer or resource whose work shares a counter.
• Window or refill: how capacity changes with time.
• Burst: how much work can arrive faster than the sustained rate.
• Cost: how many quota units one operation consumes.
• Enforcement response: reject, delay, shape, or apply another provider-defined action.

4. Fixed-window counters

A fixed-window counter groups requests into discrete time buckets, such as each clock minute. The counter starts over when the next bucket begins. This model is simple and efficient, and its reset time is easy to explain.

The boundary creates a burst edge. A client could use the end of one window and the beginning of the next in rapid succession. A policy of 100 requests per minute can therefore accept 100 requests just before a boundary and another 100 just after it, depending on the implementation and other controls.

window = floor(current_time / 60 seconds)
key = consumer + window
allow when counter[key] < 100

Boundary effect:
100 requests at 12:00:59
100 requests at 12:01:00

5. Sliding-window limits

A sliding window evaluates recent activity relative to the current request rather than one shared clock boundary. An exact sliding log can retain request timestamps and count those still inside the trailing interval. A sliding counter can approximate that result from smaller buckets or weighted adjacent windows.

Sliding policies reduce the fixed-window boundary spike, but their storage, accuracy, and implementation cost vary. Provider documentation does not always disclose whether a rolling or sliding label means an exact log, an approximation, or another internal control.

now = 12:01:30
window_start = 12:00:30
count requests with timestamp > window_start
allow when trailing_count < quota

6. Token buckets

A token bucket refills at a configured rate up to a maximum bucket capacity. A request spends one or more tokens. When enough tokens are available, the request can proceed. When the bucket is full after an idle period, the client can spend several tokens immediately and then continue near the refill rate.

The refill rate controls sustained throughput and the bucket size controls stored burst capacity. Weighted operations can spend different token amounts. Amazon API Gateway documents token-bucket throttling with account-level rate and burst targets, plus optional stage and route limits.

tokens = min(bucket_capacity, tokens + refill_rate x elapsed_time)

if tokens >= request_cost:
    tokens = tokens - request_cost
    allow
else:
    limit

7. Leaky buckets and request shaping

Leaky-bucket terminology varies between implementations. The common idea is a controlled outflow: incoming work enters a bounded queue or is measured against a steady drain rate. Work within the permitted burst can be delayed and released at the configured pace; work beyond the bound is rejected.

NGINX documents its request limiter as a leaky-bucket method. By default, excess requests within the configured burst are delayed so processing follows the defined rate, while requests beyond the burst are terminated. Its nodelay option changes how accepted burst requests are released. Always use the behavior documented by the actual product or provider.

incoming requests -> bounded queue -> steady processing rate

queue has room: accept or delay
queue is full: reject

8. Compare the same headline rate

Imagine four policies described loosely as 60 requests per minute. Their average looks similar, but the short-term experience can be different. This is why capacity planning needs more than a normalized requests-per-second value.

These examples illustrate model behavior, not universal provider implementations. A real service can combine policies, use approximate counters, vary limits dynamically, or apply additional controls.

• Fixed window: up to 60 in one clock minute, with a possible boundary spike across adjacent windows.
• Sliding window: no more than 60 inside the applicable trailing 60-second interval.
• Token bucket: refill near 1 token per second with a separate bucket capacity that permits a controlled immediate burst.
• Leaky-bucket shaper: accept a bounded burst into a queue and release work near 1 request per second, rejecting overflow.

9. Expect overlapping limits and weighted work

A request can be evaluated against several counters. A provider might enforce an account-wide request rate, a stricter search endpoint limit, a concurrent-request limit, a daily allocation, and an undisclosed abuse protection at the same time. The tightest applicable policy becomes the immediate constraint.

GitHub documents primary limits by authentication and resource as well as secondary limits that protect the service from patterns such as excessive concurrency and expensive activity. Other providers count tokens, GraphQL complexity, recipients, or endpoint-specific allocations. Preserve each dimension in the client and in operational telemetry.

• Consumer scope: user, credential, installation, project, organization, account, or IP address
• Resource scope: route, endpoint family, model, repository, workspace, or operation type
• Quota dimension: requests, tokens, points, bytes, recipients, rows, concurrency, or spend
• Time behavior: short burst, sustained rate, hourly quota, daily allocation, or dynamic protection

10. Read HTTP feedback carefully

RFC 6585 defines HTTP 429 as an optional response indicating too many requests in a given amount of time. A response can include Retry-After, but the RFC does not define how the server identifies a consumer or counts requests. Providers can also expose limit, remaining, reset, resource, or bucket fields with provider-specific meanings.

The current IETF RateLimit draft defines structured RateLimit-Policy and RateLimit fields, but it remains an Internet-Draft as of May 2026. It also warns that available quota is not granted capacity and that more than one policy can affect a request. Parse only documented formats and retain a safe local pacing policy.

HTTP/1.1 429 Too Many Requests
Retry-After: 30

# Provider-specific fields can add context,
# but their names and formats are not universal.

11. Design clients to stay below the edge

Do not use repeated 429 responses as the normal scheduling mechanism. Pace optional work before exhaustion, centralize quota coordination when several workers share one scope, and keep separate state for independent provider buckets. Bound concurrency even when the published policy only mentions a time-based rate.

When limited, stop the affected bucket, honor valid provider timing, and retry only safe operations within the remaining deadline. Use bounded backoff with jitter when documentation calls for it. Keep user-facing work separate from deferrable queues so background jobs cannot consume all available capacity.

• Normalize documented capacity for planning without discarding its original window and scope.
• Reserve headroom instead of scheduling continuously at the published ceiling.
• Coordinate shared credentials across processes, hosts, and regions when required.
• Queue and smooth deferrable work with bounded workers and explicit priorities.
• Monitor attempts, useful completions, throttles, retries, queue age, and every quota dimension.

Rate-limit checklist

• Record the quota unit, scope, window or refill rate, burst rule, and request cost.
• List every overlapping primary, endpoint, token, concurrency, and secondary policy.
• Do not infer fixed, sliding, token-bucket, or leaky-bucket behavior from a headline rate alone.
• Normalize comparable rates for capacity planning while preserving provider semantics.
• Capture HTTP status, Date, Retry-After, and documented rate-limit fields from one response.
• Pace traffic before exhaustion and reserve operational headroom.
• Bound concurrency and coordinate workers that share one quota scope.
• Retry only safe work with documented timing, backoff, jitter, attempt caps, and deadlines.
• Recheck official documentation when behavior or response fields change.