What Is Rate Limiting?

Rate limiting is the practice of capping how many requests a client may make to a service within a given window of time. When the cap is exceeded, the service rejects or delays further requests instead of processing them. It is one of the most common protections in front of any public API, login form, or shared backend.

If you have ever seen the message "Too Many Requests" or hit a wall after refreshing a page too fast, you have met a rate limiter. This guide explains how the main algorithms work, why services rely on them, and the mistakes that trip up both API providers and the clients who consume them.

What Rate Limiting Actually Does

A rate limit is a rule of the form N requests per unit of time — for example, 100 requests per minute, or 5 login attempts per hour. The service tracks how many requests a given identity has made, and once that identity crosses the threshold, the server stops accepting new requests until the window resets.

The "identity" is whatever the service uses to group requests together: the client IP address, an API key, an authenticated user ID, or an OAuth token. The choice of key matters. Limiting by IP alone can unfairly punish many users behind a shared corporate NAT or proxy, while limiting by API key gives each integration its own budget.

Rate limiting is related to, but distinct from, throttling and load shedding. Throttling usually means slowing requests down by adding delay rather than rejecting them, and load shedding means dropping requests when a system is already overloaded. The terms overlap in casual use, but a rate limit is specifically a defined ceiling tied to an identity and a window.

How Rate Limiting Works: The Core Algorithms

Several well-established algorithms implement rate limits, each with different trade-offs in accuracy, memory use, and how they handle bursts.

Fixed Window Counter

The simplest approach. The server keeps a counter for each identity that resets at the start of every fixed window (say, every minute on the clock). Each request increments the counter; when it exceeds the limit, requests are rejected until the window flips. It is cheap and easy, but it has a well-known flaw: a client can send a full window's worth of requests at the very end of one window and another full window's worth at the start of the next, briefly doubling the intended rate at the boundary.

Sliding Window

A sliding window smooths out that boundary problem by considering a rolling time range rather than a hard clock reset. Implementations either log individual request timestamps (the sliding window log) or blend the counts of the current and previous fixed windows, weighted by how far into the window you are (the sliding window counter). This gives more accurate enforcement at the cost of more bookkeeping.

Token Bucket

The token bucket models a bucket that holds a fixed number of tokens and refills at a steady rate. Each request consumes one token; if the bucket is empty, the request is rejected. Because the bucket can hold accumulated tokens up to its capacity, this algorithm naturally allows short bursts while still enforcing a long-run average rate. It is widely used precisely because real traffic is bursty.

Leaky Bucket

The leaky bucket is the inverse idea: requests enter a queue and are processed, or "leak out," at a fixed rate. If requests arrive faster than they leak and the queue fills, new requests overflow and are dropped. Token bucket favors smoothing bursts on the way in; leaky bucket favors smoothing output to a constant rate.

The table below summarizes the trade-offs.

The 429 Status Code and Rate Limit Headers

When a client exceeds a limit, the standard HTTP response is status code 429 Too Many Requests, defined in RFC 6585. The response often includes a Retry-After header telling the client how long to wait — either a number of seconds or an HTTP date — before trying again.

Many APIs also expose the client's current standing through informational headers so callers can self-regulate before they get rejected. There is no single mandated standard, but a widely seen convention looks like this:

HTTP/1.1 200 OK
RateLimit-Limit: 100
RateLimit-Remaining: 42
RateLimit-Reset: 30

Here the client is allowed 100 requests, has 42 left, and the window resets in 30 seconds. Some providers use the older X-RateLimit-* prefix for the same values. Always read the provider's documentation, since header names and reset semantics differ. For a quick reference on what 429 and related codes mean, the HTTP Status Codes tool lists them all.

Why Rate Limiting Matters

Rate limiting serves several purposes at once. The most obvious is protecting capacity: a finite pool of CPU, memory, and database connections can be exhausted by a single aggressive client, degrading service for everyone. A limiter keeps any one caller from monopolizing shared resources.

It is also a frontline security control. Throttling login and password-reset endpoints slows brute-force and credential-stuffing attacks, and limiting expensive endpoints blunts denial-of-service attempts. It does not replace a web application firewall, but it raises the bar significantly.

For commercial APIs, rate limits underpin fair usage and billing tiers: a free plan might allow 60 requests per minute while a paid plan allows thousands, and the limiter enforces the difference. Predictable limits also give the provider cost control, since traffic translates directly into compute and bandwidth spend.

Where Rate Limiting Lives in a System

Rate limiting can be enforced at many layers. Edge networks and CDNs often apply limits before traffic ever reaches your origin. API gateways and reverse proxies such as Nginx or Envoy enforce limits per route. Application code can apply fine-grained, business-aware limits, such as treating a costly operation differently from cheap reads.

In a single-server setup, an in-memory counter is enough. But once you run multiple instances behind a load balancer, each instance has its own view, so the effective limit becomes the sum across instances. To enforce a true global limit you need shared state, commonly an in-memory store like Redis that all instances consult. That central store becomes a critical dependency and must itself be fast and highly available.

Common Pitfalls

A few mistakes show up repeatedly when teams implement or consume rate limits.

Clock-boundary bursts. Fixed-window limiters allow up to double the intended rate around the reset. If precise enforcement matters, use a sliding-window or token-bucket approach instead.

Trusting client IPs blindly. Behind proxies and CDNs, the immediate connection IP is the proxy, not the user. Read the forwarded client address from a trusted header, and do not trust that header on the open internet, where it can be spoofed.

No coordination across instances. Per-instance in-memory counters silently multiply your limit by the number of servers. Decide deliberately between per-instance and global enforcement, and use shared state when you need the latter.

Clients that ignore the headers. Consumers who do not honor Retry-After or who retry instantly with no backoff make congestion worse. The correct behavior is exponential backoff with jitter: wait a growing, slightly randomized interval between retries so that many clients do not all retry in lockstep.

One-size-fits-all limits. A login endpoint, a search endpoint, and a static asset have very different costs and abuse profiles. Tuning limits per endpoint and per plan is more work but far more effective than a single global number. For a deeper look at choosing strategies, see the guide on rate limit strategies for APIs.

Frequently Asked Questions