AI Guides · 10 min read

What Are AI API Rate Limits? RPM, TPM, 429 Errors, and Batching

AI API rate limits cap requests and tokens per minute. See RPM vs TPM, usage tiers, what a 429 means, and when batch APIs avoid live limits.

Isometric illustration of glowing amber orbs passing through a slate metering gate, with a curved side channel delivering a cluster of orbs into a holding basin

AI API rate limits cap how many requests (RPM) and tokens (TPM) an organization or project can send within a fixed time window, returning an HTTP 429 error when any single threshold trips. Providers enforce these quotas to prevent abuse, keep access fair, and protect shared model infrastructure, while offering batch endpoints for offline workloads.

In my own work designing API integrations at Lotook, most production surprises happen when teams track monthly spend while ignoring per-minute throughput and short burst windows. Because providers evaluate several independent counters on every call, understanding each dimension before traffic scales keeps background jobs and user-facing features stable.

What do RPM, TPM, and RPD mean in AI APIs?

AI API rate limits measure traffic across multiple parallel dimensions, including requests per minute (RPM), tokens per minute (TPM), and daily quotas such as requests per day (RPD) or tokens per day (TPD). Depending on the endpoint, providers may also enforce unit caps like images per minute (IPM) or audio minutes per minute on streaming audio models.

If you are newer to how providers count text volume, see our guide on what a token is in AI. Some vendors track combined TPM, while others split token budgets into input tokens per minute (ITPM) and output tokens per minute (OTPM).

The most important operational rule is that exceeding any single applicable limit triggers a rate-limit error even when every other metric still has plenty of room. OpenAI’s documentation illustrates this with a simple example: sending 20 tiny requests in a minute exhausts a 20 RPM cap even if your TPM budget remains mostly unused. Conversely, sending a handful of massive prompts with large output allocations can exhaust TPM well before you get close to your RPM ceiling.

How do OpenAI rate limits and usage tiers work?

OpenAI enforces rate limits at the organization and project levels rather than the individual user level, and those limits vary by model and usage tier. As of November 17, 2026, OpenAI’s rate-limit documentation notes that some model families share a combined cap under a shared limit: for example, if shared TPM is 3.5 million, calls to every model in that shared group count toward the same 3.5 million pool.

Certain workloads also have separate buckets. Long-context models (such as GPT-5.5 in OpenAI’s docs) can carry a dedicated rate limit for long-context requests in the developer console. Vector store ingestion is capped at 300 RPM per vector store ID, shared across /vector_stores/{vector_store_id}/files and /.../file_batches, so OpenAI recommends file_batches for larger ingests.

OpenAI assigns organizations to paid tiers automatically as total credit purchases cross each threshold, unlocking higher rate limits and higher approved monthly usage limits. An approved monthly usage limit is distinct from any optional spend limit you configure yourself, and seeing a model in a rate-limit table does not confirm your organization or key has access to that model. For broader cost context across providers, see our AI API pricing comparison.

Tier Qualification Usage limits
Free User must be in an allowed geography $100 / month
Build $5 in total credit purchases $500 / month
Launch $100 in total credit purchases $5,000 / month
Grow $500 in total credit purchases $200,000 / month

OpenAI usage tiers and monthly usage limits as of November 17, 2026, per OpenAI’s rate-limits guide.

Within those paid tiers, OpenAI publishes Standard rate limits for models such as Astra, Sol, Terra, and Luna. These Standard figures differ from Ultrafast rate limits, and OpenAI advises checking your live numbers under Settings > Organization > Limits.

Tier Model RPM TPM
Build Astra, Sol, Terra 5,000 1,000,000
Build Luna 5,000 2,000,000
Launch Astra, Sol, Terra 10,000 4,000,000
Launch Luna 10,000 10,000,000
Grow Astra, Sol, Terra 15,000 40,000,000
Grow Luna 30,000 180,000,000

Published OpenAI Standard rate limits as of November 17, 2026, per OpenAI’s developer documentation.

Anthropic Claude rate limits, caching, and ramp limits

Anthropic measures Messages API rate limits in RPM, ITPM, and OTPM for each model class, returning a 429 response naming the exceeded limit when traffic crosses a threshold. When the response includes a retry-after header, Anthropic’s documentation instructs callers to wait at least that many seconds before retrying.

Anthropic also uses cache-aware ITPM accounting, which changes how prompt caching affects throughput. Default ITPM counts only input_tokens appearing after the last cache breakpoint plus cache_creation_input_tokens. Crucially, cache_read_input_tokens do not count toward ITPM. Anthropic estimates ITPM at the start of each request and adjusts the count during execution, giving cache-heavy workloads significantly more effective input capacity. (Do not assume OpenAI or Google Gemini exempt cached reads from token limits in the same way unless their own docs explicitly state it.)

Even while staying below ITPM and OTPM, sharp traffic spikes can trigger Anthropic’s ramp limits. A burst spanning just a few seconds can count as a sharp increase. Anthropic calculates the ramp limit as your organization’s highest recent per-minute usage of that model plus a fixed step; each minute you can climb one step above your recent peak until reaching your full tier limit, and the allowance never drops below a baseline starting floor even after idle periods.

Tier Model RPM ITPM OTPM
Start Claude Opus 5.5 4,000 2,000,000 400,000
Start Claude Sonnet 5.5 4,000 2,000,000 400,000
Build Claude Opus 5.5 10,000 5,000,000 1,000,000
Build Claude Sonnet 5.5 10,000 5,000,000 1,000,000
Scale Claude Opus 5.5 13,000 10,000,000 2,000,000
Scale Claude Sonnet 5.5 13,000 10,000,000 2,000,000

Representative Anthropic Messages API tier limits as of November 17, 2026, per Anthropic’s rate-limits documentation. Check your console and the full official table for additional model classes and account-specific caps.

Google Gemini API rate limits and spend caps

Google Gemini API rate limits apply per project instead of per API key, typically tracking RPM, input TPM, and RPD. Daily RPD quotas reset at midnight Pacific time. Because Google notes that published rate limits vary by model and tier, are more restricted on experimental and preview models, and are not guaranteed, you should inspect your active model numbers directly in Google AI Studio rather than relying on static estimates.

Gemini also enforces model-specific and tool-specific quotas. Image-generating models can carry IPM limits and certain models include TPD limits, while built-in tools have their own caps: as of November 17, 2026, Grounding with Google Search on Gemini 3 and newer models defaults to 1,500 RPD per project on the Gemini Developer API.

Alongside request and token caps, Google applies spend-based rate limits evaluated over a rolling 10-minute window, depending on billing history and tier:

Usage tier Spend rate limit (per 10 minutes)
Free N/A
Tier 1 $10
Tier 2 $50
Tier 3 $200

Google Gemini API rolling 10-minute spend limits as of November 17, 2026, per Google’s rate-limits documentation.

Hitting a Gemini spend-based limit returns 429 RESOURCE_EXHAUSTED. Google’s guidance is to pause and retry after a short interval, reduce the pace of expensive calls using smaller context windows or shorter outputs, and request a limit increase if normal production traffic consistently hits the ceiling.

What does a 429 error mean, and how should you retry safely?

An HTTP 429 response means a caller exceeded an API quota, rate threshold, or billing cap, though not every 429 should be retried. OpenAI’s Help Center explicitly warns that a 429 can signal a temporary rate limit, an exhausted prepaid balance, a configured hard spend limit, or a monthly usage limit. Retrying a billing or hard-spend 429 simply burns attempts; fixing it requires adjusting billing or limits.

Distinguishing 429 rate errors from 503 overload

Providers use specific error payloads to separate traffic-rate spikes from temporary server capacity issues. In OpenAI’s API, 429 paired with rate_limit_error and slow_down indicates your request rate ramped up too quickly, which can happen even inside RPM and TPM bounds. By contrast, 503 paired with service_unavailable_error and server_is_overloaded means the target model is temporarily overloaded. On Anthropic, 429 errors identify which limit tripped, while Gemini spend caps return 429 RESOURCE_EXHAUSTED.

Bursts, max_tokens, and retry mechanics

Short bursts frequently trip rate limits because providers enforce ceilings over sub-minute intervals: OpenAI notes that a 60 RPM limit may also be checked over 1-second windows. Output budgets matter as well, because OpenAI’s rate-limit calculation can take the maximum of max_tokens and a character-count estimate. Setting max_tokens close to your expected completion size prevents artificially inflating token reservations.

When a retryable 429 or 503 occurs, follow these steps:

  1. Inspect the error code first so you never retry prepaid balance, usage-limit, or hard spend-limit errors.
  2. Honor Retry-After or retry-after whenever present, waiting at least the specified seconds before the next attempt.
  3. Use exponential backoff with jitter when no header is provided, increasing the delay after each failure, adding a small random delay so workers do not collide, and capping total attempts.
  4. Account for official SDK retries by disabling or coordinating built-in SDK retry loops if you add custom retry logic, preventing multiplied request storms.
  5. Smooth client concurrency by pacing queued jobs over the full minute instead of firing a minute’s quota in one second.

When does a batch API avoid live RPM and TPM limits?

An asynchronous Batch API avoids live synchronous RPM and TPM budgets by queueing non-interactive jobs for offline completion instead of demanding immediate responses. OpenAI’s Batch API lets you submit large collections of requests without consuming synchronous rate limits; batch throughput is governed instead by total queued input tokens per model, where pending batch jobs count against the queue limit and completed jobs drop off. Google’s paid Gemini Developer API pricing documentation similarly lists a Batch API path with a 50% cost reduction (check Google’s official docs for current Gemini batch mechanics).

Do not confuse a Batch API with packing multiple tasks into one live synchronous prompt. OpenAI notes that if you hit RPM while still holding TPM headroom, combining multiple smaller tasks into one synchronous request can process more tokens per minute, especially on smaller models. However, packed synchronous calls still consume your live TPM budget and remain subject to burst and max_tokens rules. For broader strategies on matching model size and batching to workload needs, see our guide on how to reduce AI API costs. When organic production volume outgrows your tier, submit a formal limit-increase request through the provider’s official console rather than trying to work around account controls.

FAQ

Why am I getting a 429 error below my RPM and TPM limits?

Several mechanics can trigger a 429 even when your per-minute average looks safe. Providers enforce short burst windows (such as 1-second checks on a 60 RPM limit), ramp limits on sudden traffic spikes like Anthropic’s ramp rule or OpenAI’s slow_down, shared family pools, rolling 10-minute spend limits on Gemini, or exhausted prepaid balances and spend caps.

Should I retry every 429 error from an AI API?

No, you should inspect the error payload before retrying any 429 response. Temporary rate-limit 429s and 503 overload errors are safe to retry after honoring Retry-After or applying exponential backoff with jitter. Errors caused by exhausted prepaid credits, monthly usage limits, or configured hard spend limits require a billing or limit update and will keep failing if retried.

Do prompt cache reads count toward Anthropic ITPM limits?

No, Anthropic’s cache-aware ITPM calculation excludes cache_read_input_tokens from your input tokens per minute quota. Only uncached input_tokens after the last cache breakpoint and cache_creation_input_tokens count toward ITPM. Because Anthropic estimates ITPM at request start and adjusts during the call, workloads with high cache hit rates gain extra input headroom.

Are AI API rate limits shared across API keys?

Yes, creating extra API keys under the same organization or project does not give you separate rate-limit buckets. OpenAI defines rate limits at the organization and project levels rather than the user level, and Google Gemini explicitly applies limits per project rather than per API key. All keys inside that scope share the same quota.

How does setting max_tokens affect OpenAI rate limits?

OpenAI notes that its rate-limit calculation can use the maximum of your max_tokens setting and an estimate derived from character count. Setting max_tokens far higher than your actual expected response inflates the tokens reserved against your TPM limit and increases the chance of token-rate errors. Keeping max_tokens realistic protects live throughput.

Sources

Written by

Chunyu Cai

Chunyu Cai is the CEO of Lotook, LLC and an entrepreneur who writes about AI models, agents, and how they change everyday work. Chunyu holds a master's degree from Imperial College London.