How to Fix OpenAI API 429 Too Many Requests

429 Too Many Requests It doesn't just mean "requested too quickly". In the LLM API, it may also come from excessive concurrency, token rate, account quota, busy single model, or double retries between the application and the SDK.

How do you fix OpenAI API 429 errors?

First inspect the error response,Retry-After, the request ID, and current concurrency. Do not retry balance or account-status errors. For an explicit rate limit, wait according to Retry-After or use capped exponential backoff. If the model is busy, queue requests or reduce concurrency. Disable duplicate retries across the SDK and application layers so one failure does not fan out into more 429 responses.

Save complete error evidence first

Record at least the following fields:

Occurrence time：
model ID：
HTTP Status：429
Error type / code：
Retry-After：
Request ID：
Number of concurrencies at that time：
input Token Estimate：
Number of retries：

If the balance, quota or account status clearly appears in the error text, deal with the account problem first; if rate limit, requests per minute or tokens per minute appear, adjust the request rhythm.

Five common sources of 429

Try one short, non-streaming request with the same model. If even a single request consistently returns 429, check quota and model status first. If failures occur only with concurrent requests, check queuing and rate limits.

How do I handle account concurrency limits?

When you see concurrency limit exceeded for account, please retry later, first investigate too many in-flight requests. Do not assume insufficient funds or try to bypass limits by rotating keys or retrying immediately. This wording is not a standardized error code; use the responding domain, HTTP status, request ID and provider documentation to identify the restriction.

1. Pause new tasks and bulk retries. Let in-flight requests finish or cancel them normally through the client; do not restart production services as the first response.
2. Check whether terminals, agents, scheduled jobs or app instances share the account. Streaming requests may occupy concurrency slots until completion or cancellation.
3. After the server's required recovery interval, try one short task with the same model and protocol. Generation requests may incur charges; do not repeatedly run the full diagnostic suite.
4. Once a single request succeeds, queue work within the provider-confirmed shared scope and gradually restore concurrency. A process-local semaphore cannot control other processes; multiple instances need a shared queue or central scheduler.
5. If a single request still fails, send the provider a redacted timestamp, model ID, error body and request ID to check account, group or upstream limits. Never publish full keys or business content.

OpenAI documents organization- and project-level limits. RFC 6585 does not prescribe how servers identify users or count requests. Third-party gateways may use different rules, so this guide does not promise a fixed HK concurrency limit or identify a failing upstream from the error text alone.

Correct retreat strategy

Server returns Retry-After Priority is given to waiting by this value. This request header may be either a number of seconds or an HTTP date; it should first be parsed into the number of seconds to wait. Without this field, exponential backoff with random jitter is used:

No. 1 times：approx. 1 seconds
No. 2 times：approx. 2 seconds
No. 3 times：approx. 4 seconds
Stops after reaching the maximum number of attempts and returns an explicit error

Don't have all instances retry at the same second. Random jitter can reduce the second traffic spike caused by synchronous retries.

from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
import random
import time

MAX_WAIT_SECONDS = 60.0

def parse_retry_after(value: str | None) -> float | None:
    if not value:
        return None
    try:
        return max(0.0, float(value))
    except ValueError:
        try:
            retry_at = parsedate_to_datetime(value)
            if retry_at.tzinfo is None:
                retry_at = retry_at.replace(tzinfo=timezone.utc)
            return max(0.0, (retry_at - datetime.now(timezone.utc)).total_seconds())
        except (TypeError, ValueError, OverflowError):
            return None

def backoff(attempt: int, retry_after_header: str | None = None) -> None:
    retry_after = parse_retry_after(retry_after_header)
    if retry_after is not None and retry_after > MAX_WAIT_SECONDS:
        raise TimeoutError("Retry-After Exceeds the allowed waiting time for the current task")
    if retry_after is not None:
        delay = retry_after
    else:
        delay = min(8.0, 2 ** attempt) + random.uniform(0, 0.5)
    time.sleep(delay)

This code is only responsible for parsing waiting times and pauses;attempt from 0 Start. Whether to retry depends on the error type, request idempotence, business deadline and maximum number of attempts.

Avoid repeated retries between the SDK and business layer

By default, OpenAI Python SDK will automatically retry 2 times after the initial request, that is, each SDK call will be tried up to 3 times. If the business layer calls the SDK 4 times in total, it may issue 12 upstream requests at worst.

When building your own retryer, you should explicitly keep only one layer:

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://www.aifast.hk/v1",
    api_key=os.environ["AIFAST_API_KEY"],
    max_retries=0,
)

If you use the SDK default retry, the business layer should not unconditionally loop on the same exception.

How to limit concurrency between batch processing and Agent

Batch processing

• Put the task into the queue;
• Set a fixed concurrency upper limit;
• Move up one gear at a time and watch 429;
• Record the number of successes, the number of 429s and Tokens, instead of just looking at the number of tasks.

Agent and tool invocation

A user task may trigger multiple rounds of model requests and tool postbacks. Limiting "user concurrency" is not the same as limiting "model request concurrency". Should also be set:

• The most LLM round for a single task;
• Maximum number of tool calls for a single task;
• Global model request concurrency;
• The maximum number of retries after failure;
• Single task budget limit.

When not to retry

• The error text states that the balance is insufficient or the account is unavailable;
• The request has reached the business deadline;
• It is impossible to confirm whether operations with side effects have been executed;
• The same model continues to fail and the maximum number of attempts has been reached;
• The business layer does not have idempotent keys, and retrying may result in repeated tasks or repeated deductions.

Acceptance criteria for repair completion

1. Single request and target concurrency complete at least one round of verification respectively;
2. 429 will read Retry-After or enter a capped retreat;
3. There is only a clear set of retry responsibilities between the SDK and the business layer;
4. Monitoring can distinguish the number of requests, tokens, concurrency and retries;
5. It can fail quickly after reaching the retry limit and will not wait indefinitely.

If short requests also persist 429, check firstCurrent models and pricesAnd check the account status; if the error becomes 502, disconnected or read timeout, continue to check502 and stream disconnected troubleshooting.