Limits and retries

A retry is not safe for every failure. First determine whether the request was accepted, whether output started, and whether the cause is transient; only then decide whether to send the same request again.

Automatically retry only a request known not to be accepted

A network failure can happen after a request was accepted and started running, so no HTTP response or visible output does not prove that work never began. Automatically retry only when it is known that the request was not accepted. Otherwise check Usage, the request time, and exact model ID, then make an explicit decision without blind replay. Correct a 400, invalid key, insufficient balance, missing permission, or incompatible parameter instead of retrying it.

Use bounded exponential backoff:

1. when the response includes Retry-After, wait for that delay while staying inside the client's total wait budget;
2. otherwise increase the delay after each failure and add a small random jitter;
3. cap the delay, attempt count, and total operation time;
4. return the failure to the calling application after the bound instead of looping forever.

The client chooses those bounds for its task. They are not a service recovery-time promise.

Automatically retry only when request non-acceptance is confirmed.

Separate spend limits from transient 429

For 429, read the public code first. API_KEY_SPEND_LIMIT_EXCEEDED means captured spend, active reservations, and the new request estimate crossed the key budget. Wait for the response's resetAt or change the limit in API keys; exponential backoff does not create more budget.

For another transient 429, honor Retry-After when present and use the bounded policy above, but only before output. Do not calculate resetAt yourself or confuse it with Retry-After: the former belongs to the key's budget window, while the latter supplies a response retry delay.

Check context and output limits

Context size and maximum output depend on the exact model ID. Read context_length, the published completion limit, and supported_parameters from the current catalog or GET /v1/models.

Context includes the messages and other input in the current request; the API does not attach history from previous requests automatically. Send only one supported output-limit field: max_tokens or max_completion_tokens. Errors such as context_length_exceeded, max_output_tokens_exceeded, and OUTPUT_TOKEN_LIMIT_EXCEEDED require a changed input, limit, or model setting rather than the same request again.

Account for timeout stages

Connection wait, time to first output, and idle time inside a stream have separate current limits. The exact boundary therefore depends on the stage and configuration; there is no universal promise in seconds.

Enable streaming for a long response so output can arrive incrementally, but do not treat streaming as a way to disable every timeout. Apply a bounded retry when it is confirmed that the request was not accepted. Otherwise check usage, time, and model and make an explicit decision. If a stream has returned data, preserve the partial response and leave the continuation decision to the user or application.

Handle an interrupted stream separately

Chat Completions completes at [DONE]; Messages completes at message_stop. An earlier connection close means the result is incomplete. Record the last event, model ID, time, and request identifier when available, but do not replay automatically after output.

Once processing ends without confirmed output or usage, its reservation is released without a usage charge. If part of a stream was delivered or usage was confirmed, that portion can be billed. No exact reservation-update time is promised; check Balance and Usage.