Documentation menu
Development guide

Limits

Choose your inference setup

Check the admission limits that apply to your inference requests.

08 / 09

This guide explains request limits in the development app. Your effective limits can differ from the defaults below.

Check the admission limits that apply to your inference requests. These controls determine whether a request can proceed; they are not an invoice.

Platform defaults

DimensionDefault
Requests per minute120
Requests per day25,000
Tokens per day10,000,000
Monthly spend cap$250
Request payload ceiling1,048,576 bytes

The default policy is enabled. An account-specific policy can override every dimension, so your effective limits may differ from this table.

Read your effective policy

The authenticated account can read its own policy at GET /me/limit-policy.

Response valueMeaning
source: platform_defaultNo account-specific policy row exists; the platform defaults apply.
source: tenant_policyAn account-specific policy applies.
A null token or spend capThat dimension is unlimited.

GET /me/limit-policy is a relative API path. To use this endpoint, you need the customer API base URL and authentication details. The development app does not expose a limits panel yet. Until you can call this endpoint, do not assume the defaults are your effective policy.

Respond to a limit error

Limit checks happen after authentication and before a provider call. An admission rejection does not create a provider-loss measurement.

ResponseReasonSupported response
413payload_too_largeThe oversized payload cannot be retried unchanged. Keep the request within the effective payload ceiling.
429rate_limitedRetry after the limit resets.
429quota_exceededRetry after the quota resets.
429spend_cap_exceededRetry after the spend cap resets.
503policy_unavailableThe effective policy could not be read or validated, so no provider was called. Wait for Retry-After before trying again; contact support if the failure persists.

Every JSON error includes code and message. Limit rejections (413 or 429) also include reason and limitType, and can include limit, used and resetAt. A Retry-After header is set when available; use the supplied reset information for the retry. A policy-read failure (503) supplies Retry-After: 1 and does not include limit or usage fields.

If neither resetAt nor Retry-After is present, do not loop the same request. Use limitType to identify whether the minute, day, token, or monthly spend window was exceeded, wait for that window to reset, and then try once.

If the needed limit is not documented

Support and contact explains how to get help with limits.

Continue