Concept · Building with AI

Rate limit

Also called: rate limits, rate limited, rate limiting, usage limit, usage limits, quota, throttling, too many requests

A cap on how many requests or tokens an account can send to an API in a set period. Requests over the cap fail with an error until capacity frees up.

For example: A product goes viral and sends ten times its usual requests to a model provider. The provider starts refusing some of them, and users see errors until the team asks for a higher limit.

Connects to 5

Page last changed · Suggest a correction