Concept · Building with AI
Rate limit
Also called: rate limits, rate limited, rate limiting, usage limit, usage limits, quota, throttling, too many requests
A cap on how many requests or tokens an account can send to an API in a set period. Requests over the cap fail with an error until capacity frees up.
For example: A product goes viral and sends ten times its usual requests to a model provider. The provider starts refusing some of them, and users see errors until the team asks for a higher limit.
Connects to 5
- API Concept
- Token Concept
- Usage-based pricing Concept
- Scaling Concept
- Model provider Concept