Rate limits & budgets
Two independent brakes protect every service: a request quota enforced by API Gateway, and a daily spend cap enforced by the API. You need both, because one LLM request can cost anywhere from a hundred-thousandth of a cent to several cents.
Request quota
Limits apply per key: two apps on the same plan never share a quota, so one app polling hard can't exhaust another's.
| Plan | Rate | Burst | Requests per day |
|---|---|---|---|
admin role — keys of the Vovix admin | 50 req/s | 100 | 50,000 |
user role — keys issued to others | 2 req/s | 5 | 5,000 |
| An app's own plan, set by the admin | as set | — | as set |
| Whole API (all keys combined) | 50 req/s | 100 | — |
The daily quota resets at midnight UTC. When you exceed a limit, API Gateway answers before the request reaches the API:
- over the rate:
429with{"message":"Too Many Requests"}— safe to retry after a short backoff - over the daily quota:
429with{"message":"Limit Exceeded"}— every route fails until midnight UTC
Daily budget
Each service has a daily spend cap, $2 per day by default. After every call the API adds that call's cost to your service's running total for the day; once the total reaches the cap, model requests fail with 429 BUDGET_EXCEEDED until midnight UTC. GET /v1/usage shows where you are.
The budget is a circuit breaker, not an invoice. Cost is added after each call, so many requests in flight at the moment you cross the cap all complete before later ones are blocked. The authoritative bill is AWS Cost Explorer.
Cost is computed as tokens multiplied by the catalog price. A request to a model outside the catalog has no known price: its tokens are still counted, but it adds nothing to the budget.
Per-user credits
The daily budget protects Vovix from your service. To limit your users, send userId and let the API meter each user's credits: a monthly allowance per plan plus purchased credits, held before each call so a user who runs out is refused with 402 before any model runs. Unlike the budget, credits are exact under concurrency.
Hard limits
| Limit | Value | Why |
|---|---|---|
maxTokens per request | 32,000 | Required on every request, and the only hard cap on what a single call can cost. |
| Request duration | 29 seconds | API Gateway's integration timeout, which cannot be raised. |
| Model call timeout | 26 seconds | The API gives up before the gateway does, so you get a 504 TIMEOUT JSON body you can read. |
stop sequences | 4 | |
| Agent tools | 64 per request | |
| Agent turns | 16 per request |
Streaming
Streaming is not available yet. API Gateway's REST APIs buffer the whole response, so server-sent events would arrive all at once. Streaming needs a Lambda function URL, which doesn't support API keys or usage plans. POST /v1/chat/stream returns 501 NOT_IMPLEMENTED to make that explicit.
For long outputs, keep maxTokens modest or split the work so each call finishes well within 26 seconds.