Rate-limiting your own AI API before your users (or a bug) do it for you

One runaway frontend loop can burn a month's API budget in an hour. A simple, practical rate-limiting setup for a student AI project's backend.

Written by

Pranjul Rathour

GenAI Engineer at SCULT INDIA, based in Kanpur, India. Builds production RAG systems, fine-tunes open LLMs and ships agentic AI products; has mentored 200+ college developers. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks.

pranjulrathour41@gmail.com LinkedIn GitHub X Instagram Bluesky Dev.to

At the VSICS podium, Kanpur

A student's demo went viral on a class WhatsApp group, a retry loop in the frontend fired on every failed request, and the month's API budget was gone in under an hour. Rate limiting your own endpoint would have prevented it.

The minimum viable setup

Per-IP or per-user request limits on your own API, before the request ever reaches the LLM provider.
A hard daily spend cap enforced in code, not just watched on a dashboard after the fact.
Exponential backoff on your OWN retries, so a transient failure doesn't turn into a request storm.

Where to put it

At the API gateway or middleware layer, before your handler runs — never rely on the frontend to behave, since a bug, a bot, or a malicious user can bypass frontend logic entirely.

See LLM cost control and token budgets for the budget side of this same problem.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.