osFoundry / Platform
Public app rate limiting
Tunable per-IP and per-token rate limits on public apps. Defaults that handle the hug of death.
What it is
Public app rate limiting in osFoundry puts a configurable throttle in front of any app you serve publicly. You set limits two ways that work together: per-IP, which caps how many requests a single client address can make in a window, and per-token, which caps usage tied to a specific access token or API key. Per-IP blunts a flood from one source; per-token keeps a shared or leaked credential from running away with your quota. Because osFoundry is bring-your-own-key, the requests your public app fans out to models ultimately spend against keys you own, so an unbounded endpoint is not just a load problem, it is a billing one. Rate limits are the boundary that keeps a spike from turning into a surprise on your provider bill.
Inside osFoundry, this lives in the Apps & Backend layer alongside the rest of how you package and expose an app. When you make an app public, the limits are already on with defaults chosen to survive the "hug of death" — the moment a link gets shared widely and traffic arrives faster than you planned for. Rather than falling over or draining a key, excess requests are turned away at the edge so legitimate traffic keeps flowing. From there every limit is tunable: raise the ceiling for an app you expect to be busy, tighten it for something experimental, or set per-token limits per credential so different consumers get different budgets.
This matters because a public AI app has a cost surface most static sites do not. One shared URL, a scraper, or a runaway client can generate thousands of model calls in minutes. Per-IP and per-token limits give you a predictable worst case: you decide the maximum spend and load your app can incur before it starts saying no. Combined with osFoundry's local-first, no-reseller-markup model, you keep your keys, your data, and now the exposure of the endpoints those keys sit behind.
Key capabilities
- Set per-IP request limits to cap how much any single client address can hit a public app in a given window
- Set per-token limits so a specific access token or API key gets its own usage budget, independent of source IP
- Ship with defaults tuned to absorb the hug of death — a sudden shared-link traffic spike — without manual setup
- Turn away excess requests at the edge so a flood can't drain the BYOK provider keys your app spends against
- Tune every ceiling up or down per app, and per credential, as an app's real traffic becomes clear
Use cases
- Indie builder: shares a public demo link that unexpectedly goes viral and relies on the default limits to keep it up instead of burning through an API key overnight
- Team lead: issues separate access tokens to partners and sets a per-token budget on each so one integration can't starve the others
- Ops-minded developer: tightens per-IP limits on an experimental endpoint to bound the worst-case cost and load before promoting it to production
Frequently asked questions
What's the difference between per-IP and per-token rate limits?
Per-IP caps requests from a single client address, which stops a flood coming from one source. Per-token caps usage tied to a specific access token or API key regardless of where it's called from, which contains a shared or leaked credential. You can use both at once — per-IP for anonymous public traffic, per-token to give named consumers their own budgets.
What happens when my public app suddenly gets a lot of traffic?
Public apps ship with default limits chosen to handle the hug of death — a link that gets shared far wider than expected. Requests beyond the limit are turned away rather than passed through, so a spike can't drain the provider key your app spends against or knock the app over. You can raise the limits for apps you expect to be busy.
Do I have to configure rate limits before making an app public?
No. Sensible defaults are already in place when you make an app public, so you're protected without any setup. Everything is tunable when you want it — raise or lower the per-IP ceiling, or set per-token limits per credential — but the safe baseline is on from the start.