RPM vs TPM
Two limits govern throughput. For agentic coding only one usually matters, and it’s not the one people watch.
Requests per minute
How many API calls per minute, regardless of size. Binds on many small requests: classification, short completions, high-QPS endpoints.
Tokens per minute
How many tokens (input + output) per minute. Binds when requests are large, which is the agentic-coding case, where one request carries tens of thousands of context tokens.
TPM is almost always the binding constraint
RPM and TPM are independent. For large-context agentic work you exhaust the token budget long before the request budget: a single 40K-token turn hits a 100K TPM ceiling in under three requests while barely touching RPM.
So when you plan capacity or debug a 429, look at TPM first. Raising RPM does nothing if TPM is the wall.
Some serverless models couple RPM to TPM
For some third-party serverless models on Microsoft Foundry, RPM isn’t fully independent. It may scale with the underlying TPM assignment instead of being granted separately. The ratio isn’t always published, so test after any quota increase instead of trusting the documented number.
Fireworks-hosted models often ship far higher default TPM
On Microsoft Foundry, Fireworks-hosted models often get much higher default TPM (e.g. 500K) than natively-hosted ones (e.g. 100K), a real advantage on top of their better cache economics.
Rules of thumb
- ▪ Large-context agentic work → size for TPM.
- ▪ High-volume small requests → watch RPM.
- ▪ After a quota bump, measure the real ceiling. Serverless RPM/TPM coupling is often unpublished.
- ▪ Prefer a hosting arrangement with a higher default TPM when your workload is context-heavy.