API gateway / rate limiter
Single front door for APIs: authentication, rate limiting, routing and request shaping.
Problem it solves: Every service would otherwise reimplement auth, quotas and throttling.
Where it sits: After the load balancer, before application services.
When to use
- Public or partner APIs needing quotas.
- Protecting downstream capacity from bursts.
When not to use
- Deep business logic; keep the gateway thin.
Failure modes
- Limits set above downstream capacity do not protect it.
- Centralised gateway outage blocks every API.
Metrics to watch
- Admitted vs rejected (429) req/s
- Auth failures
- Gateway latency (ms)
Alternatives
- Token bucket in each service
- Service mesh policies
Configuration in Labs (illustrative defaults)
- Instances (count) = 2 — Identical copies behind this logical component. Capacity scales linearly in this model.
- Capacity per instance (req/s) = 20000 — Maximum requests one instance completes per second before it saturates.
- Base service time (ms) = 3 — Processing time per request at zero load. Queueing adds to this as utilisation rises.
- Rate limit (req/s) = 0 — Requests per second admitted; the excess is rejected with 429. 0 disables the limit.