A solution to overcome the above mentioned issues is to allow applications to use resources only up to a threshold limit and throttle the requests received after the limit is reached. The throttling limits may be applied to each user of the application or across users depending on the desired SLAs and QoS. Some of the throttling strategies that can be applied are:
- Rejecting requests from a client tagged with a particular key who has already accessed the APIs more than a particular number of times in a particular time duration
- Rejecting requests from a particular IP address after a particular number of requests have been made in a particular time duration
- Deferring operations performed by lower priority clients if the resources are nearing exhaustion
Rate Limiter (Solution)
Azure API Management has advanced throttling techniques that can be used to protect services and apply throttling limits on them.