You probably ended up here because you have a script that makes many API calls to Oracle Cloud Infrastructure and some of your requests fail due to an HTTP 429 TooManyRequests error.
Why do you get those errors?
First of all, this is normal behavior. Oracle Cloud Infrastructure applies throttling to many API requests to prevent accidental or abusive use of resources. If you make too many requests too quickly, you might see some succeed and others fail.
API limiting, which is also known as rate limiting, is an essential component of Internet security, as DoS attacks can tank a server with unlimited API requests.
What to do in this case?
You should implement an exponential back-off strategy which will pause some requests when you hit the rate limit. There are many ways of achieving this, but let’s look at how to do this with the OCI Python SDK.
Retries
As you may have seen already, the OCI Python SDK does not use a retry strategy by default, but you can pass a retry_strategy keyword argument to every API call made via the SDK operations.
There are 3 possibilities when it comes to the retry strategy that can be passed to your operations:
- The default retry strategy implemented in the SDK as DEFAULT_RETRY_STRATEGY
- The NoneRetryStrategy which will not perform any retries
- A custom retry strategy that can be built using the RetryStrategyBuilder
Break the OCI Rate Limiter
Before we implement the retry strategies, let’s just look at a simple example that will break the OCI API rate limiter so we can have something to build upon:
import oci from threading import Thread class OCICalls(object): def __init__(self): self.regions = [] # generate signer for authentication self.generate_signer_from_instance_principals() # call APIs self.get_regions() print(f"My regions are: {self.regions}") def generate_signer_from_instance_principals(self): try: # get signer from instance principals token self.signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner() except Exception: print("There was an error while trying to get the Signer") raise SystemExit # generate config info from signer with region and tenancy_id self.config = {'region': self.signer.region, 'tenancy': self.signer.tenancy_id} def get_regions(self): # initialize the IdentityClient identity_client = oci.identity.IdentityClient(config = {}, signer=self.signer ) jobs = [] # get list of OCI subscribed regions 200 times so we break OCI's rate limiter for i in range(200): thread = Thread(target = self.__get_regions, args=(identity_client,)) jobs.append(thread) # start threads for job in jobs: job.start() # join threads so we don't quit until all threads have finished for job in jobs: job.join() def __get_regions(self, identity_client): self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id).data # Initiate process OCICalls()
So, to explain the example – I am getting the list of subscribed regions 200 times using threads – the script runs from a Compute instance in OCI and I’m using Instance Principals for Authentication.
If you need help configuring the authentication check those two articles:
- For authenticating via config file (run the script from a compute in OCI or from anywhere else), check this article
- For authenticating via instance principals (run the script from a compute in OCI), check this article
This example will throw some HTTP 429 TooManyRequest errors that look something like this:
Exception in thread Thread-200: Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "no_retry.py", line 50, in __get_regions self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id).data File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/identity/identity_client.py", line 6059, in list_region_subscriptions response_type="list[RegionSubscription]") File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 248, in call_api return self.request(request) File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 363, in request self.raise_service_error(request, response) File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 533, in raise_service_error original_request=request) oci.exceptions.ServiceError: {'opc-request-id': 'XXXXXXXXXXXX', 'code': 'TooManyRequests', 'message': 'Too many requests for the user', 'status': 429}
Implementing the Retry Strategies
Let’s look at how we can implement the retry strategies in the example used earlier.
The Default Retry Strategy
Let’s look at the characteristics of the Default Retry Strategy:
- 5 total attempts
- Total allowed elapsed time for all requests of 300 seconds (5 minutes)
- Retries on the following exception types:
- Timeouts and connection errors
- HTTP 429 (throttling)
- HTTP 5xx (server errors)
- Exponential backoff with jitter, using:
- The base time to use in retry calculations will be 1 second
- An exponent of 2. When calculating the next retry time we will raise this to the power of the number of attempts
- Maximum wait time between calls of 30 seconds
You can see the full list of characteristics here.
To implement this retry strategy, you must add a retry_strategy keyword argument to your API calls. Let’s do that to the API call used earlier to get the subscribed regions of my tenancy:
def __get_regions(self, identity_client): self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id, retry_strategy=oci.retry.DEFAULT_RETRY_STRATEGY).data
It is as simple as that!
Now you should be able to run the same script and you’ll not get any HTTP 429 TooManyRequests errors anymore.
This default retry strategy should work in most cases, but if you want to configure it, you can build your own retry strategy.
The Custom Retry Strategy
If we look in the source code of the retry strategy, we can find all the parameters that we can give to our custom retry strategy, all of them are optional and have default values:
### Create Custom Retry Strategy ### #################################### self.custom_retry_strategy = oci.retry.RetryStrategyBuilder( # Whether to enable a check that we don't exceed a certain number of attempts max_attempts_check=True, # check that will retry on connection errors, timeouts and service errors service_error_check=True, # a check that we don't exceed a certain amount of time retrying total_elapsed_time_check=True, # maximum number of attempts max_attempts=10, # don't exceed a total of 900 seconds for all calls total_elapsed_time_seconds=900, # if we are checking o service errors, we can configure what HTTP statuses to retry on # and optionally whether the textual code (e.g. TooManyRequests) matches a given value service_error_retry_config={ 400: ['QuotaExceeded', 'LimitExceeded'], 429: [] }, # whether to retry on HTTP 5xx errors service_error_retry_on_any_5xx=True, # Used for exponention backoff with jitter retry_base_sleep_time_seconds=2, # Wait 60 seconds between attempts retry_max_wait_between_calls_seconds=60, # the type of backoff # Accepted values are: BACKOFF_FULL_JITTER_VALUE, BACKOFF_EQUAL_JITTER_VALUE, BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE backoff_type=oci.retry.BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE ).get_retry_strategy() ####################################
Then, we just need to pass this retry strategy over to the API calls:
def __get_regions(self, identity_client): self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id, retry_strategy=self.custom_retry_strategy).data
Conclusions
If you end up making many API calls against OCI, you may want to implement a retry strategy that can manage the HTTP 429 TooManyRequests errors. or timeouts or other types of errors.
We saw that there are two ways of doing this, either with a default retry strategy or by creating a custom retry strategy. This will help retry the requests that are “rejected” by the OCI API rate limiter, timeouts, server errors, etc, and make some pauses in between so you execute all the request successfully.
You can find on my GitHub all 3 complete examples showcased in this article:
- Script with no retry strategy that will generate the errors
- Modified script with default retry strategy
- Modified script with custom retry strategy
Resources
OCI Python SDK – Retries Documentation
OCI Python SDK – Retry Implementation