You probably ended up here because you have a script that makes many API calls to Oracle Cloud Infrastructure and some of your requests fail due to an HTTP 429 TooManyRequests error.

Why do you get those errors?

First of all, this is normal behavior. Oracle Cloud Infrastructure applies throttling to many API requests to prevent accidental or abusive use of resources. If you make too many requests too quickly, you might see some succeed and others fail.

API limiting, which is also known as rate limiting, is an essential component of Internet security, as DoS attacks can tank a server with unlimited API requests.

What to do in this case?

You should implement an exponential back-off strategy which will pause some requests when you hit the rate limit. There are many ways of achieving this, but let’s look at how to do this with the OCI Python SDK.

Retries

As you may have seen already, the OCI Python SDK does not use a retry strategy by default, but you can pass a retry_strategy keyword argument to every API call made via the SDK operations.

There are 3 possibilities when it comes to the retry strategy that can be passed to your operations:

1. The default retry strategy implemented in the SDK as DEFAULT_RETRY_STRATEGY
2. The NoneRetryStrategy which will not perform any retries
3. A custom retry strategy that can be built using the RetryStrategyBuilder

Break the OCI Rate Limiter

Before we implement the retry strategies, let’s just look at a simple example that will break the OCI API rate limiter so we can have something to build upon:

import oci
from threading import Thread

class OCICalls(object):
    def __init__(self):
        self.regions = []
        
        # generate signer for authentication
        self.generate_signer_from_instance_principals()

        # call APIs
        self.get_regions()

        print(f"My regions are: {self.regions}")
        
    
    def generate_signer_from_instance_principals(self):
        try:
            # get signer from instance principals token
            self.signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
        except Exception:
            print("There was an error while trying to get the Signer")
            raise SystemExit

        # generate config info from signer with region and tenancy_id
        self.config = {'region': self.signer.region, 'tenancy': self.signer.tenancy_id}
        
    
    def get_regions(self):
        # initialize the IdentityClient
        identity_client = oci.identity.IdentityClient(config = {}, signer=self.signer )
        
        jobs = []
        # get list of OCI subscribed regions 200 times so we break OCI's rate limiter
        for i in range(200):
            thread = Thread(target = self.__get_regions, args=(identity_client,))
            jobs.append(thread)
                            
        # start threads   
        for job in jobs:
            job.start()
            
        # join threads so we don't quit until all threads have finished
        for job in jobs:
            job.join()
        

    def __get_regions(self, identity_client):
        self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id).data


# Initiate process
OCICalls()

So, to explain the example – I am getting the list of subscribed regions 200 times using threads – the script runs from a Compute instance in OCI and I’m using Instance Principals for Authentication.

If you need help configuring the authentication check those two articles: 

• For authenticating via config file (run the script from a compute in OCI or from anywhere else), check this article 
• For authenticating via instance principals (run the script from a compute in OCI), check this article

This example will throw some HTTP 429 TooManyRequest errors that look something like this:

Exception in thread Thread-200:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "no_retry.py", line 50, in __get_regions
    self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id).data
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/identity/identity_client.py", line 6059, in list_region_subscriptions
    response_type="list[RegionSubscription]")
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 248, in call_api
    return self.request(request)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 363, in request
    self.raise_service_error(request, response)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/oci/base_client.py", line 533, in raise_service_error
    original_request=request)
oci.exceptions.ServiceError: {'opc-request-id': 'XXXXXXXXXXXX', 'code': 'TooManyRequests', 'message': 'Too many requests for the user', 'status': 429}

Implementing the Retry Strategies

Let’s look at how we can implement the retry strategies in the example used earlier.

The Default Retry Strategy

Let’s look at the characteristics of the Default Retry Strategy:

• 5 total attempts
• Total allowed elapsed time for all requests of 300 seconds (5 minutes)
• Retries on the following exception types:
  • Timeouts and connection errors
  • HTTP 429 (throttling)
  • HTTP 5xx (server errors)
• Exponential backoff with jitter, using:
  • The base time to use in retry calculations will be 1 second
  • An exponent of 2. When calculating the next retry time we will raise this to the power of the number of attempts
  • Maximum wait time between calls of 30 seconds

You can see the full list of characteristics here.

To implement this retry strategy, you must add a retry_strategy keyword argument to your API calls. Let’s do that to the API call used earlier to get the subscribed regions of my tenancy:

def __get_regions(self, identity_client):
    self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id, retry_strategy=oci.retry.DEFAULT_RETRY_STRATEGY).data

It is as simple as that!

Now you should be able to run the same script and you’ll not get any HTTP 429 TooManyRequests errors anymore.

This default retry strategy should work in most cases, but if you want to configure it, you can build your own retry strategy.

The Custom Retry Strategy

If we look in the source code of the retry strategy, we can find all the parameters that we can give to our custom retry strategy, all of them are optional and have default values:

### Create Custom Retry Strategy ###
####################################
self.custom_retry_strategy = oci.retry.RetryStrategyBuilder(
# Whether to enable a check that we don't exceed a certain number of attempts
max_attempts_check=True,
# check that will retry on connection errors, timeouts and service errors 
service_error_check=True,
# a check that we don't exceed a certain amount of time retrying
total_elapsed_time_check=True,
# maximum number of attempts
max_attempts=10,
# don't exceed a total of 900 seconds for all calls
total_elapsed_time_seconds=900,
# if we are checking o service errors, we can configure what HTTP statuses to retry on
# and optionally whether the textual code (e.g. TooManyRequests) matches a given value
service_error_retry_config={
    400: ['QuotaExceeded', 'LimitExceeded'],
    429: []
},
# whether to retry on HTTP 5xx errors
service_error_retry_on_any_5xx=True,
# Used for exponention backoff with jitter
retry_base_sleep_time_seconds=2,
# Wait 60 seconds between attempts
retry_max_wait_between_calls_seconds=60,
# the type of backoff
# Accepted values are: BACKOFF_FULL_JITTER_VALUE, BACKOFF_EQUAL_JITTER_VALUE, BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE
backoff_type=oci.retry.BACKOFF_FULL_JITTER_EQUAL_ON_THROTTLE_VALUE
).get_retry_strategy()
####################################

Then, we just need to pass this retry strategy over to the API calls:

def __get_regions(self, identity_client):
    self.regions = identity_client.list_region_subscriptions(self.signer.tenancy_id, retry_strategy=self.custom_retry_strategy).data

Conclusions

If you end up making many API calls against OCI, you may want to implement a retry strategy that can manage the HTTP 429 TooManyRequests errors. or timeouts or other types of errors.

We saw that there are two ways of doing this, either with a default retry strategy or by creating a custom retry strategy. This will help retry the requests that are “rejected” by the OCI API rate limiter, timeouts, server errors, etc, and make some pauses in between so you execute all the request successfully.

You can find on my GitHub all 3 complete examples showcased in this article:

• Script with no retry strategy that will generate the errors
• Modified script with default retry strategy
• Modified script with custom retry strategy

Resources

OCI Python SDK – Retries Documentation

OCI Python SDK – Retry Implementation

OCI Python SDK – Retry Examples

Background photo created by fanjianhua – www.freepik.com