Docker Rate Limit Issue — Resolved in ECR

Recently docker announced its limit on pulling & pushing images to the docker hub. The Docker plays a vital role in managing the custom images which can be pulled locally by the users for running their application or they can push their custom application images to the docker.

From the deployment perspective, Dev/QA & Testing will have multiple builds on their applications every single day, which will execute multiple Push & Pull commands on the docker hub. When the application is a free user of Docker then rate limit error raises when multiple pull commands are executed in a single day.

When the application pulls the base images like python, ubuntu, etc from Docker the below error araises.

You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits

This issue made the developers to wait till the refresh time for their next step of action. It generally takes 6 hours again to get the window refresh and to pull the docker images.

In this blog I would like to share the ideology which I have implemented to over the come issue.

I have a Python application which has the source in BitBucket and uses private Docker ECR (Elastic Container Registry) in AWS and gets executed in ECS (Elastic Container Service) in AWS. The whole deployment process of updating the ECR when ever their is push to Bitbucket report is automated by CodePipeline and CodeBuild to build the docker and push to ECR.

Initial Docker File :

The below shown docker file will pull the Python 3.7 base image from Docker and install the packages in the requirements.txt file and run the application.

Inital Docker File

This Docker file gives rate issue when executed multiple times. Now lets see how to update this file and overcome the rate issue.

Created Custom Python Docker Image

As a first step, we need to create another ECR repo containing the Python Image.

Steps Involved in creating the custom Image

On the local server perform the following.

1. mkdir python_base
2. cd python_base
3. Create a Docker File as shown below

4. Create a new repository in ECR and name it as pythonbase

Creating a repository

5. Once the repo setup done, execute the below commands and updated the repo.

aws ecr get-login-password --region us-east-1
docker build -t pythonbase .
docker tag pythonbase:latest $account_id.dkr.ecr.us-east-1.amazonaws.com/pythonbase:latest
docker push $account_id.dkr.ecr.us-east-1.amazonaws.com/pythonbase:latest

After updating the new ECR repo with Python Image update the Docker file in the application as shown below.

Here instead of pulling the base image from docker we are referring to our custom ECR repo.

Update Docker file

Now build the dockerfile with the below command .

Account_ID = Enter your aws account ID where the new Python ECR repo is avaiable.

docker build --build-arg Account_ID=XXXXXX .

Now lets see how to automate this in CodeBuild

1. Login to your AWS console and go to CodeBuild
2. Select your CodeBuild project and go to Enviroment section
3. Set the environment variable as AWS_ACCOUNT_ID and your account ID as the value

4. Read the environment variables in the BuildSpec file as shown below as shown below

Buildspec.yml file for CodeBuild

Note : Instead of the AWS account Id the entire ECR repo URI has also be passed like shown below.

Application Docker file

Buildspec.yml

Hope the workaround will work for anyone facing the same issue!!!