Securing and Accelerating Content Delivery: Using Google Cloud CDN to Serve Private AWS S3 Bucket Data

For one of our customers, we came across a requirement to access the AWS private S3 bucket through Cloud CDN, for seamless connectivity to AWS S3 data via Cloud CDN, enhancing retrieval efficiency and security for our client. we integrated AWS S3 with Cloud CDN to minimize latency and fortified data against unauthorized access, optimizing user experience and compliance. This synergy empowers our client with unparalleled agility and resilience in their data accessibility endeavors.

Let’s configure the necessary components:

Step 1: Creating AWS S3 Bucket and adding dummy data

1. Log in to your AWS account and go to the AWS S3 dashboard to create a bucket. Click on Create Bucket and give it a unique name.

Make sure to block all public access.

Click on create bucket.

As you can see bucket is ready.

2. Now upload some dummy data in the bucket.

Step 2: Creating an AWS User

1. Create a user that will be used by GCP to access your AWS S3 bucket, head to IAM > Users > Create user and give the user a name.

Attach AmazonS3ReadOnlyAccess policy to the user.

Create a access key and secret key for the user and save it for later use.

Step 3: Creating Network Endpoint Group

Navigate to Compute Engine > Network Endpoint Group console and click on create Network Endpoint Group

• Network endpoint group type → choose Internet NEG
• Fully qualified domain name → <bucket name>.s3.<bucket region>.amazonaws.com

Now click on Create to create Network Endpoint Group

Step 4: Creating Application Load Balancer

1. For creating Application Load Balancer, navigate the load balancer in the search field.

Choose type as Application Load Balancer.

2. Now click on create

3. Configure the LoadBalancer frontend and set Protocol to HTTP. Leave the other settings as default and save it.

4. Click on create a backend service.

5. Give a name to the backend service and configure the below fields

• Name → give a name
• Backend Type → set the type to Internet network endpoint group
• Protocol → set the protocol to HTTPS
• Backends → choose the Network Endpoint Group created in previous steps

6. Configure the Cloud CDN as shown in below images

7. In the custom request header option, provide the header name and value, then click on create.

• Header name → Host
• Header value → <bucket name>.s3.<bucket region>.amazonaws.com

8. After selecting the ADD HOST AND RULE option, add the following rule to configure the routing rules.

• Host → *
• Path → /*
• Backend → choose the backend created in the previous steps

9. Now click on create

Step 5: Updating Backend Service to access

1. Execute the below command to generate a yaml file which will be used to update backend service

gcloud beta compute backend-services describe <backend service name> --global > cdn-private-origin.yaml

Remove the below lines from the yaml that you get from the output of running the command in the above step

• fingerprint: <value>
• Id: <value>

2. Add the following code snippet to the file that was created in the previous step, then run the following command.

securitySettings:
 awsV4Authentication:
  accessKeyId: <access key>
  accessKey: <secret key>
  originRegion: <bucket region>

gcloud beta compute backend-services import <backend service>--source cdn-private-origin.yaml --global

Now use LoadBalancer IP to access the data from AWS S3 Bucket.

Ping LB_IP/object_name.

Hope you found this blog helpful. Thanks for reading :)