New serverless pattern - CloudFront to APIGW with alternative origin for large uploads

cloudfront-apigw-large-uploads/README.md

@@ -0,0 +1,130 @@
+# Amazon Cloudfront to APIGW routing to alternative origin for large payloads
+API Gateway has a [payload limit of 10mb](https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#http-api-quotas) which cannot be increaed. This pattern shows how to use Lambda@Edge to route client POST request to an alternative origin based on payload size of the POST request or URL.
+
+Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/cloudfront-apigw-large-uploads
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [AWS Cloud Development Kit](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html) (AWS CDK) installed
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+1. Change directory to the pattern directory:
+    ```
+    cd cloudfront-apigw-large-uploads
+    ```
+1. From the command line, use AWS CDK to deploy the AWS resources for the pattern as specified in the `large_uploads_test/test_stack.py` file.
+    ```
+    python3 -m pip install -r requirements.txt
+    cdk synth
+    cdk deploy
+    ```
+
+## How it works
+A Cloudfront distribution is created with 2 origins:
+* Default Origin - API Gateway with a Lambda Integration that returns stub responses based on HTTP request method.  
+* Custom Origin - A mock HTTP endpoint which sends a response with information from the request. We are using echo.free.beeceptor.com but you can substitue this with other endpoint URLs.
+
+When a HTTP POST request is handled by CloudFront, a Lambda@Edge function is invoked as an Origin Request. This Lambda@Edge function uses the method and `ceontent-length` HTTP headers from the client request to determine whether the request should go to a custom origin. 
+![diagram](diagram.png)
+
+
+## Testing
+
+1. Set region to us-east-1 for your CDK environment, this is because Lambda@edge functions need to exist us-east-1 region.
+1. cdk bootstrap
+1. cdk deploy
+1. Once deployed, find the CloudFront distribution URL and use it in the following `curl` commands
+
+#### Test 1: 
+A GET request that CloudFront will route to API gateway
+
+`curl https://<CloudFront distribution URL> -i`
+
+Example output:
+```
+    HTTP/2 200
+    content-type: application/json
+    ...more headers...
+
+    {"message": "GET request processed by API Gateway!"}
+```
+
+#### Test 2: 
+A POST request that CloudFront will route to API gateway as the payload size is smaller than `MAX_FILE_SIZE` defined in lambda.mjs.
+
+`curl https://<CloudFront distribution URL> -i -X POST -F '[email protected]' -H 'content-type: appli
+cation/json'`
+
+Example output:
+```
+    HTTP/2 201
+    content-type: application/json
+    ...more headers...
+
+    {"message": "POST request processed by API Gateway!"}
+```
+
+#### Test 3: 
+A POST request that CloudFront will route to a custom origin as the payload size is larger than `MAX_FILE_SIZE` defined in `lambda.mjs`.
+
+`curl https://<CloudFront distribution URL> -i -X POST -F '[email protected]`
+
+Example output:
+```
+    HTTP/2 200
+    content-type: application/json
+    ...more headers...
+
+    {
+        "method": "POST",
+        "protocol": "https",
+        "host": "echo.free.beeceptor.com",
+    ...more response body
+```
+
+
+#### Test 4: 
+A POST request to a specific URL that CloudFront will route to a custom origin based on the Behaviour defined in `app.py`.
+
+`curl https://<CloudFront distribution URL>/upload/ -i -X POST -F '[email protected]`
+
+Example output:
+```
+    HTTP/2 200
+    content-type: application/json
+    ...more headers...
+
+    {
+        "method": "POST",
+        "protocol": "https",
+        "host": "echo.free.beeceptor.com",
+    ...more response body
+```
+
+
+## Cleanup
+
+1. Delete the stack
+    ```
+    cdk destroy
+    ```
+
+
+## Notes
+There are restrictions to what the Lambda function can modify as part of the Origin Request. For example, CloudFront will not allow you to change the protocol from HTTPS to HTTP. Doing so will result in an origin configuration error from CloudFront.
+
+
+----
+Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

cloudfront-apigw-large-uploads/app.py

@@ -0,0 +1,133 @@
+import aws_cdk as cdk
+from aws_cdk import aws_cloudfront as cloudfront
+from aws_cdk import aws_cloudfront_origins as origins
+from aws_cdk import aws_apigateway as apigw
+from aws_cdk import aws_lambda as lambda_
+from aws_cdk import aws_iam as iam
+
+class CloudFrontApigwLargeUploadsStack(cdk.Stack):
+    def __init__(self, scope, construct_id, **kwargs):
+        super().__init__(scope, construct_id, **kwargs)
+
+        # Create an API Gateway HTTP endpoint that will return a mock response 
+        nonUploadApi = apigw.RestApi(
+            self,
+            "nonUploadApi",
+            endpoint_types=[apigw.EndpointType.REGIONAL],
+            deploy_options=apigw.StageOptions(
+                stage_name="mock",
+                throttling_rate_limit=100,
+                throttling_burst_limit=1000,
+            ),
+            cloud_watch_role=True,
+            deploy=True,
+        )
+
+        # Create a mock integration for the nonUploadApi / path that returns status 200 for GET requests
+        nonUploadApi.root.add_method(
+            "GET",
+            apigw.MockIntegration(
+                integration_responses=[
+                    apigw.IntegrationResponse(
+                        status_code="200",
+                        response_templates={
+                            "application/json": '{"message": "GET request processed by API Gateway!"}'
+                        },
+                    )
+                ],
+                passthrough_behavior=apigw.PassthroughBehavior.NEVER,
+                request_templates={"application/json": '{"statusCode": 200}'},
+            ),
+            method_responses=[apigw.MethodResponse(status_code="200")],
+        )
+
+        # Create a mock integration for the nonUploadApi / path that returns status 200 for POST requests
+        nonUploadApi.root.add_method(
+            "POST",
+            apigw.MockIntegration(
+                integration_responses=[
+                    apigw.IntegrationResponse(
+                        status_code="201",
+                        response_templates={
+                            "application/json": '{"message": "POST request processed by API Gateway!"}'
+                        },
+                    )
+                ],
+                passthrough_behavior=apigw.PassthroughBehavior.NEVER,
+                request_templates={"application/json": '{"statusCode": 200}'},
+            ),
+            method_responses=[apigw.MethodResponse(status_code="201")],
+        )
+
+        # Create Lamdba@edge function
+        edge_function = lambda_.Function(
+            self,
+            "OriginRequestFunction",
+            runtime=lambda_.Runtime.NODEJS_LATEST,
+            handler="lambda.handler",
+            code=lambda_.Code.from_asset("lambda.zip")
+        )
+
+        # Add Lambda@Edge permission
+        edge_function.add_permission(
+            "EdgeFunctionPermission",
+            principal=iam.ServicePrincipal("edgelambda.amazonaws.com"),
+            action="lambda:InvokeFunction"
+        )
+
+        # Add execution role permissions for Lambda@Edge
+        edge_function.role.add_to_policy(
+            iam.PolicyStatement(
+                effect=iam.Effect.ALLOW,
+                actions=[
+                    "logs:CreateLogGroup",
+                    "logs:CreateLogStream",
+                    "logs:PutLogEvents"
+                ],
+                resources=["arn:aws:logs:*:*:*"]
+            )
+        )
+
+        # Create a CloudFront distribution with custom origin
+        distribution = cloudfront.Distribution(
+            self, 
+            "testLargeUploadDistribution",
+            price_class=cloudfront.PriceClass.PRICE_CLASS_100, # Change this if you need more regions enabled in CloudFront
+            # Default behavior now points to API Gateway
+            default_behavior=cloudfront.BehaviorOptions(
+                origin=origins.RestApiOrigin(
+                    nonUploadApi,
+                    origin_path="/mock"
+                ),
+                edge_lambdas=[
+                    cloudfront.EdgeLambda(
+                        function_version=edge_function.current_version,
+                        event_type=cloudfront.LambdaEdgeEventType.ORIGIN_REQUEST
+                    )
+                ],
+                viewer_protocol_policy=cloudfront.ViewerProtocolPolicy.HTTPS_ONLY,
+                origin_request_policy=cloudfront.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER,
+                cache_policy=cloudfront.CachePolicy.CACHING_DISABLED,
+                allowed_methods=cloudfront.AllowedMethods.ALLOW_ALL,
+            ),
+            # Additional behavior for /upload/* paths to hit a test endpoint httpbin which will provide a generic response.
+            additional_behaviors={
+                "/upload/*": cloudfront.BehaviorOptions(
+                    origin=origins.HttpOrigin(
+                        "echo.free.beeceptor.com", # This is the URL of a mock file server.
+                        protocol_policy=cloudfront.OriginProtocolPolicy.HTTPS_ONLY,
+                        https_port=443,
+                        origin_path="/" # The origin path is set to / for the example endpoint. It will need to be modified to the correct URI for real world use-cases.
+                    ),
+                    viewer_protocol_policy=cloudfront.ViewerProtocolPolicy.HTTPS_ONLY,
+                    cache_policy=cloudfront.CachePolicy.CACHING_DISABLED,
+                    allowed_methods=cloudfront.AllowedMethods.ALLOW_ALL,  # Allow all HTTP methods for uploads
+                    origin_request_policy=cloudfront.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER  # Forward all headers and query strings
+                )
+            },
+            comment="Distribution to test alternative origin for large uploads"
+        )
+
+app = cdk.App()
+CloudFrontApigwLargeUploadsStack(app, "CloudFrontApiGatewayLargeUploads")
+app.synth()

cloudfront-apigw-large-uploads/cdk.json

@@ -0,0 +1,26 @@
+{
+    "app": "python3 app.py",
+    "watch": {
+        "include": [
+          "**"
+        ],
+        "exclude": [
+          "README.md",
+          "cdk*.json",
+          "requirements*.txt",
+          "**/__init__.py",
+          "python/__pycache__",
+          "tests"
+        ]
+    },
+    "context": {
+        "@aws-cdk/core:target-partitions": [
+          "aws",
+          "aws-cn"
+        ],
+        "@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
+        "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+        "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+        "@aws-cdk/core:checkSecretUsage": true
+    }
+}

cloudfront-apigw-large-uploads/cloudfront-apigw-large-uploads.json

@@ -0,0 +1,44 @@
+{
+    "title": "Amazon Cloudfront to APIGW routing to alternative origin for large payloads",
+    "description": "Amazon Cloudfront to APIGW routing to alternative origin for large payloads using Lambda@Edge",
+    "language": "Python",
+    "level": "200",
+    "framework": "CDK",
+    "introBox": {
+        "headline": "How it works",
+        "text": [
+            "This pattern shows how to use Lambda@Edge to route client POST request to an alternative origin when a POST request exceeds the 10mb payload limit for API Gateway."
+        ]
+    },
+    "gitHub": {
+        "template": {
+            "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cloudfront-apigw-large-uploads",
+            "templateURL": "serverless-patterns/cloudfront-apigw-large-uploads",
+            "projectFolder": "cloudfront-apigw-large-uploads",
+            "templateFile": "app.py"
+        }
+    },
+    "deploy": {
+        "text": [
+            "cdk deploy"
+        ]
+    },
+    "testing": {
+        "text": [
+            "See the GitHub repo for detailed testing instructions."
+        ]
+    },
+    "cleanup": {
+        "text": [
+            "Delete the stack: <code>cdk destroy</code>."
+        ]
+    },
+    "authors": [
+        {
+            "name": "Shaun Guo",
+            "image": "https://media.licdn.com/dms/image/C5103AQG3KMyMdEIKpA/profile-displayphoto-shrink_800_800/0/1517283953925?e=1692835200&v=beta&t=AxJ9ST_8K_bw8nqTPDaJB2F5dnQspES9FuJ64DBScC8",
+            "bio": "Shaun is a Senior Technical Account Manager at Amazon Web Services based in Australia",
+            "linkedin": "shaun-guo"
+        }
+    ]
+}

cloudfront-apigw-large-uploads/lambda.mjs

@@ -0,0 +1,27 @@
+// This is a copy of the file found in lambda.zip.
+
+// Using a small value of 300 bytes to demonstrate the behaviour of Lambda@Edge function.
+// In real world applications, you would set this higher.
+const MAX_FILE_SIZE = 300;
+
+// Using a random public facing endpoint that will respond to requests.
+const LARGE_UPLOAD_ORIGIN = "echo.free.beeceptor.com"; 
+
+export const handler = async (event) => {
+    const request = event.Records[0].cf.request;
+    const headers = request.headers;
+    const origin = event.Records[0].cf.request.origin;
+
+    if (request.method === 'POST') {
+        if (headers['content-length'] && parseInt(headers['content-length'][0].value) < MAX_FILE_SIZE ) {
+            console.log(`Request size less than: ` + MAX_FILE_SIZE); 
+        } else {
+            console.log(`Request has no content-length or request size is greater than: ` + MAX_FILE_SIZE);
+            origin.custom.domainName = LARGE_UPLOAD_ORIGIN;
+            request.headers.host[0].value = LARGE_UPLOAD_ORIGIN;
+            request.uri = '/';
+        }
+    }
+
+    return request;
+};

cloudfront-apigw-large-uploads/requirements.txt

@@ -0,0 +1 @@
+aws-cdk-lib==2.181.1

cloudfront-apigw-large-uploads/smallFile.example

@@ -0,0 +1 @@
+1