Enhance apigw-lambda-sns pattern with production-ready improvements

apigw-lambda-sns/Readme.md

@@ -2,13 +2,22 @@
 
 The SAM template deploys a API Gateway REST API with Lambda function integration, an SNS topic and the IAM permissions required to run the application. Whenever the REST API is invoked, the Lambda function publishes a message to the SNS topic. The AWS SAM template deploys the resources and the IAM permissions required to run the application.
 
+## Features
+
+- **API Gateway REST API** with Lambda integration
+- **Lambda function** that publishes messages to SNS
+- **SNS topic** for message publishing
+- **CloudWatch Alarm** monitoring API errors
+- **Amazon CloudWatch Synthetics Canary** for automated API endpoint monitoring
+- **AWS X-Ray tracing** enabled for distributed tracing on LAmbda and APIGW (incurs additional costs)
+- **S3 bucket** for Synthetics artifacts storage
+
 Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/apigw-lambda-sns/.
 
-Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. **Note: AWS X-Ray tracing is enabled which incurs additional charges based on traces recorded and retrieved.** You are responsible for any AWS costs incurred. No warranty is implied in this example.
 
 ## Requirements
 
-
 * [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
 * [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
 * [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
@@ -50,19 +59,27 @@ curl --location --request GET 'https://<api_id>.execute-api.<region>.amazonaws.c
 ```
 In order to receive a notification, please make sure to configure subscription in the SNS topic.
 
+### Additional Features
 
-## Cleanup
+- **CloudWatch Alarm**: Monitor the Synthetics Canary failures. The alarm triggers when the canary fails at least once within a 5-minute period.
+- **Synthetics Canary**: Automatically tests the API endpoint every minute to ensure availability. If you want to alarm on this, you must manually create a CloudWatch Alarm or update the template
+- **X-Ray Tracing**: Distributed tracing is enabled for both API Gateway and Lambda to help with debugging and performance analysis.
 
 
+## Cleanup
+
 1. Delete the stack 
     ```
     aws cloudformation delete-stack —stack-name STACK_NAME
     ```
-2. Confirm the stack has been deleted 
+2. **Manually delete the S3 bucket** - The Synthetics artifacts bucket must be manually emptied and deleted after stack deletion
+3. Confirm the stack has been deleted 
     ```
     aws cloudformation list-stacks —query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
     ```
 
+**Important**: You must manually delete the S3 bucket created for Synthetics artifacts after deleting the CloudFormation stack, as it will contain canary run artifacts.
+
 ----
 Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 

apigw-lambda-sns/api.yaml

@@ -17,15 +17,23 @@ paths:
             application/json:
               schema:
                 $ref: "#/components/schemas/Empty"
+        "400":
+          description: "400 response"
+        "500":
+          description: "500 response"
       x-amazon-apigateway-integration:
         httpMethod: "POST"
         uri: "arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:LambdaFunctionName/invocations"
         responses:
           default:
             statusCode: "200"
+          ".*4\\d{2}.*":
+            statusCode: "400"
+          ".*5\\d{2}.*":
+            statusCode: "500"
         passthroughBehavior: "when_no_match"
         contentHandling: "CONVERT_TO_TEXT"
-        type: "aws"
+        type: "aws_proxy"
 components:
   schemas:
     Empty:

apigw-lambda-sns/src/code.py

@@ -11,23 +11,38 @@ def lambda_handler(event, context):
     logger.setLevel(logging.INFO)
     logger.info("request: " + json.dumps(event))
 
-    topic_arn = os.environ.get('TOPIC_ARN')
-
-    sns_client = boto3.client("sns")
-
     try:
+        topic_arn = os.environ.get('TOPIC_ARN')
+        if not topic_arn:
+            logger.error("Missing TOPIC_ARN environment variable")
+            return {
+                "statusCode": 500,
+                "body": json.dumps({"error": "Server configuration error"})
+            }
+
+        sns_client = boto3.client("sns")
         sent_message = sns_client.publish(
             TargetArn=topic_arn,
             Message=json.dumps({'default': json.dumps(event)})
         )
 
-        if sent_message is not None:
-            logger.info(f"Success - Message ID: {sent_message['MessageId']}")
+        logger.info(f"Success - Message ID: {sent_message['MessageId']}")
         return {
             "statusCode": 200,
-            "body": json.dumps("Success")
+            "body": json.dumps({"status": "Success", "messageId": sent_message['MessageId']})
         }
 
     except ClientError as e:
-        logger.error(e)
-        return None
+        error_code = e.response['Error']['Code']
+        error_message = e.response['Error']['Message']
+        logger.error(f"ClientError: {error_code} - {error_message}")
+        return {
+            "statusCode": 500,
+            "body": json.dumps({"error": "Failed to publish message to SNS"})
+        }
+    except Exception as e:
+        logger.error(f"Unexpected error: {str(e)}")
+        return {
+            "statusCode": 500,
+            "body": json.dumps({"error": "Internal server error"})
+        }

apigw-lambda-sns/template.yaml

@@ -10,11 +10,18 @@ Resources:
     Type: AWS::Serverless::Api
     Properties:
       StageName: s1
-      DefinitionBody: # an OpenApi definition
-        'Fn::Transform':
-          Name: 'AWS::Include'
+      TracingEnabled: true
+      MethodSettings:
+        - ResourcePath: '/*'
+          HttpMethod: '*'
+          MetricsEnabled: true
+          DataTraceEnabled: true
+          LoggingLevel: INFO
+      DefinitionBody:
+        Fn::Transform:
+          Name: AWS::Include
           Parameters:
-            Location: './api.yaml'
+            Location: ./api.yaml
       OpenApiVersion: 3.0.3
       EndpointConfiguration:
           Type: REGIONAL
@@ -28,22 +35,13 @@ Resources:
       Handler: code.lambda_handler
       MemorySize: 128
       Timeout: 3
-      Runtime: python3.8
+      Runtime: python3.13
+      Tracing: Active
       Environment:
         Variables:
           TOPIC_ARN: !Ref MySnsTopic
-      AssumeRolePolicyDocument:
-        Version: 2012-10-17
-        Statement:
-          - Action:
-              - sts:AssumeRole
-            Effect: Allow
-            Principal:
-              Service:
-                - lambda.amazonaws.com
+          API_URL: !Sub 'https://${RestApi}.execute-api.${AWS::Region}.amazonaws.com/s1'
       Policies:
-        - S3FullAccessPolicy:
-            BucketName: severlesspatternlambda
         - SNSPublishMessagePolicy:
             TopicName: !GetAtt MySnsTopic.TopicName
       Events:
@@ -53,6 +51,172 @@ Resources:
             Path: /
             Method: GET
             RestApiId: !Ref RestApi
+
+  # CloudWatch Alarm for API Gateway 5XX Errors
+  ApiGateway5XXErrorAlarm:
+    Type: AWS::CloudWatch::Alarm
+    Properties:
+      AlarmName: !Sub '${AWS::StackName}-API-Gateway-5XX-Error'
+      AlarmDescription: Monitor API Gateway 5XX errors
+      MetricName: 5XXError
+      Namespace: AWS/ApiGateway
+      Dimensions:
+        - Name: ApiName
+          Value: RestApi
+        - Name: Stage
+          Value: s1
+      Statistic: Sum
+      Period: 300
+      EvaluationPeriods: 1
+      Threshold: 3
+      ComparisonOperator: GreaterThanThreshold
+      TreatMissingData: notBreaching
+
+  # S3 Bucket for Synthetics Canary Artifacts
+  SyntheticsArtifactsBucket:
+    Type: AWS::S3::Bucket
+    DeletionPolicy: Retain
+    Properties:
+      PublicAccessBlockConfiguration:
+        BlockPublicAcls: true
+        BlockPublicPolicy: true
+        IgnorePublicAcls: true
+        RestrictPublicBuckets: true
+
+  # IAM Role for Synthetics Canary
+  SyntheticsCanaryRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Version: '2012-10-17'
+        Statement:
+          - Effect: Allow
+            Principal:
+              Service: 
+                - synthetics.amazonaws.com
+                - lambda.amazonaws.com
+            Action: sts:AssumeRole
+      Policies:
+        - PolicyName: SyntheticsPolicy
+          PolicyDocument:
+            Version: '2012-10-17'
+            Statement:
+              - Effect: Allow
+                Action:
+                  - s3:PutObject
+                  - s3:GetObject
+                  - s3:ListBucket
+                  - s3:ListAllMyBuckets
+                  - s3:GetBucketLocation
+                Resource:
+                  - !Sub '${SyntheticsArtifactsBucket.Arn}/*'
+                  - !GetAtt SyntheticsArtifactsBucket.Arn
+              - Effect: Allow
+                Action:
+                  - logs:CreateLogGroup
+                  - logs:CreateLogStream
+                  - logs:PutLogEvents
+                  - cloudwatch:PutMetricData
+                  - synthetics:*
+                Resource: '*'
+
+  # Synthetics Canary
+  ApiGatewayCanary:
+    Type: AWS::Synthetics::Canary
+    Properties:
+      Name: !Sub '${AWS::StackName}-api-gw-canary'
+      RuntimeVersion: syn-nodejs-puppeteer-9.0
+      ExecutionRoleArn: !GetAtt SyntheticsCanaryRole.Arn
+      ArtifactS3Location: !Sub 's3://${SyntheticsArtifactsBucket}/'
+      Schedule:
+        Expression: 'rate(1 minute)'
+        DurationInSeconds: 0
+      RunConfig:
+        TimeoutInSeconds: 60
+        MemoryInMB: 960
+      FailureRetentionPeriod: 30
+      SuccessRetentionPeriod: 30
+      StartCanaryAfterCreation: true
+      Code:
+        Handler: canary.handler
+        Script: !Sub |
+          const { URL } = require('url');
+          const synthetics = require('Synthetics');
+          const log = require('SyntheticsLogger');
+          const syntheticsConfiguration = synthetics.getConfiguration();
+          const syntheticsLogHelper = require('SyntheticsLogHelper');
+
+          const loadBlueprint = async function () {
+              const urls = ['https://${RestApi}.execute-api.${AWS::Region}.amazonaws.com/s1'];
+              const takeScreenshot = true;
+
+              syntheticsConfiguration.disableStepScreenshots();
+              syntheticsConfiguration.setConfig({
+                 continueOnStepFailure: true,
+                 includeRequestHeaders: true,
+                 includeResponseHeaders: true,
+                 restrictedHeaders: [],
+                 restrictedUrlParameters: []
+              });
+
+              let page = await synthetics.getPage();
+
+              for (const url of urls) {
+                  await loadUrl(page, url, takeScreenshot);
+              }
+          };
+
+          const resetPage = async function(page) {
+              try {
+                  await page.goto('about:blank',{waitUntil: ['load', 'networkidle0'], timeout: 30000} );
+              } catch (e) {
+                  synthetics.addExecutionError('Unable to open a blank page. ', e);
+              }
+          }
+
+          const loadUrl = async function (page, url, takeScreenshot) {
+              let stepName = null;
+              let domcontentloaded = false;
+
+              try {
+                  stepName = new URL(url).hostname;
+              } catch (e) {
+                  const errorString = 'Error parsing url: ' + url + '. ' + e;
+                  log.error(errorString);
+                  throw e;
+              }
+
+              await synthetics.executeStep(stepName, async function () {
+                  const sanitizedUrl = syntheticsLogHelper.getSanitizedUrl(url);
+                  const response = await page.goto(url, { waitUntil: ['domcontentloaded'], timeout: 30000});
+
+                  if (response) {
+                      domcontentloaded = true;
+                      const status = response.status();
+                      const statusText = response.statusText();
+
+                      if (response.status() < 200 || response.status() > 299) {
+                          throw new Error('Failed to load url: ' + sanitizedUrl + ' ' + response.status() + ' ' + response.statusText());
+                      }
+                  } else {
+                      const logNoResponseString = 'No response returned for url: ' + sanitizedUrl;
+                      log.error(logNoResponseString);
+                      throw new Error(logNoResponseString);
+                  }
+              });
+
+              if (domcontentloaded && takeScreenshot) {
+                  await new Promise(r => setTimeout(r, 15000));
+                  await synthetics.takeScreenshot(stepName, 'loaded');
+              }
+
+              await resetPage(page);
+          };
+
+          exports.handler = async () => {
+              return await loadBlueprint();
+          };
+
 Outputs:
   lambdaArn:
     Value: !GetAtt lambdaFunction.Arn
@@ -62,4 +226,16 @@ Outputs:
     Value: !Ref MySnsTopic
 
   apiGatewayInvokeURL:
-    Value: !Sub https://${RestApi}.execute-api.${AWS::Region}.amazonaws.com/s1
+    Value: !Sub https://${RestApi}.execute-api.${AWS::Region}.amazonaws.com/s1
+
+  CloudWatchAlarmName:
+    Description: Name of the API Gateway 5XX Error Alarm
+    Value: !Ref ApiGateway5XXErrorAlarm
+
+  SyntheticsCanaryName:
+    Description: Name of the Synthetics Canary
+    Value: !Ref ApiGatewayCanary
+
+  SyntheticsArtifactsBucket:
+    Description: S3 Bucket for Synthetics artifacts (delete manually after stack deletion)
+    Value: !Ref SyntheticsArtifactsBucket