Recently, I tried to create an AWS Lambda trigger to process a stream from a DynamoDB table which triggers SNS notification to subscribers.
Think about a scenario that you have deployed and a website in which a user fille the feedback form and you want to get an email notification for every feedback that has been filled.
So, this is where AWS SNS, Dynamodb, lambda, and API gateway comes in.
Please find my previous tutorial on how to deploy the serverless website here
In this case, I used Terraform and AWS CLI as usual.
Note: Just follow up the tutorial, I'll mention when to use AWS CLI (only for SNS Subscription)
Lest's Dig in,
Requirement:
Install all nessacery things to run Terraform and AWS CLI.
Steps
Step 1: Create a DynamoDB Table with a Stream Enabled
Step 2: Create a Lambda Execution Role
Step 3: Create an Amazon SNS Topic
Step 4: Create and Test a Lambda Function
Step 5: Create and Test a Trigger
Step 1: Create a DynamoDB Table with a Stream Enabled
Create a DynamoDB table (mywebsite) to store all of the barks from Woofer users. The primary key as email (partition key) attribute type string.
[fayasak@controller]$ cat iam.tfresource "aws_dynamodb_table" "dynamodb"{ name="mywebsite" hash_key="email" attribute=[ { name="email" type="s" } ] stream_enabled = "true" stream_view_type ="NEW_AND_OLD_IMAGES"}
Step 2: Create a Lambda Execution Role
Create an AWS Identity and Access Management (IAM) role (lamda-dynamodb) and assign permissions to it. This role is used by the Lambda function that you create later.
Also create a policy for the role. The policy contains all of the permissions that the Lambda function needs at runtime.
[fayasak@controller]$ cat dynamo.tfresouce "aws_iam_role" "lamda-iam"{ #role name name="lamda-dynamodb" #assume role to a services assume_role_policy =<<EOF { "Version": "2012-10-17", "Statement": [ { "Action": "sts:AssumeRole", "Principal": { "Service": "lambda.amazonaws.com" }, "Effect": "Allow", "Sid": "" } ] }EOF}
Also Create an inline policy with the following contents. (Replace region and accountID with your AWS Region and account ID where 4444 is placed.)
[fayasak@controller]$ cat iam.tfresource "aws_iam_role_policy" "dynamodb-lambda-policy"{ name = "dynamodb_lambda_policy" role = "${aws_iam_role.lamda-iam.id}"
policy = <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "dynamodb:DescribeStream", "dynamodb:GetRecords", "dynamodb:GetShardIterator", "dynamodb:ListStreams" ], "Resource": "${aws_dynamodb_table.dynamodb.arn}" }, { "Effect": "Allow", "Action": "lambda:InvokeFunction", "Resource": "${aws_lamda_function.lambda_test.arn}" }, { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": ":aws:logs:eu-west-1:444-444444:*" }, { "Effect": "Allow", "Action": [ "sns:Publish" ], "Resource": [ "*" ] } ] } EOF}
The policy has four statements that allow us to do the following:
1. Execute a Lambda function (lambda_dbstream). You create the function later in this tutorial.
2.Access Amazon CloudWatch Logs. The Lambda function writes diagnostics to CloudWatch Logs at runtime.
3. Read data from the DynamoDB stream for mywebsite .
4. Publish messages to Amazon SNS (since we deploying this via AWS CLI I cannot pass the resource ARN to it. though I required you to manually add the ARN name above ).
Step 3: Create an Amazon SNS Topic
As I mentioned earlier, I used AWS CLI Command and made it to shell script to run efficiently.
Note: You have to run this file manually
[fayasak@controller]$ cat create-subscption.sh#!/usr/bin/env bashread -p "Enter sns topic name: " topicnameread -p "Enter sns region: " regionread -p "Enter sns accountID: " accountIDread -p "Enter sns email address to subscribe: " emailaws sns create-topic --name $topicname
aws sns subscribe \ --topic-arn arn:aws:sns:$region:$accountID:$topicname \ --protocol email \ --notification-endpoint $email
This creates an Amazon SNS topic and subscribe to an email address to it. Your Lambda function uses this topic to publish new inputs from mywebsite table.
Step 4: Create and Test a Lambda Function
Create an AWS Lambda function handler to process stream records from mywebsite table.
The lambda_dbstream function processes only the stream events that correspond to new items in mywebsite. The function reads data from such an event, and then invokes Amazon SNS to publish it.
Note: I used python runtime here
[fayasak@controller]$ cat dbstream.py import jason import boto3 AWS = require("aws-sdk") sns = boto3.resource('sns') def lamda_handle(event, context): for record in event['Record']: if record['EventName'] == 'INSERT': newImgae = record['dynamodb'['NewImage'] newEmail = NewImage['Email']['S'] fname = NewImage['fname']['S'] feedback = NewImage['feedback']['S'] try: sns.publish( Subject: 'A new feedback from ' + fname, Message: 'A new user ' + fname + 'left a feed which is '+ feedback + ':\n\n And his Email is ' + newEmail, MessageStructure='json', TopicArn: 'arn:aws:sns:$region:$accountID:$topicname' ) except Exception as e: print (e)
Create a zip file to contain dbstream.py. If you have the zip command-line utility, you can enter the following command to do this
Now create a lambda function using terraform to attach our lambda handler file
[fayasak@controller]$ cat lambda.tfresource "aws_lambda_function" "lambda_dbstream"{ function_name = "dbstream" filename = "./dbstream.zip/dbstream.py" role = "${aws_iam_role.lamda-iam.arn}" handler = "dbstream.handler" runtime = "python3.7" }
Note: At this point goto the console and test with test data to make sure that everything works great
Step 5: Create and Test a Trigger
I hope you all tested the Lambda function to ensure that it ran correctly. In this step, you create a trigger by associating the Lambda function (lambda_dbstream) with an event source (the Dynamodb table stream).
[fayasak@controller]$ cat lambda.tfresource "aws_lambda_event_source_mapping" "lamda-dynamodb" {
event_source_arn = "${aws_dynamodb_table.dynamodb.stream_arn}"
function_name = "${aws_lambda_function.lambda_dbstream.arn}"
starting_position = "LATEST"
}
find the full code here
That's pretty much it,
"I created a test.sh file which tests our case and once the script completion success you will get an email based on your input"
Catch you guys next TIME!
No comments:
Post a Comment