Optimize the Cost by Shutting down EC2 Instances using Lambda

Recently, I have written an article about launching Instances on the AutoScaling Group in which  I have configured CloudWatch alarms to trigger to launch  EC2 instances based on aggregate CPU usage within the autoscaling group (used scaling policy). 

find the previous article here

Then, I came across that if  EC2 instances in AWS are managed through Auto Scaling Groups, it is easy to schedule startup and shutdown of those instances and It opens up a question that a cloud infrastructure always comes at a cost and how to Optimize the Cost of our AWS full Architecture?

Scenario: Developers working on a testing phase of their project and to reduce the cost. The manager decided to Shut down all the AWS Infrastructure services after office hours (shutting down the instances at nights and launch them back on the working mornings). So it becomes cumbersome for the cloud team to re-provision all the environment on a daily basis.

Though, I decided to automate this process using lambda and cloud watch rule.  

Note: As usual, I used IaaC (terraform) to all the processes.

Let's dive in,

Steps:
1. Create AWS IAM Role and Policies for Lambda
2. Create AWS LAMBDA function to start and stop instances
3. Use Cron AWS Cloudwatch Rules to trigger Lambda function

Find the full code here

1. Create AWS IAM Role and Policies for Lambda
I created an IAM role for assuming lambda and then attached an inline policy to access autoscaling group

# /iam.tf
#create a role for lamda access aws resouces
resouce "aws_iam_role" "lamda-iam"{
 #role name
 name="lamda-asg"
 
 #assume role to a services
 assume_role_policy =<<EOF
 {
  "Version": "2012-10-17",
  "Statement": [
  {
   "Action": "sts:AssumeRole",
   "Principal": {
   "Service": "lambda.amazonaws.com"
  },
  "Effect": "Allow",
  "Sid": ""
    }
  ]
 }
EOF
}
# create policy for role to assume autoscaling
resource "aws_iam_role_policy" "autoscaling-lambda-policy"{
 name = "autoscaling_lambda_policy"
 role = "${aws_iam_role.lamda-iam.id}"

 policy = <<EOF
 {
  "Version": "2012-10-17",
  "Statement": [
  {
   "Effect": "Allow",
   "Action": [
   "autoscaling:*"
   ],
   "Resource": "*"
  }
  ]
 }
 EOF
}

2. Create AWS LAMBDA function to start and stop instances
The below function creates AWS Lambda function and autoscaling.py lambda code has attached to the function.

# /scale-up-daown-lamda.tf
# create lambda 
# to invoke autoscaling group
resource "aws_lambda_function" "lambda_test"{
 function_name = "lambda_stop_start_autoscaling_instances"
 filename = "./autoscaling.zip/autoscaling.py"
 role = "${aws_iam_role.lamda-iam.arn}"
 handler = "autoscaling.handler"
 runtime = "python2.7" 
}

Lambda function

# /autoscaling.py
import boto3
import os
# Boto Connection
# aws_region, autoscaling-group-name,min,max and desired capacity has to be passed
asg = boto3.client('autoscaling','os.environ[aws_region]')
def lambda_handler(event, context):
  response = asg.update_auto_scaling_group(AutoScalingGroupName=os.environ['asg_name'],MinSize=os.environ['min'],DesiredCapacity=os.environ['desired'],MaxSize=os.environ['max'])

Now lambda function has created and let's create both start and stop CloudWatch rules
Not please write down below scale-up-daown-lamda.tf file

To Stop the autoscaling EC2 Instance

# /scale-up-daown-lamda.tf
#...

#_______stop instances at 10pm everyday_________________
resource "aws_cloudwatch_event_rule" "stop" {
  name                = "everyday-10"
  description         = "stop instances everyday 10 pm" 
  schedule_expression = "0 0 22 1/1 * ? *"
  event_pattern = <<PATTERN
{
 "aws_region" : "eu-west-1",
    "asg_name" : "webapp",
    "min" : "0",
    "desired" : "0",
    "max" : "10"
}
PATTERN
}
resource "aws_cloudwatch_event_target" "lambda-stop" {
  rule      = "${aws_cloudwatch_event_rule.stop.name}"
  target_id = "lambda"
  arn       = "${aws_lambda_function.lambda_test.arn}"
}
resource "aws_lambda_permission" "lamda-permision" {
  statement_id  = "AllowExecutionbycloudwatchrule"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.lambda_test.function_name}"
  principal     = "events.amazonaws.com"
  source_arn    = "${aws_cloudwatch_event_rule.stop.arn}"
}
#_______end of stop instances at 10pm_________________

As you can see from the above code,  schedule_expression = "0 0 22 1/1 * ? *"  is a Cron expression and which specify at which day(s) and time this event should trigger the event (in my case, 10 pm). 
See this website for a handy Cron calculator.
To stop the instance, we have set the MinSize and DesiredSize to 0. So I passed min, desired value as 0 to my lambda function as Jason format below.

 event_pattern = <<PATTERN
{
 "aws_region" : "eu-west-1",
    "asg_name" : "webapp",
    "min" : "0",
    "desired" : "0",
    "max" : "10"
}
PATTERN

Now, the stop rule has been created. resource "aws_cloudwatch_event_target" "lambda-stop".. used to select a target to invoke when an event matches. (literally trigger the lambda function on cloud watch rule event).

To Sart the autoscaling EC2 Instance

# /scale-up-daown-lamda.tf
#...

#_______start instances at 7am everyday_________________
resource "aws_cloudwatch_event_rule" "start" {
  name                = "everyday-7am"
  description         = "start instances everyday 7am" 
  schedule_expression = "0 0 7 1/1 * ? *"
  event_pattern = <<PATTERN
{
 "aws_region" : "eu-west-1",
    "asg_name" : "webapp",
    "min" : "2",
    "desired" : "5",
    "max" : "10"
}
PATTERN
}
resource "aws_cloudwatch_event_target" "lambda-start" {
  rule      = "${aws_cloudwatch_event_rule.start.name}"
  target_id = "lambda1"
  arn       = "${aws_lambda_function.lambda_test.arn}"
}
resource "aws_lambda_permission" "lamda-permision" {
  statement_id  = "AllowExecutionbycloudwatchrule"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.lambda_test.function_name}"
  principal     = "events.amazonaws.com"
  source_arn    = "${aws_cloudwatch_event_rule.start.arn}"
}

The code Exactly the same as the stop instances.schedule_expression = "0 0 7 1/1 * ? *" Cron expression specified to start the autoscaling group at 7am moring.
And MinSize and DesiredSize have given as we configured autoscaling from the beginning. Meaning that it brings back the EC2 instances as Exactly we created the autoscaling group earlier at launch first launch.

#luanch instance as we configured autoscaling from the begining
event_pattern = <<PATTERN
{
 "aws_region" : "eu-west-1",
    "asg_name" : "webapp",
    "min" : "2",
    "desired" : "5",
    "max" : "10"
}
PATTERN

Now we can easily we can stop our EC2 instances at night to save money on our cloud bill and automate this process using lambda. To make sure your autoscaling is done properly, we should monitor the EC2 instance with Cloudwatch.

Find the full repo here

That's pretty much it guys! Thank you