Showing posts with label Scripting. Show all posts
Showing posts with label Scripting. Show all posts

AWS Automation using Terraform


Like I've said in an older blog, I have been learning about automation and CI/CD.
oh no! "CI/CD" not there yet.
Here's what curious me to focus on Automation of infrastructure,
 This year I did the research so call "Mult-Cloud Docker Container Communication using SDN" in which I built our own platform for 3rd level infrastructure to deploy the container on cross-cloud platforms with SDN technology and I developed a tool to provision container communication just like Kubernetes. So, Automation in the end achieved and gave me some interest.

Alright! let's move on to Topic,

In this tutorial, I am gonna cover AWS instance deployment using terraform and Other key things below.

Please refer the Architecture below to understand my idea



The figure shows how VPC and Subnet are being created to have an isolated virtual network to deploy instances.

Assume in the AWS selected region we have 3 Availability Zone(AZ1, AZ2, AZ3) and on top of it, I have created a VPC (10.1.0.0/16) and on each AZ, I create Public Subnets and Private Subnets.

Then, I created aws_internet_gateway (main-gw) which is to connect the internet and I have created Public Gateway (main-public) for the public subnets and I did create route associations of all public instance in multiple availability_zone to the main-public route table.

That is pretty much it. let move on to the steps to do it.
find the code here

1. mkdir terraform-code
2. In order to create an instance via CLI or anything, we need AWS IAM roles. please make sure you have created IAM roles as with admin privileges.
3. Create these files in your working directory to add AWS IAM keys.
terraform.tfvars


provider.tf
this is something like a variable which refer from tfvar file


vars.tf

this is where we store actual variables. below code is what region we are going to deploy our instances and as per our region, we are selecting AWS AMI for instances.


So, our IAM key is being passed and since we are launching a VM it needs to be accessed through SSH. So, we have to have private key in our machine and upload the public key to the instance.

To generate key type below code in your terminal. it will create private and public keys in your directory.
ssh-keygen -f 

those key to be uploaded so,

modify the vars.tf file below
vars.tf

# this ssh private key and compulsory to ssh AWS instance

# I create ssh key by ssh-keygen -f command in my working directory

# I have created a variable for my private key which points to my key location
variable "PATH_TO_PRIVATE_KEY" {
  default = "mykey"
}

# I have created a variable for my public key path which points to my key location
variable "PATH_TO_PUBLIC_KEY" {
  default = "mykey.pub"

}




create key.tf to announce aws key resource

key.tf 

resource "aws_key_pair" "mykeypair" {

  key_name   = "mykeypair"
#this is refer my vars.tf file and vars.tf file points to the key location
  public_key = "${(file("${var.PATH_TO_PUBLIC_KEY}")}"
}






Create VPC and Subnets
  A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. You can launch your AWS resources, such as Amazon EC2 instances, into your VPC.
 for more VPC & Subnet refer here

As I mentioned earlier just create vpc.tf file and enter this code 

Create a Security Group to allow certain traffic to an instance

create securitygroup.tf

ingress - the traffic which we are allowing meaning that and instance opened ssh port for listening.

egress -allow all IP meaning that anyone incoming IP is accepted

#protoc -1 = all protocol

resource "aws_security_group" "allow-ssh" {
  vpc_id      = "${var.aws_vpc.main.id}"
  name        = "allow-ssh"
  description = "security group that allows ssh and all egress traffic"
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = {
    Name = "allow-ssh"
  }
}

So, let's create instance.tf file to specify instance resource including additional volume.


resource "aws_instance" "example" {
ami = "${lookup(var.AMIS, var.AWS_REGION)}"
instance_type = "t2.micro"
# the VPC subnet
#launch instances from public subnet
subnet_id = "${aws_subnet.main-public-1.id}"
# the security group
vpc_security_group_ids = ["${aws_security_group.allow-ssh.id}"]
# the public SSH key
key_name = "${aws_key_pair.mykeypair.key_name}"
}
The above code explained below,
1. We create an instance group called "example" (to identify a group of resource)
2. We are selecting AMI type (looking based on our region passed through vars.tf)
3. Instance type t2.micro free tier
4. Assing VPC and Subnet to the instance
5. As I mention about key.tf file, we are referring public key to upload

on the same file lets add root volume and addtional volume.
resource "aws_ebs_volume" "ebs-volume-1" {
availability_zone = "eu-west-1a"
size = 20
type = "gp2"
tags = {
Name = "extra volume data"
}
}
resource "aws_volume_attachment" "ebs-volume-1-attachment" {
device_name = "/dev/xvdh"
volume_id = "${aws_ebs_volume.ebs-volume-1.id}"
instance_id = "${aws_instance.example.id}"
}


find the full code here

How to Use Powershell Script to Execute Linux Shell -POSH


This article demonstrates how to use Powershell POSH-SSH module to login Linux Machine and Execute Shell Commands
Posh-SSH is a PowerShell module for Windows, which allows you to establish SSH connections as well as SCP functionality to remote computers. Let's dig in...

WHY?
I've been learning about AWS Linux instances and automation. I used Terraform for a spin up AWS instances and execute scripts. But, Also often think about what if create a web console to do such a task from local pc.
I will write how to automate AWS using Terraform Later. Now I'll show how to create a web console to execute Linux shell using PowerShell script.
To achieve this,
1. Create Powershell Script to SSH AWS Linux instances
2. Create ASP.NET web console

Create a PowerShell script - POSH

Step 01: To verify whether POSH-SSH installed
Find-Module Posh-SSH

Step 02:  Install Posh-SSH
Get-Command -Module Posh-SSH

Step 03: Once It's installed All you need to Know Posh CommandI'll Be Link all the POSH-SSH Command to Learn...
https://github.com/5sfayas/POSH-SHH


Since I did not create any web console yet, though, I'll add a Simple Powershell Script.
The below code will be changed as per the web console. So, Please  follow up the tutorial as I go

Write-Host "Please Enter your parameter to ping"
$cloud=Read-Host

#connect to controller

$nopasswd = new-object System.Security.SecureString
$Crendential= New-Object System.Management.Automation.PSCredential ("ubuntu", $nopasswd)
  
#SSH Session is created connect aws instance using private key
$ssh=New-SSHSession –ComputerName ec2-13-232-138-59.ap-south-1.compute.amazonaws.com -KeyFile 'D:\w\tom.pem' -Credential $Crendential


#Script which deployed in aws instance and which also take parameters
$command="sh new.sh $cloud"
Write-Host "Pinging....."

#This is Write the output to powershell
$result=Invoke-SSHCommand -SSHSession $ssh -Command $command | select Output -ExpandProperty Output | Out-String
Write-Host " Ping is: $result"

#End ssh session
$end_result=Remove-SSHSession -SSHSession $ssh

Let's Create Web Console To Exexute Shell

All you Have to do is insert parameter and click right away

Here's the ASP.NET code which was written in..

find this in repo  https://github.com/5sfayas/POSH-SHH/blob/master/POSH-SSH/Ping_Tool.aspx.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Management.Automation;
using System.Text;

namespace Final_CloudApp
{
    public partial class Ping_Tool : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {

        }


        //****Function start*****************
        public void ping(string cloud, string parameter)
        {

            //Location of the script
            string script = @cloud;
            TextBox3.Text = string.Empty;

            // Initialize PowerShell engine
            var shell = PowerShell.Create();

            // Add the script to the PowerShell object
            shell.Commands.AddCommand(script, false);

            //Name of the parameter as in the script
            shell.Commands.AddParameter("com1", source); //Parameter-0
            shell.Commands.AddParameter("com2", dest);//Parameter-1

            // Execute the script
            var results = shell.Invoke();

            // Note : use |out-string for console-like output
            if (results.Count > 0)
            {
                // We use a string builder ton create our result text
                var builder = new StringBuilder();

                foreach (var psObject in results)
                {
                    // Convert the Base Object to a string and append it to the string builder.
                    // Add \r\n for line breaks
                    builder.Append(psObject.BaseObject.ToString() + "\r\n");
                }

                // Encode the string in HTML (prevent security issue with 'dangerous' caracters like < >
                // TextBox2.Text = Server.HtmlEncode(builder.ToString());
                TextBox3.Text = (builder.ToString());
            }

        }
        //Function Finish

        protected void Button1_Click(object sender, EventArgs e)
        {
           
                ping("C:\\D drive\\Ransara\\aws_ping.ps1", TextBox2.Text);
           
        }
    }
}



then fully modify your PowerShell script like below
find this in repo  https://github.com/5sfayas/POSH-SHH/blob/master/POSH-SSH/execute_shell1.ps1

[CmdletBinding()]
param(
    [parameter(position=0)]
    [string]$cloud,
)


#connect to controller

$nopasswd = new-object System.Security.SecureString
$Crendential= New-Object System.Management.Automation.PSCredential ("ubuntu", $nopasswd)
 
#SSH Session is created connect aws instance using private key
$ssh=New-SSHSession –ComputerName ec2-13-232-138-59.ap-south-1.compute.amazonaws.com -KeyFile 'D:\w\tom.pem' -Credential $Crendential
#Script which deployed in aws instance and which also take parameters
$command="sh new.sh $cloud"
Write-Host "Pinging....."

#This is Write the output to powershell
$result=Invoke-SSHCommand -SSHSession $ssh -Command $command | select Output -ExpandProperty Output | Out-String
Write-Host " Ping is: $result"

#End ssh session
$end_result=Remove-SSHSession -SSHSession $ssh