Showing posts with label VPC. Show all posts
Showing posts with label VPC. Show all posts

Things to know about set_facts Ansible


Hello Everyone!

 I've been reading and learning about AWS automation using Ansible these days!
I have decided to deploy a LAMP stack on AWS and I had an Issue on how to provide different subnets for different stack (EC2 instance) while we create a single role for the subnet using Ansible!

 Let me explain which helps us to connect the dot before digging into the solution.

 Use case: We need to spin up 3 instances and each instance needs to be assigned in different subnets. this will be set up on the fly when instances spun up!

 Solution 
Register and set_facts go hand in hand

Register: registering the result of that command as a variable. When you execute a task and save the return value in a variable to use later tasks, In such case you create a registered variable.

set_facts:  on Ansible Document page, it says that set_facts for host specific and use to register variable against the playbook we are running!

 This is sucks and it provides very least explanation. It's not sufficient for our issue to solve it!

 I was thinking that how do I capture each subnet ID using that module when I create a common subnet role EC2 instance creation playbook.

 Explanation gives you nothing unless you don't look into the code!
Let's dig in,

 Myplaybook
 |---- EC2_webserver.yml 
 |---- EC2_application.yml
 |---- Subnet.yml
 |---- roles
     |--- subnet.yml
 |--- web_subnet.yml
 |--- app_subnet.yml

 Subnet.yml


- import_playbook web_subnet.yml- import_playbook app_subnet.yml
web_subnet.yml

 ---
- name: create subnet for webserver1
  hosts: controt
# ask input from users
  vars_prompt:
    - name: "cidr_block_subnet1"
      prompt: "Enter the CIDR block you want for web server 1 subnet"
      private: no
    - name: "subnet_name1"
      prompt: "Enter the name of the web server 1 subnet"
      private: no
    - name: "subnet_az1"
      prompt: "Enter the availability zone of web server 1 subnet"
      private: no

  tasks:
    - set_fact:
        info: {}

    - name: creating webserver1
      include roles:
        name: ./roles/subnet
      vars:
        cidr_block_subnet: "{{cidr_block_subnet1}}"
        subnet_name: "{{subnet_name1}}"
        az: "{{subnet_az1}}"
 # subnetinfo is used to store subnet infomation
 # when we executing subnet roles    subnetinfo: wb1_subnet

    - name:  print webserver1 output
      debug:
        var: info
app_subnet.yml

 ---
- name: create subnet for appserver
  hosts: controt
# ask input from users
  vars_prompt:
    - name: "cidr_block_subnet2"
      prompt: "Enter the CIDR block you want for app server 1 subnet"
      private: no
    - name: "subnet_name2"
      prompt: "Enter the name of the app server 1 subnet"
      private: no
    - name: "subnet_az1"
      prompt: "Enter the availability zone of app server 1 subnet"
      private: no

  tasks:
    - set_fact:
        info: {}

    - name: creating app1
      include roles:
        name: ./roles/subnet
      vars:
        cidr_block_subnet: "{{cidr_block_subnet1}}"
        subnet_name: "{{subnet_name1}}"
        az: "{{subnet_az1}}"
 # subnetinfo is used to store subnet infomation
# when we executing subnet roles
    subnetinfo: wb1_subnet

    - name:  print app1 output
      debug:
        var: info

 Since it is using the same subnet role, on each playbook we need to capture web server and app server subnet information.
I used subnetifor variable and its value replace with each subnet role output.
Don't worry!
 if you see roles/subnet.yml
you will understand how it replaces subnet information.

 roles/subnet.yml

 ---
# tasks file for subnet
# this can be use as common roles for each LAMP stack
# name provide which server we are creating subnets
- name: creating subnets "{{subnet_name}}"
    ec2_vpc_subnet:
      state: present
      vpc_id: "{{ vpc.vpc.id }}"
      region: "{{default_region}}"
      az: "{{subnet_az}}"
      cidr: "{{cidr_block}}"
      resource_tags:
        Name: "{{subnet_name}}"
register: output


# register stores all the output of each subnet information.
# So!
# I want store each Subnet ID for Each stack
# I found combined jina function which replace variable value
# if so, what if I combined this output to a variable of stack
# then luanch instanes on specific subnets

- name: get subnetid of particular Stack
  set_fact:
    info: ""{{ info | combine({subnetinfo: output}) }}"


# subnetinfo is dynamic variable and it is passed with each stack subnet.yml
# and I am overwriting the output

Note: You can cache a fact set from set_facts the module so that when you execute your playbook next time, it's retrieved from the cache. You can set cacheable to yes to store variables across your playbook executions using a fact cache. You may need to look into precedence strategies used by ansible to evaluate the cacheable facts mentioned in their documentation.

 Find the full code here

 That's pretty much it for today, PEACE!

Thank you!



AWS EC2 intance Automation using Ansible

Hi folks,
It has been a long time since I wrote the last blog. and I have gone through little emotional stress. Though it took me little time to overcome the stress!

 Like I've said in an older blog, I have been learning about automation and CI/CD. I learned terraform a bit and I did spin up EC2 instances, public subnet, a private subnet, Internet gateway, security group and deploy some shell script.

 you can find the blog here

 In this blog, I wanna write about Ansible (spin-up EC2 instances and other key things. same as the previous blog). It was a pretty good experience. you can create a simple YAML file and run playbooks.

 My Idea is,
1. Create VPC
2. Create public and private subnets for each Availability Zones
3. Create an internet gateway
4. Create a public gateway and make associations with public gateway
5. spin up instances on a specific subnet.

 find the full code here

Please refer the Architecture below to understand my idea

 Before moving onto the subject, Ansible requires some requirements to run ansbile module
  
Ansible : sudo pip install ansible
Boto : sudo pip install boto
here are many ways to set our AWS credentials, in this tutorial, we'll create a file under our user home folder (~/.boto):
[Credentials]
AWS_ACCESS_KEY_ID=KID...CWU
AWS_SECRET_ACCESS_EY=3qv...DSP
 AWS CLI : sudo pip install awscli
Note that when launching an EC2 instance with ansible via the ansible ec2 module, the hosts variable should point to localhost and gather_facts should be set to False.

 
- hosts: local  gather_facts: flase  roles:    - vpc
Create VPC

  I did include some other variables under group_vars directory to fetch some essential variables.
To create the VPC ec2_vpc _net module used.
- include_vars: ./group_vars/all.yml

- name: create vpc with 10.0.0.0/16
  ec2_vpc_net:
    name: ansibletest
    cidr_block: 10.0.0.0/16
    region: "{{default_region}}"
    tags:
      Name: ansibletest
    state: present
    aws_access_key: "{{ aws_access_key }}"
    aws_secret_key: "{{ aws_secret_key }}"
    tenancy: default
    dns_hostnames: yes
    dns_support: yes
  register: vpc_info
  #store output of ec2 infroamtion
likewise, we can use available modules to create AWS services. Some other modules are listed below.

 ec2_vpc_subnet (this modules used create subnets)
ec2_vpc_igw (create internet gateway
ec2_vpc_route_table (create routing table and make association for subnets)

The vpc role which I created to deploy those services here

 Create security groups

 security groups especially stand to allow certain traffic to instances.
when I created security groups, I had in mind to spin up instances for deploy LAMP stack. So, I wanted to allow ports related to the LAMP stack.
Later I developed different efficient codes to do this. 

 But here's the security group role which I created.

 

- include_vars: ./group_vars/all.yml

- name: security group with 22,80,443 port enable
  ec2_group:
    name: LAMP_Stack
    description: sg with 22,80,443 port enable
    #right now I'am using vpc_id from ./group_vars/all.yml
    vpc_id: "{{vpc_id}}"
    rules:
      - proto: tcp
        from_port: 80
        to_port: 80
        cidr_ip: 0.0.0.0/0
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: 0.0.0.0/0

let's move onto final steps

AWS Automation using Terraform


Like I've said in an older blog, I have been learning about automation and CI/CD.
oh no! "CI/CD" not there yet.
Here's what curious me to focus on Automation of infrastructure,
 This year I did the research so call "Mult-Cloud Docker Container Communication using SDN" in which I built our own platform for 3rd level infrastructure to deploy the container on cross-cloud platforms with SDN technology and I developed a tool to provision container communication just like Kubernetes. So, Automation in the end achieved and gave me some interest.

 Alright! let's move on to Topic,

 In this tutorial, I am gonna cover AWS instance deployment using terraform and Other key things below.

 Please refer the Architecture below to understand my idea

The figure shows how VPC and Subnet are being created to have an isolated virtual network to deploy instances.

 Assume in the AWS selected region we have 3 Availability Zone(AZ1, AZ2, AZ3) and on top of it, I have created a VPC (10.1.0.0/16) and on each AZ, I create Public Subnets and Private Subnets.

 Then, I created aws_internet_gateway (main-gw) which is to connect the internet and I have created Public Gateway (main-public) for the public subnets and I did create route associations of all public instance in multiple availability_zone to the main-public route table.

 That is pretty much it. let move on to the steps to do it.
find the code here

 1. mkdir terraform-code
2. In order to create an instance via CLI or anything, we need AWS IAM roles. please make sure you have created IAM roles as with admin privileges.
3. Create these files in your working directory to add AWS IAM keys.
terraform.tfvars
provider.tf
this is something like a variable which refer from tfvar file
vars.tf
this is where we store actual variables. below code is what region we are going to deploy our instances and as per our region, we are selecting AWS AMI for instances.
So, our IAM key is being passed and since we are launching a VM it needs to be accessed through SSH. So, we have to have private key in our machine and upload the public key to the instance.

 To generate key type below code in your terminal. it will create private and public keys in your directory.
ssh-keygen -f 

 those key to be uploaded so,
modify the vars.tf file below
vars.tf

 # this ssh private key and compulsory to ssh AWS instance
# I create ssh key by ssh-keygen -f command in my working directory

 # I have created a variable for my private key which points to my key location
variable "PATH_TO_PRIVATE_KEY" {
  default = "mykey"
}

 # I have created a variable for my public key path which points to my key location
variable "PATH_TO_PUBLIC_KEY" {
  default = "mykey.pub"

 }


create key.tf to announce aws key resource
key.tf 

 resource "aws_key_pair" "mykeypair" {
  key_name   = "mykeypair"
#this is refer my vars.tf file and vars.tf file points to the key location
  public_key = "${(file("${var.PATH_TO_PUBLIC_KEY}")}"
}




Create VPC and Subnets
  A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. You can launch your AWS resources, such as Amazon EC2 instances, into your VPC.
 for more VPC & Subnet refer here

 As I mentioned earlier just create vpc.tf file and enter this code 

 Create a Security Group to allow certain traffic to an instance

 create securitygroup.tf

 ingress - the traffic which we are allowing meaning that and instance opened ssh port for listening.
egress -allow all IP meaning that anyone incoming IP is accepted

 #protoc -1 = all protocol
resource "aws_security_group" "allow-ssh" {
  vpc_id      = "${var.aws_vpc.main.id}"
  name        = "allow-ssh"
  description = "security group that allows ssh and all egress traffic"
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

   ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  tags = {
    Name = "allow-ssh"
  }
}

 So, let's create instance.tf file to specify instance resource including additional volume.
resource "aws_instance" "example" {
ami = "${lookup(var.AMIS, var.AWS_REGION)}"
instance_type = "t2.micro"
# the VPC subnet
#launch instances from public subnet
subnet_id = "${aws_subnet.main-public-1.id}"
# the security group
vpc_security_group_ids = ["${aws_security_group.allow-ssh.id}"]
# the public SSH key
key_name = "${aws_key_pair.mykeypair.key_name}"
}
The above code explained below,
1. We create an instance group called "example" (to identify a group of resource)
2. We are selecting AMI type (looking based on our region passed through vars.tf)
3. Instance type t2.micro free tier
4. Assing VPC and Subnet to the instance
5. As I mention about key.tf file, we are referring public key to upload

 on the same file lets add root volume and addtional volume.
resource "aws_ebs_volume" "ebs-volume-1" {
availability_zone = "eu-west-1a"
size = 20
type = "gp2"
tags = {
Name = "extra volume data"
}
}
resource "aws_volume_attachment" "ebs-volume-1-attachment" {
device_name = "/dev/xvdh"
volume_id = "${aws_ebs_volume.ebs-volume-1.id}"
instance_id = "${aws_instance.example.id}"
}
find the full code here
Subscribe to: Posts (Atom)