diff options
Diffstat (limited to 'docs/automation/terraform/terraformAWS.rst')
| -rw-r--r-- | docs/automation/terraform/terraformAWS.rst | 579 |
1 files changed, 579 insertions, 0 deletions
diff --git a/docs/automation/terraform/terraformAWS.rst b/docs/automation/terraform/terraformAWS.rst new file mode 100644 index 00000000..c81fe906 --- /dev/null +++ b/docs/automation/terraform/terraformAWS.rst @@ -0,0 +1,579 @@ +:lastproofread: 2024-01-11 + +.. _vyos-terraform: + +Terraform for VyOS +================== + +VyOS supports development infrastructure via Terraform and provisioning via Ansible. +Terraform allows you to automate the process of deploying instances on many cloud and virtual platforms. +In this article, we will look at using terraforms to deploy vyos on platforms - AWS, AZURE, and vSphere. +More detailed about what is Terraform you can write using the link_. + +Need to install_ Terraform + +Structure of files in the standard Terraform project: + +.. code-block:: none + + . + ├── main.tf # The main script + ├── version.tf # File for the changing version of Terraform. + ├── variables.tf # The file of all variables in "main.tf" + └── terraform.tfvars # The value of all variables (passwords, login, ip adresses and so on) + + +General commands that we will use for running Terraform scripts + + +.. code-block:: none + + #cd /<your folder> # go to the Terrafom project + #terraform init # install all addons and provider (aws az and so on) + #terraform plan # show wtah is changing + #terraform apply # run script + #yes # apply running + + +Deploying vyos in the AWS cloud +------------------------------- +With the help of terraforms, you can quickly deploy Vyos-based infrastructure in the AWS cloud. If necessary, the infrastructure can be removed using terraform. +Also we will make provisioning using Ansible. + + +.. image:: /_static/images/aws.png + :width: 50% + :align: center + :alt: Network Topology Diagram + +In this case, we'll create the necessary files for Terraform and Ansible next using Terraform we'll create a single instance on the AWS cloud and make provisioning using Ansible. + + +Preparation steps for deploying VyOS on AWS +------------------------------------------- + +How to create a single instance and install your configuration using Terraform+Ansible+AWS +Step by step: + +AWS + + +1.1 Create an account with AWS and get your "access_key", "secret key" + +1.2 Create a key pair_ and download your .pem key + +.. image:: /_static/images/keypairs.png + :width: 50% + :align: center + :alt: Network Topology Diagram + +1.3 Create a security group_ for the new VyOS instance and open all traffic + +.. image:: /_static/images/sg.png + :width: 50% + :align: center + :alt: Network Topology Diagram + + +.. image:: /_static/images/traffic.png + :width: 50% + :align: center + :alt: Network Topology Diagram + +Terraform + + +2.1 Create a0 UNIX or Windows instance + +2.2 Download and install Terraform + +2.3 Create the folder for example /root/awsterraform + +.. code-block:: none + + mkdir /root/awsterraform + +2.4 Copy all files into your Terraform project "/root/awsterraform" (vyos.tf, var.tf, terraform.tfvars,version.tf), more detailed see `Structure of files Terrafom for AWS`_ + +2.5 Type the commands : + +.. code-block:: none + + #cd /<your folder> + #terraform init + + +Ansible + + +3.1 Create a UNIX instance whenever you want (local, cloud, and so on) + +3.2 Download and install Ansible + +3.3 Create the folder for example /root/aws/ + +3.4 Copy all files into your Ansible project "/root/aws/" (ansible.cfg, instance.yml, mykey.pem and "all"), more detailed see `Structure of files Ansible for AWS`_ + +mykey.pem you have to get using step 1.2 + + +Start + + +4.1 Type the commands on your Terrafom instance: + +.. code-block:: none + + #cd /<your folder> + #terraform plan + #terraform apply + #yes + + +Start creating an AWS instance and check the result +--------------------------------------------------- + +.. code-block:: none + + root@localhost:~/awsterraform# terraform apply + + Terraform used the selected providers to generate the following execution plan. + Resource actions are indicated with the following symbols: + + create + + Terraform will perform the following actions: + + # aws_instance.myVyOSec2 will be created + + resource "aws_instance" "myVyOSec2" { + + ami = "ami-************62c2d" + + arn = (known after apply) + + associate_public_ip_address = (known after apply) + + availability_zone = (known after apply) + + cpu_core_count = (known after apply) + + cpu_threads_per_core = (known after apply) + + disable_api_stop = (known after apply) + + disable_api_termination = (known after apply) + + ebs_optimized = (known after apply) + + get_password_data = false + + host_id = (known after apply) + + host_resource_group_arn = (known after apply) + + iam_instance_profile = (known after apply) + + id = (known after apply) + + instance_initiated_shutdown_behavior = (known after apply) + + instance_lifecycle = (known after apply) + + instance_state = (known after apply) + + instance_type = "t2.micro" + + ipv6_address_count = (known after apply) + + ipv6_addresses = (known after apply) + + key_name = "awsterraform" + + monitoring = (known after apply) + + outpost_arn = (known after apply) + + password_data = (known after apply) + + placement_group = (known after apply) + + placement_partition_number = (known after apply) + + primary_network_interface_id = (known after apply) + + private_dns = (known after apply) + + private_ip = (known after apply) + + public_dns = (known after apply) + + public_ip = (known after apply) + + secondary_private_ips = (known after apply) + + security_groups = [ + + "awsterraformsg", + ] + + source_dest_check = true + + spot_instance_request_id = (known after apply) + + subnet_id = (known after apply) + + tags = { + + "name" = "VyOS System" + } + + tags_all = { + + "name" = "VyOS System" + } + + tenancy = (known after apply) + + user_data = (known after apply) + + user_data_base64 = (known after apply) + + user_data_replace_on_change = false + + vpc_security_group_ids = (known after apply) + } + + # local_file.ip will be created + + resource "local_file" "ip" { + + content = (known after apply) + + content_base64sha256 = (known after apply) + + content_base64sha512 = (known after apply) + + content_md5 = (known after apply) + + content_sha1 = (known after apply) + + content_sha256 = (known after apply) + + content_sha512 = (known after apply) + + directory_permission = "0777" + + file_permission = "0777" + + filename = "ip.txt" + + id = (known after apply) + } + + # null_resource.SSHconnection1 will be created + + resource "null_resource" "SSHconnection1" { + + id = (known after apply) + } + + # null_resource.SSHconnection2 will be created + + resource "null_resource" "SSHconnection2" { + + id = (known after apply) + } + + Plan: 4 to add, 0 to change, 0 to destroy. + + Changes to Outputs: + + my_IP = (known after apply) + + Do you want to perform these actions? + Terraform will perform the actions described above. + Only 'yes' will be accepted to approve. + + Enter a value: yes + + aws_instance.myVyOSec2: Creating... + aws_instance.myVyOSec2: Still creating... [10s elapsed] + aws_instance.myVyOSec2: Still creating... [20s elapsed] + aws_instance.myVyOSec2: Still creating... [30s elapsed] + aws_instance.myVyOSec2: Still creating... [40s elapsed] + aws_instance.myVyOSec2: Creation complete after 44s [id=i-09edfca15aac2fe0a] + null_resource.SSHconnection1: Creating... + null_resource.SSHconnection2: Creating... + null_resource.SSHconnection1: Provisioning with 'file'... + null_resource.SSHconnection2: Provisioning with 'remote-exec'... + null_resource.SSHconnection2 (remote-exec): Connecting to remote host via SSH... + null_resource.SSHconnection2 (remote-exec): Host: 10.217.80.104 + null_resource.SSHconnection2 (remote-exec): User: root + null_resource.SSHconnection2 (remote-exec): Password: true + null_resource.SSHconnection2 (remote-exec): Private key: false + null_resource.SSHconnection2 (remote-exec): Certificate: false + null_resource.SSHconnection2 (remote-exec): SSH Agent: false + null_resource.SSHconnection2 (remote-exec): Checking Host Key: false + null_resource.SSHconnection2 (remote-exec): Target Platform: unix + local_file.ip: Creating... + local_file.ip: Creation complete after 0s [id=e8e91f2e24579cd28b92e2d152c0c24c3bf4b52c] + null_resource.SSHconnection2 (remote-exec): Connected! + null_resource.SSHconnection1: Creation complete after 0s [id=7070868940858935600] + + null_resource.SSHconnection2 (remote-exec): PLAY [integration of terraform and ansible] ************************************ + + null_resource.SSHconnection2 (remote-exec): TASK [Wait 300 seconds, but only start checking after 60 seconds] ************** + null_resource.SSHconnection2: Still creating... [10s elapsed] + null_resource.SSHconnection2: Still creating... [20s elapsed] + null_resource.SSHconnection2: Still creating... [30s elapsed] + null_resource.SSHconnection2: Still creating... [40s elapsed] + null_resource.SSHconnection2: Still creating... [50s elapsed] + null_resource.SSHconnection2: Still creating... [1m0s elapsed] + null_resource.SSHconnection2 (remote-exec): ok: [54.144.84.120] + + null_resource.SSHconnection2 (remote-exec): TASK [Configure general settings for the vyos hosts group] ********************* + null_resource.SSHconnection2: Still creating... [1m10s elapsed] + null_resource.SSHconnection2 (remote-exec): changed: [54.144.84.120] + + null_resource.SSHconnection2 (remote-exec): PLAY RECAP ********************************************************************* + null_resource.SSHconnection2 (remote-exec): 54.144.84.120 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 + + null_resource.SSHconnection2: Creation complete after 1m16s [id=4902256962410024771] + + Apply complete! Resources: 4 added, 0 changed, 0 destroyed. + + Outputs: + + my_IP = "54.144.84.120" + + + +After executing all the commands you will have your VyOS instance on the AWS cloud with your configuration, it's a very convenient desition. +If you need to delete the instance please type the command: + +.. code-block:: none + + #terraform destroy + + +Troubleshooting +--------------- + +1. Ansible doesn't connect via SSH to your AWS instance: you have to check that your SSH key has copied into the path /root/aws/. +Also, increase the time in the file instance.yml from 300 sec to 500 sec or more. (It depends on your location). +Make sure that you have opened access to the instance in the security group. + +2. Terraform doesn't connect via SSH to your Ansible instance: you have to check the correct login and password in the part of the file vyos. tf + +.. code-block:: none + + connection { + type = "ssh" + user = "root" # open root access using login and password on your Ansible + password = var.password # check password in the file terraform.tfvars isn't empty + host = var.host # check the correct IP address of your Ansible host + } + + +Make sure that Ansible is pinging from Terrafom. + +Structure of files Terrafom for AWS +----------------------------------- + +.. code-block:: none + + . + ├── vyos.tf # The main script + ├── var.tf # File for the changing version of Terraform. + ├── versions.tf # The file of all variables in "vyos.tf" + └── terraform.tfvars # The value of all variables (passwords, login, ip adresses and so on) + + + +File contents of Terrafom for AWS +--------------------------------- + +vyos.tf + +.. code-block:: none + + + ############################################################################## + # Build an VyOS VM from the Marketplace + # To finde nessesery AMI image_ in AWS + # + # In the script vyos.tf we'll use default values (you can chang it as you need) + # AWS Region = "us-east-1" + # AMI = "standard AMI of VyOS from AWS Marketplace" + # Size of VM = "t2.micro" + # AWS Region = "us-east-1" + # After deploying the AWS instance and getting an IP address, the IP address is copied into the file + #"ip.txt" and copied to the Ansible node for provisioning. + ############################################################################## + + provider "aws" { + access_key = var.access + secret_key = var.secret + region = var.region + } + + variable "region" { + default = "us-east-1" + description = "AWS Region" + } + + variable "ami" { + default = "ami-**************3b3" # ami image please enter your details + description = "Amazon Machine Image ID for VyOS" + } + + variable "type" { + default = "t2.micro" + description = "Size of VM" + } + + # my resource for VyOS + + resource "aws_instance" "myVyOSec2" { + ami = var.ami + key_name = "awsterraform" # Please enter your details from 1.2 of Preparation steps for deploying VyOS on AWS + security_groups = ["awsterraformsg"] # Please enter your details from 1.3 of Preparation steps for deploying VyOS on AWS + instance_type = var.type + tags = { + name = "VyOS System" + } + } + + ############################################################################## + # specific variable (to getting type "terraform plan"): + # aws_instance.myVyOSec2.public_ip - the information about public IP address + # of our instance, needs for provisioning and ssh connection from Ansible + ############################################################################## + + output "my_IP"{ + value = aws_instance.myVyOSec2.public_ip + } + + ############################################################################## + # + # IP of aws instance copied to a file ip.txt in local system Terraform + # ip.txt looks like: + # cat ./ip.txt + # ххх.ххх.ххх.ххх + ############################################################################## + + resource "local_file" "ip" { + content = aws_instance.myVyOSec2.public_ip + filename = "ip.txt" + } + + #connecting to the Ansible control node using SSH connection + + ############################################################################## + # Steps "SSHconnection1" and "SSHconnection2" need to get file ip.txt from the terraform node and start remotely the playbook of Ansible. + ############################################################################## + + resource "null_resource" "SSHconnection1" { + depends_on = [aws_instance.myVyOSec2] + connection { + type = "ssh" + user = "root" + password = var.password + host = var.host + } + + #copying the ip.txt file to the Ansible control node from local system + + provisioner "file" { + source = "ip.txt" + destination = "/root/aws/ip.txt" # The folder of your Ansible project + } + } + + resource "null_resource" "SSHconnection2" { + depends_on = [aws_instance.myVyOSec2] + connection { + type = "ssh" + user = "root" + password = var.password + host = var.host + } + #command to run Ansible playbook on remote Linux OS + provisioner "remote-exec" { + inline = [ + "cd /root/aws/", + "ansible-playbook instance.yml" # more detailed in "File contents of Ansible for AWS" + ] + } + } + + +var.tf + +.. code-block:: none + + variable "password" { + description = "pass for Ansible" + type = string + sensitive = true + } + variable "host"{ + description = "The IP of my Ansible" + type = string + } + variable "access" { + description = "my access_key for AWS" + type = string + sensitive = true + } + variable "secret" { + description = "my secret_key for AWS" + type = string + sensitive = true + } + +versions.tf + +.. code-block:: none + + terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 5.0" + } + } + } + +terraform.tfvars + +.. code-block:: none + + password = "" # password for Ansible SSH + host = "" # IP of my Ansible + access = "" # access_key for AWS + secret = "" # secret_key for AWS + + +Structure of files Ansible for AWS +---------------------------------- + +.. code-block:: none + + . + ├── group_vars + └── all + ├── ansible.cfg + ├── mykey.pem + └── instance.yml + + +File contents of Ansible for AWS +-------------------------------- + +ansible.cfg + +.. code-block:: none + + [defaults] + inventory = /root/aws/ip.txt + host_key_checking= False + private_key_file = /root/aws/awsterraform.pem # check the name + remote_user=vyos + +mykey.pem + +.. code-block:: none + + Copy your key.pem from AWS + + +instance.yml + + + +.. code-block:: none + + ############################################################################## + # About tasks: + # "Wait 300 seconds, but only start checking after 60 seconds" - try to make ssh connection every 60 seconds until 300 seconds + # "Configure general settings for the vyos hosts group" - make provisioning into AWS VyOS node + # You have to add all necessary cammans of VyOS under the block "lines:" + ############################################################################## + + + - name: integration of terraform and ansible + hosts: all + gather_facts: 'no' + + tasks: + + - name: "Wait 300 seconds, but only start checking after 60 seconds" + wait_for_connection: + delay: 60 + timeout: 300 + + - name: "Configure general settings for the vyos hosts group" + vyos_config: + lines: + - set system name-server 8.8.8.8 + save: + true + + +all + +.. code-block:: none + + ansible_connection: ansible.netcommon.network_cli + ansible_network_os: vyos.vyos.vyos + ansible_user: vyos + +Sourse files for AWS from GIT +----------------------------- + +All files about the article can be found here_ + + +.. _link: https://developer.hashicorp.com/terraform/intro +.. _install: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli +.. _pair: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html +.. _group: https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-sg.html +.. _image: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html +.. _here: https://github.com/vyos/vyos-automation/tree/main/TerraformCloud/AWS_terraform_ansible_single_vyos_instance-main
\ No newline at end of file |