terraform, then ansible

Terraform creates the infra, 

then we want to use ansible to actually configure it...

Different solutions exist to run Terraform, and them ansible:

• Using Terraform Output as Ansible Inventory :
  https://github.com/adammck/terraform-inventory
   $ terraform-inventory -inventory terraform.tfstate [all] 10.10.1.2 10.10.1.3
  
  
  
• `local-exec` / `remote-exec` : terraform runs ansible locally 
  
   ``` provisioner "local-exec" {command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u {var.user} -i '${self.ipv4_address},' --private-key ${var.ssh_private_key} playbook.yml"} ```
  

  key component here is the ${self.ipv4_address} variable. After provisioning the machine, Terraform knows its IP address. And we need to pass an IP address for Ansible.

   (cf https://www.cprime.com/resources/blog/terraform-and-ansible-tutorial-integrating-terraform-managed-instances-with-ansible-control-nodes )

   

• using dynamic inventory and cloud providers specific ansible modules 

•  AWS : https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html 
•  GCP : https://docs.ansible.com/ansible/latest/collections/google/cloud/gcp_compute_inventory.html#ansible-collections-google-cloud-gcp-compute-inventory and so on...

AWS example of the dynamic inventory:

ansible.cfg :

enable_plugins=aws_ec2

aws_ec2.yml (example):

plugin: aws_ec2

regions:

  - "us-east-1"

keyed_groups:

  - key: tags.Name

  - key: tags

    prefix: tag

  - prefix: instance_type

    key: instance_type

  - key: placement.region

    prefix: aws_region

filters:

  instance-state-name : running

  # All instances with their `Environment` tag set to `dev`

  tag:Environment: dev

  # All dev and QA hosts

  tag:Environment:

    - dev

    - qa

compose:

  ansible_host: public_ip_address

Good way to test : 

ansible-inventory -i aws_ec2.yml --graph

ansible all –list-hosts

To run a playbook have/generate 4 files : myplaybook.yml, key.pem, aws_ec2.yml (seen before), and ansible.cfg (seen after)

ansible.cfg

[defaults]
inventory=./aws_ec2.yml

host_key_chekcing=false

remote_user=ec2-user

private_key_file=key.pem

[privilege_escalation]

become=true

become_method=sudo

become_user=root

Run 

ansible <group-name> -i aws_ec2.yaml -m ping --private-key=<private-key-name> 

ansible-playbook myplaybook.yml

 (with hosts: _Ansible_TargetNode or whatever you have validated in the output of the inventory)

# sources : 

https://www.cloudthat.com/resources/blog/step-by-step-guide-to-integrate-ansible-dynamic-inventory-plugin-for-aws-ec2-instances#why-ansible-dynamic-inventory-

https://medium.com/geekculture/a-complete-overview-of-ansible-dynamic-inventory-a9ded104df4c

Ansible resources / links

 

 Jeff Geerling : author of the main ansible book, also author of a lot of important ansible roles and modules. In 2020, he did a serie of youtube weekly stream walking through each chapters of his book.

• https://www.jeffgeerling.com/blog/2020/ansible-101-jeff-geerling-youtube-streaming-series

• https://www.ansible.com/resources/webinars-training
  

• mailing list : https://us19.campaign-archive.com/home/?u=56d874e027110e35dea0e03c1&id=d6635f5420

* https://ara.recordsansible.org/ : ARA Records Ansible and makes it easier to understand and troubleshoot... 

Run Ansible playbook on AWS target with SSM System manager

AWS configuration

SSM State manager : Association Parameters

 documentParameters with an archive (zip) containing multiple yml files

{ "InstallDependencies":"False", "SourceType":"S3", "SourceInfo":"{\"path\":\"https://name_of_bucket_hosting_sources.s3-eu-west-42.amazonaws.com/prefix_key/archive.zip\"}", "PlaybookFile":"main.yml" }

 documentParameters with only one yml files

{ "InstallDependencies":"False", "SourceType":"S3", "SourceInfo":"{\"path\":\"https://name_of_bucket_hosting_sources.s3-eu-west-42.amazonaws.com/prefix_key/playbook.yml\"}", "PlaybookFile":"playbook.yml" }

sourceInfo

{ "name": "AWS-ApplyAnsiblePlaybooks" }

* Association Target

Depending on where you want to run the playbook, select what's appropriate

Ansible playbook 

example : daily export from an EC2 instance directory to an s3 bucket

- hosts: localhost
  become: yes
  become_method: sudo
  tasks:
  - name: Find zips in /path/to/data/*.zip older than 7d
    find:
      paths: /path/to/data/
      patterns: '*.zip'
      age: 7d
    register: files_to_delete
  - name: Remove zips in /path/to/data/ older than 7d
    file:
      path: "{{ item.path }}"
      state: absent
    with_items: "{{ files_to_delete.files }}"

  - name: Upload content of /path/to/data/ directory, ommiting structure-*.zip files
    community.aws.s3_sync:
      bucket: target-s3-share-name
      key_prefix: s3-prefix-dir-name/subdirectory/
      file_root: /path/to/data/
      include: "*.zip"
      exclude: "structure-*.zip"
      delete: no # if set to yes, removes remote files that exist in bucket but are not present in the file root.
  - name: Upload content of /path/to/data/ directory
    community.aws.s3_sync:
      bucket: target-s3-share-name
      key_prefix: s3-prefix-dir-name/subdirectory/
      file_root: /path/to/data/
      include: "*"

References :

• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-association.html
• https://aws.amazon.com/blogs/mt/running-ansible-playbooks-using-ec2-systems-manager-run-command-and-state-manager/
• https://docs.aws.amazon.com/systems-manager/latest/userguide/integrations-partners.html

ansible FQCN for modules names > 2.10

 Starting in ansible 2.10, ansible now recommends using the FQCN for each module (Fully Qualified Collection Name) 

This might become mandatory in a future release.

To identify the redirection from the default / previous/ still working to the FQCN :

# ansible-playbook deploy.yaml -vv
redirecting (type:module) ansible.builtin.timezone to community.general.timezone

namely, this is an example for the helm (example from Montreal Ansible meetup on 30-sept-2020)