Run Ansible playbook on AWS target with SSM System manager

AWS configuration

SSM State manager : Association Parameters

 documentParameters with an archive (zip) containing multiple yml files

{ "InstallDependencies":"False", "SourceType":"S3", "SourceInfo":"{\"path\":\"https://name_of_bucket_hosting_sources.s3-eu-west-42.amazonaws.com/prefix_key/archive.zip\"}", "PlaybookFile":"main.yml" }

 documentParameters with only one yml files

{ "InstallDependencies":"False", "SourceType":"S3", "SourceInfo":"{\"path\":\"https://name_of_bucket_hosting_sources.s3-eu-west-42.amazonaws.com/prefix_key/playbook.yml\"}", "PlaybookFile":"playbook.yml" }

sourceInfo

{ "name": "AWS-ApplyAnsiblePlaybooks" }

* Association Target

Depending on where you want to run the playbook, select what's appropriate

Ansible playbook 

example : daily export from an EC2 instance directory to an s3 bucket

- hosts: localhost
  become: yes
  become_method: sudo
  tasks:
  - name: Find zips in /path/to/data/*.zip older than 7d
    find:
      paths: /path/to/data/
      patterns: '*.zip'
      age: 7d
    register: files_to_delete
  - name: Remove zips in /path/to/data/ older than 7d
    file:
      path: "{{ item.path }}"
      state: absent
    with_items: "{{ files_to_delete.files }}"

  - name: Upload content of /path/to/data/ directory, ommiting structure-*.zip files
    community.aws.s3_sync:
      bucket: target-s3-share-name
      key_prefix: s3-prefix-dir-name/subdirectory/
      file_root: /path/to/data/
      include: "*.zip"
      exclude: "structure-*.zip"
      delete: no # if set to yes, removes remote files that exist in bucket but are not present in the file root.
  - name: Upload content of /path/to/data/ directory
    community.aws.s3_sync:
      bucket: target-s3-share-name
      key_prefix: s3-prefix-dir-name/subdirectory/
      file_root: /path/to/data/
      include: "*"

References :

• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-association.html
• https://aws.amazon.com/blogs/mt/running-ansible-playbooks-using-ec2-systems-manager-run-command-and-state-manager/
• https://docs.aws.amazon.com/systems-manager/latest/userguide/integrations-partners.html

Rsync and exclusions

 

an old post found back on disqr about rsync (original date : circa 2016)

You might want to use the rsync exclude options directly in your script either by specifying a file with all the exclusions to perform, or by specifying them in the command line directly :

--exclude-from <file-name with 1 pattern by line>
--exclude <file or="" dir="">

For example :

rsync -r -a -v -e "ssh -l Username" \

        --exclude 'dbconfig.xml' --exclude 'WEB-INF/classes/' --delete \

        /local/Directory remote.server:/remoteDirectory

One important thing to keep in mind when excluding a directory is that rsync will always consider the path to be relative to the source directory.

This can be used for example when you want to push your production from a Production JIRA instance to a Staging JIRA instance, but your dbconfig.xml is different (different DB auth parameters for example), and hence want to avoid some files.