mardi 1 décembre 2020

ansible FQCN for modules names > 2.10

Starting in ansible 2.10, ansible now recommends using the FQCN for each module (Fully Qualified Collection Name)

This might become mandatory in a future release.

To identify the redirection from the default / previous/ still working to the FQCN :

# ansible-playbook deploy.yaml -vv
redirecting (type:module) ansible.builtin.timezone to community.general.timezone

namely, this is an example for the helm (example from Montreal Ansible meetup on 30-sept-2020)

mercredi 25 novembre 2020

OpsGenie : AWS SNS message to Jira Description Wiki markup (+ links to S3 logs and SSM output)

If you're using the AWS SNS opsgenie integration and want to publish to JIRA, you can for example use the following code to present the data in a slightly better way :

I this use-case I'm using the SNS channel to publish outputs from a system manager (AWS SSM) command that also publishes it's outputs to an S3, so we're using this extraction to provide the direct links to the s3 logs and the SSM run command history.

And in the end, we copy the message we received from the SNS channel "raw"..

In Opsgenie, in the specific Amazon SNS integration (Incoming Amazon SNS), in the Alert Fields, you can for example modify the "Description" so that it transforms the Message received like this :

h3. Details
|| AWS region | {{ TopicArn.extract(/arn:aws:sns:([^:]*):.*/) }} |
|| Status | {{ Message.extract(/.*"status":"([^"]*)".*/) }} |
|| Instance ID | {{ Message.extract(/.*"instanceId":"([^"]*)".*/) }} [(aws link)|https://{{ TopicArn.extract(/arn:aws:sns:([^:]*):.*/) }}.console.aws.amazon.com/ec2/v2/home?region={{ TopicArn.extract(/arn:aws:sns:([^:]*):.*/) }}#InstanceDetails:instanceId={{ Message.extract(/.*"instanceId":"([^"]*)".*/) }}]|
|| Command ID | {{ Message.extract(/.*"commandId":"([^"]*)".*/) }} [(aws cmd)|https://console.aws.amazon.com/systems-manager/run-command/{{ Message.extract(/.*"commandId":"([^"]*)".*/) }}] [(s3 logs)|https://console.aws.amazon.com/s3/buckets/ssm-output/ssm-log/{{ Message.extract(/.*"commandId":"([^"]*)".*/) }}/{{ Message.extract(/.*"instanceId":"([^"]*)".*/) }}/?region={{ TopicArn.extract(/arn:aws:sns:([^:]*):.*/) }}&showversions=false ]
|
|| documentName | {{ Message.extract(/.*"documentName":"([^"]*)".*/) }} |
|| requestedDateTime | {{ Message.extract(/.*"requestedDateTime":"([^"]*)".*/) }} |
|| eventTime | {{ Message.extract(/.*"eventTime":"([^"]*)".*/) }} |
h3. Opsgenie info
|| EventType | {{eventType}} |
|| Timestamp (opsgenie) | {{Timestamp}}|
|| Tags | {{tags}} |
|| TopicArn | {{TopicArn}} |
|| Actions | {{actions}} |
h3. Original Message (raw):
{code}
{{Message}}
{code}

Nb: this might only be available in certain OpsGenie subscriptions unfortunately :-(

mardi 3 novembre 2020

Gitops & git tracking modifications & Nagios

Context :

I needed to have all my nagios files on my computer to be able to run some python scripts trying to figure out what refactoring needed to be done, and identifying gaps in the configuration I inherited.

I took that opportunity to version all our nagios configuration files, with a git repository configured at the /usr/local/nagios/etc level.

That proved itself useful to gain some confidence that we're not going to loose anything.

Initial idea was taken from :

Script to automatically commit changes done in Nagios and push them to the central repo.
NB : still a few things to investigate, but ...

auto-git-commit-push.sh

#!/bin/bash
cd /usr/local/nagios/etc \
&& /bin/git pull \
&& /bin/git add -A \
&& /bin/git commit -m "updated nagios dynamic files $(date) -- automatic commit" \
&& /bin/git push origin master \
&& /bin/git pull \
&& if ! $(grep -lr '<<<<<<<' . ) ; then grep -lr '<<<<<<<' . | xargs git checkout --ours; ./$0; fi

identified issues :

* why so many conflicts while we are only modifying and thus commiting files locally and pushing them !?

* I thought about putting in place a workflow that would take the files from the git repository and push them to nagios import with some git hooks, but I eventually found the Nagios REST API which is way easier to delivery to the team handling those creations : just 1 script per use-case and they'll do everything at once.

lundi 26 octobre 2020

make some internet noise / anti-recommendation algorithms tools

http://makeinternetnoise.com/ : opens random webpages every 5 sec. This creates some artificial clutter in your web history => recommendation algorithms are lost.

(source : https://getpocket.com/explore/item/how-to-disappear )

Com' - Empathetic Communication (Atlassian skillbuilder / accreditation ressource)

empathy vs sympathy : https://www.youtube.com/watch?v=1Evwgu369Jw

https://www.amazon.com/Leadership-Step-Become-Person-Others/dp/0814437931

https://www.amazon.com/Your-Brain-Work-Strategies-Distraction/dp/0061771295

https://www.amazon.com/Business-Communication-Strategies-Skills-management/dp/003033067X

https://www.amazon.com/Dark-Side-Interpersonal-Communication/dp/080585780X

https://www.amazon.com/Leadership-Step-Become-Person-Others/dp/0814437931

lundi 12 octobre 2020

Mac Os X - mount issues on /Users after upgrade to mac os X 10.15 / error 77

fixing /Users defined partition after an upgrade of mac os X.

The Mac I was using was configured before I got it, and I needed to upgrade it to use some more recent applications, but I ran into some issues due to a specific configuration of the /Users partition being located on a different partition than the OS data.

I could not mount it properly, due to 2 errors (might have been possible to mount it directly in HFS, but I also fixed it during the debug, just publishing my fix here with both steps).

Error :

sudo mount -uw /Users

Password:

mount_apfs: volume could not be mounted: Operation not permitted

mount: /Users failed with 77

Step 1 : convert from HFS to APFS

with Disk Utility tool > menu "Edit" > "Convert to APFS"...

Step 2 : edit mount description

/etc/fstab, edited with (sudo) vifs

was :

UUID=12QW345E-1234-5678-123QWE123QWE /Users hfs auto

changed to :

UUID=12QW345E-1234-5678-123QWE123QWE /Users apfs rw 1 2

Step 3 : grant mount on /Users

Boot in recovery mode ( mac + R )

Launch (menu Utilities) > Terminal

csrutil status
csrutil enable --without fs

reboot

lundi 28 septembre 2020

Canadian Permanent Residency - general notes on the Express Entry program

Started in 2019 and obtained in 2020, I describe here a few general considerations on how I organized myself to get my Canadian permanent residency work permit.

I need to thanks all the friends who helped me in the process, in particular P.W. and G.L. for their numerous advices and the time they spend answering my never ending questions :-)

There are various immigration path, this present the step by step process of how I applied to the Express Entry program. I think that most of the considerations on how to present the documents, how to organize yourself should be consistent among the other existing path, but I haven't tested them.

This document reflects my point of view and shall not be taken as legal or immigration advices, please refer to the appropriate sources of data for consistent and up to date information.

This is how this document is organized :

Comprehensive Ranking System
Know your CRS points
Know the current CRS "cut-off"
Organize yourself / project management
Task organisation (how I did it with Trello)
Consideration on document format and signatures
Jobs
NOC NOC what am I even doing ?
Job offer
Proof past jobs
Proof of job experience -- template
Planning : a good timing is important
Timing considerations

Comprehensive Ranking System

Know your CRS points (Comprehensive Ranking System)

Use the online calculation tool to evaluate the number of points you can claim.

You can play with the tool (and you will have to until you get the first results... for example the language tests !) to evaluate how many points you can claim. This will allow you to understand if you are eligible at all.

CRS / SGC calculation tool, the interactive version :

https://www.cic.gc.ca/english/immigrate/skilled/crs-tool.asp (English)
https://www.cic.gc.ca/francais/immigrer/qualifie/scg-outil.asp (Français)

CRS / SGC same data without the calculation tool good to understand the repartition of points :

When you'll actually send your file, don't over estimate yourself : you'll need to prove each and every points you claimed, but don't under estimate yourself either.
As you'll notice the CRS calculation tool doesn't need any registration so you can play with it to evaluate different hypothesis (you'll find other ressources online with advices on how to enhance your score, I didn't need it but it usually varies between completing some Canadian experience either working or studying, finishing and validating a degree, enhancing your language skills, getting a job offer letter...)

Note on Language skills

You'll need to prove your language levels by passing some recognized tests. French brings a few more points than English as you can see playing with the CRS calculation tools.
You should pass your exam before sending your file since you'll need the actual results to prove the claimed points, but for the initial CRS calculations just start by assessing yourself what you think your level is worth in either language.

Know the current CRS "cut-off"

There are various pages with historical data, follow the newsletters, and keep an eye on the current tendency. This is important to understand if you qualify for the express entry programs and what are your chances to be invited to submit a demand.

With those 2 official resources you'll be able to compare the number of points you can claim with the others express entry candidates, to figure out what your chances are...

Official CRS cut-off : https://www.canada.ca/en/immigration-refugees-citizenship/services/immigrate-canada/express-entry/submit-profile/rounds-invitations.html
CRS score distribution of candidates in the Express Entry pool : https://www.canada.ca/en/immigration-refugees-citizenship/services/immigrate-canada/express-entry/submit-profile/rounds-invitations.html

I also registered to a few newsletters and was following a few websites to understand how this was working (I'm not sponsored for those links, and feel free to comment with other interesting ones...). It's one of the ways to keep up with the cut-off every 2 weeks.

Organize yourself

Task organisation (how I did it with Trello)

Personally, I used a Trello board with the following columns, since I'm familiar with the tool.

Todo
Waiting External
Added to the File
Done !
(not applicable)

That allowed me to track all the independent documents I had to gather, and all the different processes that I needed to deal with.

Consideration on document, document formats, and handwritten signatures

I was able to process everything through numeric format, and I didn't need to carry the original when doing the flagpole turn to get the actual card. Prefer PDF for any document you'll upload.

All signed document should be signed by hand and scanned after. Avoid signatures from a touchpad of pasted images as they are apparently refused !

Make sure to give a recognizable name to your file (just for the proof of employments I had to submit around 40 files, and that doesn't count the revisions for the employers certificates that were missing data).

I'm not sure how long you should keep the original paper version, but all the different files reached around 1.5 kg of paper.

More on what to write, format and send to request the job references documents in the next section.

Jobs

NOC NOC what am I even doing ? - Job reference

Canada has a precise description of most existing jobs on the market called National Occupational Classification (NOC) or Classification nationale des professions (CNP) in French. This is what all of your past, current and future (in case of a job offer) work experience will be compared to. Get familiar with it and try to look for the NOC reference the closer possible to your jobs.

As explained later, in most cases your employers won't even know the existence of this classification, and the Canadian immigration doesn't know your job, but will never the less try to match it with this classification to see where you fit regarding Canada immigration policies.

Job offer

If you required a Job offer for the CRS number of points, make sure to validate the requirements :

you might need an LMIA (Labor Market Impact Assessment), in this case it's a bit more tricky: it takes time and requires your employer to pay some fees, do some administrative tasks properly and be willing to wait for your work-permit. Most of them will rather go for a closed work permit in this case. Also, note that employers in Canada are used to have a higher turnover than in France (it's a suggested 2 weeks resignation notice for example; they're usually filling the gaps ASAP, so waiting a few months to a year is not always
so if you claim points with a job offer, try to find a "LMIA exempt" case.

Links :

Job offer format : https://www.cic.gc.ca/english/helpcentre/answer.asp?qnum=695&top=29
LMIA exemptions : https://www.canada.ca/en/immigration-refugees-citizenship/services/immigrate-canada/express-entry/documents/offer-employment/lmia-exempt.html

Proof of past jobs

The immigration agents will evaluate your job history, but they do not know your jobs. They will try to compare your job certificates with the NOC descriptions, that's the administrative tool they've got on hand.
Your past/current employers do not know the NOC (usually) but know what you've been doing, for how long, and how much you were paid, so of course, they can validate this and sign some certificates about this, but they'll most likely have other most urgent task than writing this => do you work and ask them to validate what you need on the proof of employement.
=> You are the one who understands both your jobs at the time, the NOC and the immigration process.

Advice : You should write to your former and current employers with precise data of what you want to see on the employment certificate you are requesting from them, trying to keep the job descriptions and tasks as close as possible to the NOC descriptions.

for Canadian employers it even makes sense to write the NOC reference explicitly
for foreign employers, it does not makes sense to write the NOC ref, but you can nevertheless still describe your jobs duties to be as close as possible to the NOC tasks descriptions you found and are claiming points for !
one NOC reference per job. That's it. Even if it would matches 2 of them. Pick the closest match.

Send exactly what you need on the documents to your employers. It'll be their responsibility to check before they sign of course, but you'll save a lot of time in the process.

Another advantage of sending precisely what you need on your documents to all your past employers is that you'll end up with the same format for each document, which will greatly help the immigration agent dealing with your files. Following some advices I received, I sent to each of my past and current employers an e-mail stating exactly what I was expecting to see, all in the same format. The could hence verify the details, and send me back a proper scan + paper version of the document with their paper heading and details, but they mostly had to validate the data and do a copy-paste.

example in French of employment attestation.
This was actually a copy-paste from a previous e-mail.

Proof of job experience / template

Scan all your pay-slips, contracts, contract modifications, etc. those also are proof of employment (the more, the merrier).
Send a template already filled with what you would like to see on the certificate to your past employers : As a minimum, you should include in your template the following informations and requests :
- This certificate should be written on an official letterhead paper of your employer.
- Precise start date, precise end date (or stating that this is your current employer).
- Number of hours worked per week.
- Positions occupied, and precise start date and end date for each position.
- Annual salary, including benefits if any.
- Employer contact
- Official HANDWRITTEN signature of your direct manager, or the official HR person in charge, with their name written clearly in capital letters. Let me insist on the handwritten : Canadian immigration do not like seeing a numerically copy-pasted signature, even if you are primarily sending your documents in PDF make sure to receive the paper version for your archives (just in case), and if in a hurry ask for the scanned PDF version right away.
- Date of the signature

For each employer I submitted the following proofs :

Up to date certificate (with all the details mentioned before)
ALL my payslips scanned
ALL my contracts scanned, including revisions

Planning : a good timing is important

Understand that if you claim a certain amount of points above the cut-off, you'll be invited soon to present your file to prove all your points.

This gave me 60 days (in 2019), which can be short for some procedures, so START BEFORE ! (note : you can apparently extend this period just by requesting it, but I didn't have to)

Plan in advance, I was luckily suggested to start those before applying :

Language exam (takes ~ 2.5 months): finding the proper exam took 1h, booking it was online in the city where I was living at the time, and the closest date was ... 1 month later ! it then took ~ 1.5 month to get the result

Make sure to select a Language exam that matches the Canadian Immigration criteria
Passing 2 different languages will not give you much points as you'll see in the CRS calculation link.
French test gives a few more points than English (7 if I recall correctly).

List all your residencies adresses
List all your travels
List all your education
List all jobs / employers and prepare to reach them : make sure you've got the proper contacts up to date, prepare the requests you'll send them... (More on this topic on the previous section... )

Note Date format : it's helpful to get all the dates in YYYY-MM-DD format, since this is how you'll have to enter it.

Note Avoid gaps (not even a single day): For each of the lists with dates (residencies, educations, employers, travels), avoid gaps (if you finish on 2020-sept-13, the next should start on 2020-sept-14).

Note Since 18th birthday: For each of the lists with dates (residencies, educations, employers, travels), even if you're more than 28 and can theoretically declare only the last 10 years, but you can/should gather and declare everything since you were 18.

Note Travels: For your residencies and travels, if you've been living in any country for more than 6 months you'll need to provide a police certificate issued from this country. France provides it in a few seconds (demande d'extrait de casier judiciaire), but I heard that it could be quite longer is some countries.

Education assessment (2 to 6 months) : this depends on your institution(s), my university didn't sent the proper files so I had to do a back and forth that cost me a few extra months.

You need to do an education assessments for the diplomas if you are claiming some points. I used "WES", which at the time seemed the be the fastest one, other options are listed on the Canadian immigration website :
https://www.canada.ca/en/immigration-refugees-citizenship/services/immigrate-canada/express-entry/documents/education-assessed/how.html

each

past

current

National Occupational Classification

NOC ( https://noc.esdc.gc.ca/ )

Timing considerations

It took me around 1 year from the day when I decided to apply for my permanent residency through the express entry program and the day when I eventually received the PR card.

In parallel, before formerly applying :

2.5 months for the Language Exam
6 months for the Education Assessment
some time to prepare the employment proofs certificates
some time to prepare all the data for the residencies, education, travels (!), etc.

< 1 week between my application and being drawn from the PR pool
~ 40 days to gather all my employers certificates, pass the medical exam and put all the data on the website.
~ 4 months to get the "invitation to become PR". This is the invitation to present at the immigration agents to actually become PR. If you're already in Canada you'll need to exit the country and re-enter, aka. flagpole turn. You will need a safe mailing address in Canada to receive your car afterward.
~ 1 month to get the PR card by mail

PS : Make sure to keep all this data in your records since you'll need most of it to apply for citizenship 2 to 3 years later.

jeudi 3 septembre 2020

Terraform support channels + best practices

This email tf-cloud@hashicorp.support provides support with Terraform Cloud related issues only.

For Terraform OSS related questions, use cases and best practices, please check the Terraform community for assistance: https://www.terraform.io/community.html

Our support engineers frequent the Terraform community forum and would be able to share some guidance there: https://discuss.hashicorp.com/c/terraform-core

(from a question sent to hashicorp zendesk )

lundi 31 août 2020

Rsync and exclusions

an old post found back on disqr about rsync (original date : circa 2016)

You might want to use the rsync exclude options directly in your script either by specifying a file with all the exclusions to perform, or by specifying them in the command line directly :

--exclude-from <file-name with 1 pattern by line>
--exclude <file or="" dir="">

For example :

rsync -r -a -v -e "ssh -l Username" \

--exclude 'dbconfig.xml' --exclude 'WEB-INF/classes/' --delete \

/local/Directory remote.server:/remoteDirectory

One important thing to keep in mind when excluding a directory is that rsync will always consider the path to be relative to the source directory.

This can be used for example when you want to push your production from a Production JIRA instance to a Staging JIRA instance, but your dbconfig.xml is different (different DB auth parameters for example), and hence want to avoid some files.

Commandes en Vrac