MySQL to CSV file

SELECT order_id,product_name,qty
FROM orders
WHERE foo = 'bar'
INTO OUTFILE '/tmp/orders.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n';

TODO : There is still to find how to print the column titles along with the result...

Source : http://stackoverflow.com/questions/356578/how-to-output-mysql-query-results-in-csv-format

ref : http://www.tech-recipes.com/rx/1475/save-mysql-query-results-into-a-text-or-csv-file/

VirtualBox install / config de base

 * Config Clavier Mac sous Debian :
https://lokan.fr/2013/12/11/configurer-correctement-son-clavier-mac-sous-debian/
XKBMODEL="pc105"
XKBLAYOUT="fr"
XKBVARIANT="mac"
XKBOPTIONS="lv3:switch,compose:lwin”
* Config redimensionner fenêtre VirtualBox
(non testé ) : https://forums.virtualbox.org/viewtopic.php?f=2&t=68966
* http://download.virtualbox.org/virtualbox/5.0.14/
* http://download.virtualbox.org/virtualbox/5.0.14/UserManual.pdf
* Be sure to have the kernel headers, gcc and other basic build-tools installed
apt-cache search/apt-get install build-essential linux-headers-$(uname -r) dkms

* If present, it should be in /media/cdrom (or /media/cdrom0)
cd /media/cdrom0
sh VBoxLinuxAdditions.run

* NB : you might need to reboot the host.
* You can now add some shared folders (for example). They will be owned by root with drwxrwx--- root vboxsf (group)
* Add user in group vboxsf :
sudo usermod cm -a -G vboxsf

Monitoring : POC around Monit + M/Monit

Monit + M/Monit

OpenSouce, on bitbucket. https://bitbucket.org/tildeslash/monit/

First commit date 2014-01-23: https://bitbucket.org/tildeslash/monit/commits/branch/master?page=28

Monit : "Agent" or "Slave", running on each server where monit his used.
https://mmonit.com/monit/

M/Monit : "Master" allowing to connect, get and coordinate events and actions to&from all monit agents connected.
https://www.mmonit.com/



mmonit manual :
https://mmonit.com/documentation/mmonit_manual.pdf
https://mmonit.com/wiki/Monit/ConfigurationExamples



idea 1 : how to enhance this project : contribute a "log snippet" =
along side with the "start/stop program" in the config file, add a "logfile path" configuration setup that would watch this file(s) and make it available to the agent, and then to the master.

idea 2 : interface monit & elasticsearch (or implement monit within elasticsearch ?)





-----
Other monitoring tools :

* Prometheus "

An open-source service monitoring system and time series database."

http://prometheus.io/docs/introduction/getting_started/
 https://github.com/prometheus/prometheus


 * Sensu : A monitoring framework that aims to be simple, malleable, and scalable
https://sensuapp.org/
https://github.com/sensu/sensu

 * Ganglia
 http://ganglia.info/

Scripting : set variables depending how the where launched (terminal mode, cron/auto mode, ...)

You all know the deal : when you launch a command it (obviously) comes with all your environment variables. But when you want to cron it, none of them is present. We hence want to set some variables, but only in some cases. The "tty" command will help us do so.


You can use the tty tool to check if a script is called from the standard output :


if ! tty -s
then
    exec >/dev/null 2>&1
else
    MAIL_DEST="email@server.ext"
fi

• The command tty returns :

User Commands                                              tty(1)

NAME     tty - return user's terminal name

SYNOPSIS     tty [-l] [-s]

DESCRIPTION     The tty utility writes to the standard output  the  name  of
     the  terminal  that is open as standard input. The name that
     is used is equivalent to the string that would  be  returned
     by the ttyname(3C) function.

OPTIONS     The following options are supported:

     -l       Prints the synchronous line  number  to  which  the
              user's terminal is connected, if it is on an active
              synchronous line.

     -s       Inhibits printing of the terminal path name, allow-
              ing one to test just the exit status.

EXIT STATUS
     The following exit values are returned:

     0        Standard input is a terminal.

     1        Standard input is not a terminal.

     >1       An error occurred.
 

MS Word shortcuts

Sometimes you're bound to use it for professional purposes...

3 shortcuts on MS-Word :

• Shift + F3 : change the current word type case between ALLUPPER <-> alllower <-> Firstcharup.

• Shift + F4 / F5 : navigate 1 screen forward (f4) / backward (f5.

• Control + Y : repeat the last action performed.

Gnuplot by line (instead of columns)

In case you have some data not sorted by column (the easiet way to deal with gnuplot), and you want to plot it


Gnuplot definition: (file myplotrow.gpl)

set terminal x11 1 noraise
set autoscale
set grid
set xdata time
set timefmt "%Y-%m-%d_%H:%M:%S"
set format x "%m/%d\n%H:%M"
plot "data.dat" \
        using 1:(stringcolumn(2) eq "valAAA"? $3:1/0) title "A" lc rgb "blue",\
     "" using 1:(stringcolumn(2) eq "valBBB"? $3:1/0) title "B" lc rgb "red",\
     "" using 1:(stringcolumn(2) eq "valCCC"? $3:1/0) title "C" lc rgb "green",\
     "" using 1:(stringcolumn(2) eq "valDDD"? $3:1/0) title "D" lc rgb "orange",\
     "" using 1:(stringcolumn(2) eq "valEEE"? $3:1/0) title "E" lc rgb "purple",\
     "" using 1:(stringcolumn(2) eq "valFFF"? $3:1/0) title "F" lc rgb "violet"

pause 100
reread

That you can launch with :

gnuplot < myplotrow.gpl

Or, if you change the output in the first line :

set terminal postscript color

with :

gnuplot < myplotrow.gpl > myoutput.ps

Exemple values :

2015-09-24_10:00:00 valAAA 37.000
2015-09-24_10:00:00 valBBB 37.000
2015-09-24_10:00:00 valCCC 37.000
2015-09-24_10:00:00 valDDD 37.000
2015-09-24_10:00:00 valEEE 37.000
2015-09-24_10:00:00 valFFF 37.000
2015-09-24_13:30:00 valAAA 40.000
2015-09-24_13:30:00 valBBB 40.000
2015-09-24_13:30:00 valCCC 41.000
2015-09-24_13:30:00 valDDD 42.000
2015-09-24_13:30:00 valEEE 42.000
2015-09-24_13:30:00 valFFF 42.000
2015-09-25_02:00:00 valAAA 5.000
2015-09-25_02:00:00 valBBB 5.000
2015-09-25_02:00:00 valCCC 5.000
2015-09-25_02:00:00 valDDD 5.000
2015-09-25_02:00:00 valEEE 5.000
2015-09-25_02:00:00 valFFF 5.000
2015-09-25_03:30:00 valAAA 15.000
2015-09-25_03:30:00 valBBB 15.000
2015-09-25_03:30:00 valCCC 16.000
2015-09-25_03:30:00 valDDD 16.000
2015-09-25_03:30:00 valEEE 15.000
2015-09-25_03:30:00 valFFF 15.000
2015-09-25_05:30:00 valAAA 19.000
2015-09-25_05:30:00 valBBB 19.000
2015-09-25_05:30:00 valCCC 19.000
2015-09-25_05:30:00 valDDD 20.000
2015-09-25_05:30:00 valEEE 19.000
2015-09-25_05:30:00 valFFF 19.000
2015-09-25_06:00:00 valAAA 19.000
2015-09-25_06:00:00 valBBB 19.000
2015-09-25_06:00:00 valCCC 18.000
2015-09-25_06:00:00 valDDD 19.000
2015-09-25_06:00:00 valEEE 19.000
2015-09-25_06:00:00 valFFF 20.000

References :

• Gnuplot terciary condition:

http://stackoverflow.com/questions/6564561/gnuplot-conditional-plotting-plot-col-acol-b-if-col-c-x

• Colors

 http://gnuplot.sourceforge.net/demo/rainbow.html
 

Flatten and XML file (perl)

This perl one-liner is useful in case you need to "flatten" and XML file.

PERL_CONVERT="use XML::Simple; print XMLout(XMLin('-', KeepRoot => 1), AttrIndent => 1, KeepRoot => 1, XMLDecl => '');"

To be used like this :

/usr/bin/perl -e "$PERL_CONVERT" > $TMPFILE

Whatever input you have (in XML), the goal is to have a "comparable" ouptut that will respect the XML structure :
FROM:


...

...

 TO:


attr1="value1"
attr2="value2"
...
attr3="value3"
>

...




...


Specifically, this is really useful if you need to merge 2 different versions of  your XML file, like for example in SVN.

Ansible vs. Chef vs. Puppet vs. Salt

There are currently various tools to maintain automatically an infrastructure. The four listed below seem to be the main ones.

·         Ansible http://docs.ansible.com/intro_installation.html#getting-ansible

Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.

http://docs.ansible.com/intro_installation.html#getting-ansible

·         Chef https://www.chef.io/

“Chef turns infrastructure into code. With Chef, you can automate how you build, deploy, and manage your infrastructure. Your infrastructure becomes as versionable, testable, and repeatable as application code."

·         Puppet https://puppetlabs.com

“Puppet is a configuration management solution that allows you to define the state of your IT infrastructure, and then automatically enforces the desired state. Puppet automates every step of the software delivery process, from provisioning of physical and virtual machines to orchestration and reporting; from early-stage code development through testing, production release and updates.”

·         Salt : http://saltstack.com

“SaltStack takes a new approach to infrastructure management by developing software that is easy enough to get running in seconds, scalable enough to manage tens of thousands of servers, and fast enough to control and communicate with them in milliseconds. SaltStack delivers a dynamic infrastructure communication bus used for orchestration, remote execution, configuration management and much more. The Salt project was launched in 2011 and today is the fastest-growing, most-active infrastructure orchestration and configuration management open source project in the world. The SaltStack community is committed to keeping the Salt project focused, friendly, healthy and open.”

http://salt.readthedocs.org/en/latest/contents.html

And some comparisions : 

• [infoworld] Puppet vs. Chef :  http://www.infoworld.com/article/2614204/data-center/puppet-or-chef--the-configuration-management-dilemma.html
• [infoworld] Review : Puppet vs. Chef vs. Ansible vs. Salt : http://www.infoworld.com/article/2609482/data-center/data-center-review-puppet-vs-chef-vs-ansible-vs-salt.html 
• [infoworld] Review: Ansible orchestration is a veteran Unix admin's dream : http://www.infoworld.com/article/2612397/data-center/review--ansible-orchestration-is-a-veteran-unix-admin-s-dream.html 
• [infoworld] Review: Salt keeps server automation simple : http://www.infoworld.com/article/2612536/data-center/review--salt-keeps-server-automation-simple.html
• Review: Puppet 3.0 pulls more strings http://www.infoworld.com/article/2611099/data-center/review--puppet-3-0-pulls-more-strings.html
  
  
  
  
   

CVSS : Common Vulnerability Scoring System (v3) calculator

As explained in wikipedia:CVSS "Common Vulnerability Scoring System (CVSS) is a free and open industry standard for assessing the severity of computer system security vulnerabilities. It is under the custodianship of the Forum of Incident Response and Security Teams (FIRST). It attempts to establish a measure of how much concern a vulnerability warrants, compared to other vulnerabilities, so efforts can be prioritized. The scores are based on a series of measurements (called metrics) based on expert assessment. The scores range from 0 to 10. Vulnerabilities with a base score in the range 7.0-10.0 are High, those in the range 4.0-6.9 as Medium, and 0-3.9 as Low.^[1]"

•  FIRST announces CVSS version 3 

• Common Vulnerability Scoring System Version 3.0 Calculator (FIRST)
  https://www.first.org/cvss/calculator/3.0
• Common Vulnerability Scoring System Version 2 Calculator (Nvd.Nist)
  https://nvd.nist.gov/CVSS-v2-Calculator

Replace all occurences of a string in a bunch of files by another string (in place)

• Find all files on containing STRING_AAAA and replace it by STRING_BBBB in place (directly in the file).

find . -type f -exec grep -l STRING_AAAA {} \; -exec perl -pi -e 's!STRING_AAAA!STRING_BBBB!g' {} \;

Pre-tests :

• Find all files on containing STRING_AAAA.

find . -type f -exec grep -l STRING_AAAA {} \;