Scripting : set variables depending how the where launched (terminal mode, cron/auto mode, ...)

You all know the deal : when you launch a command it (obviously) comes with all your environment variables. But when you want to cron it, none of them is present. We hence want to set some variables, but only in some cases. The "tty" command will help us do so.


You can use the tty tool to check if a script is called from the standard output :


if ! tty -s
then
    exec >/dev/null 2>&1
else
    MAIL_DEST="email@server.ext"
fi

• The command tty returns :

User Commands                                              tty(1)

NAME     tty - return user's terminal name

SYNOPSIS     tty [-l] [-s]

DESCRIPTION     The tty utility writes to the standard output  the  name  of
     the  terminal  that is open as standard input. The name that
     is used is equivalent to the string that would  be  returned
     by the ttyname(3C) function.

OPTIONS     The following options are supported:

     -l       Prints the synchronous line  number  to  which  the
              user's terminal is connected, if it is on an active
              synchronous line.

     -s       Inhibits printing of the terminal path name, allow-
              ing one to test just the exit status.

EXIT STATUS
     The following exit values are returned:

     0        Standard input is a terminal.

     1        Standard input is not a terminal.

     >1       An error occurred.
 

MS Word shortcuts

Sometimes you're bound to use it for professional purposes...

3 shortcuts on MS-Word :

• Shift + F3 : change the current word type case between ALLUPPER <-> alllower <-> Firstcharup.

• Shift + F4 / F5 : navigate 1 screen forward (f4) / backward (f5.

• Control + Y : repeat the last action performed.

Gnuplot by line (instead of columns)

In case you have some data not sorted by column (the easiet way to deal with gnuplot), and you want to plot it


Gnuplot definition: (file myplotrow.gpl)

set terminal x11 1 noraise
set autoscale
set grid
set xdata time
set timefmt "%Y-%m-%d_%H:%M:%S"
set format x "%m/%d\n%H:%M"
plot "data.dat" \
        using 1:(stringcolumn(2) eq "valAAA"? $3:1/0) title "A" lc rgb "blue",\
     "" using 1:(stringcolumn(2) eq "valBBB"? $3:1/0) title "B" lc rgb "red",\
     "" using 1:(stringcolumn(2) eq "valCCC"? $3:1/0) title "C" lc rgb "green",\
     "" using 1:(stringcolumn(2) eq "valDDD"? $3:1/0) title "D" lc rgb "orange",\
     "" using 1:(stringcolumn(2) eq "valEEE"? $3:1/0) title "E" lc rgb "purple",\
     "" using 1:(stringcolumn(2) eq "valFFF"? $3:1/0) title "F" lc rgb "violet"

pause 100
reread

That you can launch with :

gnuplot < myplotrow.gpl

Or, if you change the output in the first line :

set terminal postscript color

with :

gnuplot < myplotrow.gpl > myoutput.ps

Exemple values :

2015-09-24_10:00:00 valAAA 37.000
2015-09-24_10:00:00 valBBB 37.000
2015-09-24_10:00:00 valCCC 37.000
2015-09-24_10:00:00 valDDD 37.000
2015-09-24_10:00:00 valEEE 37.000
2015-09-24_10:00:00 valFFF 37.000
2015-09-24_13:30:00 valAAA 40.000
2015-09-24_13:30:00 valBBB 40.000
2015-09-24_13:30:00 valCCC 41.000
2015-09-24_13:30:00 valDDD 42.000
2015-09-24_13:30:00 valEEE 42.000
2015-09-24_13:30:00 valFFF 42.000
2015-09-25_02:00:00 valAAA 5.000
2015-09-25_02:00:00 valBBB 5.000
2015-09-25_02:00:00 valCCC 5.000
2015-09-25_02:00:00 valDDD 5.000
2015-09-25_02:00:00 valEEE 5.000
2015-09-25_02:00:00 valFFF 5.000
2015-09-25_03:30:00 valAAA 15.000
2015-09-25_03:30:00 valBBB 15.000
2015-09-25_03:30:00 valCCC 16.000
2015-09-25_03:30:00 valDDD 16.000
2015-09-25_03:30:00 valEEE 15.000
2015-09-25_03:30:00 valFFF 15.000
2015-09-25_05:30:00 valAAA 19.000
2015-09-25_05:30:00 valBBB 19.000
2015-09-25_05:30:00 valCCC 19.000
2015-09-25_05:30:00 valDDD 20.000
2015-09-25_05:30:00 valEEE 19.000
2015-09-25_05:30:00 valFFF 19.000
2015-09-25_06:00:00 valAAA 19.000
2015-09-25_06:00:00 valBBB 19.000
2015-09-25_06:00:00 valCCC 18.000
2015-09-25_06:00:00 valDDD 19.000
2015-09-25_06:00:00 valEEE 19.000
2015-09-25_06:00:00 valFFF 20.000

References :

• Gnuplot terciary condition:

http://stackoverflow.com/questions/6564561/gnuplot-conditional-plotting-plot-col-acol-b-if-col-c-x

• Colors

 http://gnuplot.sourceforge.net/demo/rainbow.html
 

Flatten and XML file (perl)

This perl one-liner is useful in case you need to "flatten" and XML file.

PERL_CONVERT="use XML::Simple; print XMLout(XMLin('-', KeepRoot => 1), AttrIndent => 1, KeepRoot => 1, XMLDecl => '');"

To be used like this :

/usr/bin/perl -e "$PERL_CONVERT" > $TMPFILE

Whatever input you have (in XML), the goal is to have a "comparable" ouptut that will respect the XML structure :
FROM:


...

...

 TO:


attr1="value1"
attr2="value2"
...
attr3="value3"
>

...




...


Specifically, this is really useful if you need to merge 2 different versions of  your XML file, like for example in SVN.

Ansible vs. Chef vs. Puppet vs. Salt

There are currently various tools to maintain automatically an infrastructure. The four listed below seem to be the main ones.

·         Ansible http://docs.ansible.com/intro_installation.html#getting-ansible

Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.

http://docs.ansible.com/intro_installation.html#getting-ansible

·         Chef https://www.chef.io/

“Chef turns infrastructure into code. With Chef, you can automate how you build, deploy, and manage your infrastructure. Your infrastructure becomes as versionable, testable, and repeatable as application code."

·         Puppet https://puppetlabs.com

“Puppet is a configuration management solution that allows you to define the state of your IT infrastructure, and then automatically enforces the desired state. Puppet automates every step of the software delivery process, from provisioning of physical and virtual machines to orchestration and reporting; from early-stage code development through testing, production release and updates.”

·         Salt : http://saltstack.com

“SaltStack takes a new approach to infrastructure management by developing software that is easy enough to get running in seconds, scalable enough to manage tens of thousands of servers, and fast enough to control and communicate with them in milliseconds. SaltStack delivers a dynamic infrastructure communication bus used for orchestration, remote execution, configuration management and much more. The Salt project was launched in 2011 and today is the fastest-growing, most-active infrastructure orchestration and configuration management open source project in the world. The SaltStack community is committed to keeping the Salt project focused, friendly, healthy and open.”

http://salt.readthedocs.org/en/latest/contents.html

And some comparisions : 

• [infoworld] Puppet vs. Chef :  http://www.infoworld.com/article/2614204/data-center/puppet-or-chef--the-configuration-management-dilemma.html
• [infoworld] Review : Puppet vs. Chef vs. Ansible vs. Salt : http://www.infoworld.com/article/2609482/data-center/data-center-review-puppet-vs-chef-vs-ansible-vs-salt.html 
• [infoworld] Review: Ansible orchestration is a veteran Unix admin's dream : http://www.infoworld.com/article/2612397/data-center/review--ansible-orchestration-is-a-veteran-unix-admin-s-dream.html 
• [infoworld] Review: Salt keeps server automation simple : http://www.infoworld.com/article/2612536/data-center/review--salt-keeps-server-automation-simple.html
• Review: Puppet 3.0 pulls more strings http://www.infoworld.com/article/2611099/data-center/review--puppet-3-0-pulls-more-strings.html
  
  
  
  
   

CVSS : Common Vulnerability Scoring System (v3) calculator

As explained in wikipedia:CVSS "Common Vulnerability Scoring System (CVSS) is a free and open industry standard for assessing the severity of computer system security vulnerabilities. It is under the custodianship of the Forum of Incident Response and Security Teams (FIRST). It attempts to establish a measure of how much concern a vulnerability warrants, compared to other vulnerabilities, so efforts can be prioritized. The scores are based on a series of measurements (called metrics) based on expert assessment. The scores range from 0 to 10. Vulnerabilities with a base score in the range 7.0-10.0 are High, those in the range 4.0-6.9 as Medium, and 0-3.9 as Low.^[1]"

•  FIRST announces CVSS version 3 

• Common Vulnerability Scoring System Version 3.0 Calculator (FIRST)
  https://www.first.org/cvss/calculator/3.0
• Common Vulnerability Scoring System Version 2 Calculator (Nvd.Nist)
  https://nvd.nist.gov/CVSS-v2-Calculator

Replace all occurences of a string in a bunch of files by another string (in place)

• Find all files on containing STRING_AAAA and replace it by STRING_BBBB in place (directly in the file).

find . -type f -exec grep -l STRING_AAAA {} \; -exec perl -pi -e 's!STRING_AAAA!STRING_BBBB!g' {} \;

Pre-tests :

• Find all files on containing STRING_AAAA.

find . -type f -exec grep -l STRING_AAAA {} \;

Security links

• Bruce Schneier on security (blog) : https://www.schneier.com/

• CERT (French government) : http://www.cert.ssi.gouv.fr/site/index.html
• including a list of links towards France companies CSERT

• SANS  Daily security awareness tips : http://www.sans.org/tip_of_the_day.php
• SANS "securing the human" blog : http://www.securingthehuman.org/blog
• SANS DFIR (digital forensic & incident response) http://digital-forensics.sans.org/community/links
• Found on Sans daily tips : "Security
• Phishing http://www.onguardonline.gov/games/phishing-scams.aspx 
• Spyware http://www.onguardonline.gov/games/beware-spyware.aspx 
• Identity Theft http://www.onguardonline.gov/games/id-theft-faceoff.aspx 
• Social Networking http://www.onguardonline.gov/games/friend-finder.aspx
• ...

get return value ($?) of a tee'd command in ksh

How to get the return value of "command1" if passed to a " | command2" , for example "| tee OUTFILE" :

    command1 | tee OUTFILE

 With recent shells, you can use the array storing this : bash is ${PIPESTATUS[x]} , zsh is $pipestatus[x] to get this value :

      command1 | tee OUTFILE

      echo ${PIPESTATUS[0]}

With older shells, this feature is not available, but you still can use posix features and play around with the file descriptors :

    exec 4>&1; RETVAL=$({ { command1 ; echo $? >&3 ; } | { tee $OUTFILE >&4; } } 3>&1); exec 4>&-

 

Explainations 

1. Create a file descriptor fd4 and map it to STDOUT
2. Fill the RETVAL value with the fork.
1. fd4 is transmitted, but STDOUT and STDERR are recreated for the forked process so fd4 still exist and is available.
2. create the fd3 and redirect it to STDOUT
3. map 'tee' STDOUT to the previously created fd4 so that it gets printed to the terminal; 
4. tee then forks the piped 'command1' which output value $? is sent to fd3 with 'echo $? >&3', and hence outputed in the RETVAL variable; 
3. Eventually, close the fd4

References :

• file descriptor trick : http://stackoverflow.com/questions/2413166/bash-redirect-and-append-stdout-and-stderr-to-file-and-terminal-and-get-prope
• Closing & opening  http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_07  / http://pubs.opengroup.org/onlinepubs/9699919799/
• C More examples for bash,
  http://unix.stackexchange.com/questions/76162/how-do-i-capture-the-return-status-and-use-tee-at-the-same-time-in-korn-shell

Firefox Detect if a page is loaded in the "sidebar" (Javascript)

Firefox has this wonderful option when you create a bookmark : you can load a specific page directly in the sidebar. But it's not very well documented on how you can detect as a web developer that your page is actually shown in such a sidebar.
This is the javascript trick I used (validated with Firefox version=20.0.1):

{ if (window.content == window) /* not sidebar */ else /* sidebar */ }

Source :

• http://forums.mozillazine.org/viewtopic.php?p=1309266#p1309266
• https://developer.mozilla.org/en-US/docs/Web/API/Window/sidebar

Aspect-Oriented Software Development

FROM : http://www.aspectprogramming.com/aosd

What is Aspect-Oriented Software Development?

Aspect-Oriented Software Development (AOSD), sometimes just called Aspect-Oriented Programming (AOP), is a new approach to software design that addresses modularity problems that are not handled well by other approaches, including Structured Programming and Object-Oriented Programming (OOP). AOSD complements, but doesn't replace those approaches.

Outlook VBA : mark a folder/subfolders tree as read

I have been forced to use Outlook (2010), and one of the main features I've been missing was to be able to mark all a folder and sub-folders trees as read in 1 shot. The solution I found was to activate the VBA macros, and to link one as a task icon in the usual Outlook views.
Note that the macro given below is prompting a list of your outlook folders. This could be enhanced by choosing the selected folder since outlook doesn't allow to select

How to activate the Developer tab in outlook and authorize the Macro execution

• Go to the "File"
• Go to "Options" to open the "Outlook Options" window
• In "Customize Ribbon", on the right panel, select "Developer".
• Finish this step with "Ok".
• Go to "Trust Center", then "Trust Center Settings" to open the "Trust Center" window
• Go to "Macro Settings".
• Select "Enable all macros (not recommended; potentially dangerous code can run)".
• Finish by clicking "Ok" to close the "Trust Center" and the "Outlook option" windows.

How to create a VBA macro in Outlook

A macro is any public subroutine in a code module. A function or a private subroutine cannot be a macro, and a macro cannot be located in a class or form module. To create a new macro :

1. In Outlook, on the Developer tab of the Microsoft Office Fluent ribbon, click Visual Basic.
2. In the Project window, double-click the module you want to contain the macro.
3. On the Insert menu, click Procedure.
4. In the Name box, type a name for the macro. The name cannot contain spaces.
5. Click OK. The template for the macro subroutine appears in the code module window.
6. Type the code you want to run in the body of the subroutine.

(from ref [2])

Source Code of the Macro

Sub MarkAllRead()

Dim ResultFolder As Folder
Dim Folder As Folder
Dim item As MailItem
Dim BaseFolder As Outlook.MAPIFolder
Dim WalkResult As Long

Set BaseFolder = Application.GetNamespace("MAPI").PickFolder
Set ResultFolder = GetFolder(BaseFolder.FolderPath)

For Each Folder In ResultFolder.Folders
WalkResult = GetNextLevel(ResultFolder.FolderPath)

For Each item In Folder.Items.Restrict("[unread] = true")
item.UnRead = False
Next
Next
Set ResultFolder = Nothing
Set Folder = Nothing
Set item = Nothing
End Sub

Function GetNextLevel(strFolderPath As String) As Long

Dim WalkResultFolder As Folder
Dim Folder As Folder
Dim item As MailItem
Dim WalkResult As Long
Set WalkResultFolder = GetFolder(strFolderPath)
For Each Folder In WalkResultFolder.Folders

WalkResult = GetNextLevel(Folder.FolderPath)

For Each item In Folder.Items.Restrict("[unread] = true")
item.UnRead = False
Next
Next
Set ResultFolder = Nothing
Set Folder = Nothing
Set item = Nothing
End Function

Function GetFolder(strFolderPath As String) As MAPIFolder

Dim colFolders As Outlook.Folders
Dim objFolder As Outlook.MAPIFolder
Dim arrFolders() As String
Dim i As Long
On Error Resume Next

strFolderPath = Replace(strFolderPath, "\\", "")

strFolderPath = Replace(strFolderPath, "/", "\")

arrFolders() = Split(strFolderPath, "\")

Set objFolder = Application.GetNamespace("MAPI").Folders.item(arrFolders(0))
If Not objFolder Is Nothing Then
For i = 1 To UBound(arrFolders)
Set colFolders = objFolder.Folders
Set objFolder = Nothing
Set objFolder = colFolders.item(arrFolders(i))

If objFolder Is Nothing Then
Exit For
End If
Next
End If
Set GetFolder = objFolder
Set colFolders = Nothing
End Function

(from ref [1] )

How to add a quick access link to your macro

Right click on the arrow at the right end of the Quick Access Toolbar (top right of the outlook window)

Select "More Commands"

On the left panel, select "Macros" (by default it's on "Popular Commands")

Select your Macro on the list, and hit "Add >>". If you have some other icons, you can reorganize them with the up & down arrow on the right.

Sources / references

• [1]Source code of the macro, User=Stonywall, website(forum)=www.mrexcel.com
• [2]How to set up a VBA Macro in outlook2010

To Do

Change the macro so that it directly takes the selected Folder as an input.

REF / Link : Benchmark of Python Web Servers

http://nichol.as/benchmark-of-python-web-servers

An extensive analysis of the different python web servers available on the market.

awk 'system()' vs xargs

Two different syntaxes for processing a command over a list of arg.

Let's take for example, the list of PID you want to kill(all the processes matching "toto") :

ps | grep toto | awk -F' ' '{print $2}' | xargs kill -9

is equivalent to

ps | grep toto | awk -F' ' '{system("kill -9 "$2}'

note that the space at the end of the string "kill -9 " is important since the $2 will be concatenated.

while true for WPS

While true permettant de se connecter dès que quelqu'un initie une connexion avec du WPS

wpa_cli
while : ; do sudo wpa_cli wps_pbc any ; sleep 120 ; done &

Source :
http://korben.info/intrusion-dans-un-reseau-wifi-grace-au-wps.html?utm_source=feedburner&utm_medium=email&utm_campaign=Feed:+Korben_nl+%28Korben%29

Find big files

In the current path, find all files bigger than, let's say 10MiB  (10^7Bytes).


$ find . -type f -size +10000000c -exec ls -l {} \;