Strony

wtorek, 17 września 2013

Encrypt command to execute

On one of servers I have access to I wanted to have a bash script knocking to my home server and then – after the sesame is open – ssh into it. But I didn’t want to store knock sequence in the bash script or file, that everyone has (a potential) access to. Here is small how-to use gpg to encrypt/decrtypt commands to execute.

 


(1) Create a file with command to execute, eg in file command.txt:
knock example.com 120:tcp 121:udp 122:tcp && sleep 1s && ssh username@example.com

 


(2) Encrypt the file with gpg:
gpg -a --symmetric --cipher-algo TWOFISH command.txt
(cipher algorithm here is TWOFISH, but can be anything else). New file command.txt.asc should be created.

 


(3) Create a script which decrypt and execute above command:
#!/bin/bash
command=`gpg -d  command.txt.asc  2> /dev/null`
eval "$command"
This command will ask for the passphase and after successfull decryption will run desired command.

piątek, 17 maja 2013

(simple) webcam image analysis

Here I present simple webcam image analysis and it's variation in time.

Intro


I have a webcam, connected to my Raspberry Pi server. Photos are taken every five minutes with fswebcam. Such collection of photos you can use for creating eg. time lapse video (here is an example of one-day video taken from my window). But you can also perform an analysis of image composition and it's variation over time.

Howto


Gathering data


Wen we have photos taken during a day gargered in one directory, we can use a simple script to analyse subsequent frames. ImageMagick is used to convert photos to 1x1 px size and get their HSB and RGB values.

#!/bin/bash

OUT=webcam.dat

echo "#i        h       s       v       r       g       b" > $OUT

for image in *.jpg
do

((i++))

hsb=`convert "$image" -colorspace hsb  -resize 1x1  txt:- | sed 1d  | tr '()%,:' ' '`

h=`echo $hsb | awk '{print $8}'`
s=`echo $hsb | awk '{print $9}'`
v=`echo $hsb | awk '{print $10}'`

rgb=`convert "$image" -colorspace rgb  -resize 1x1  txt:- | sed 1d  | tr '()%,:' ' '`

r=`echo $rgb | awk '{print $3}'`
g=`echo $rgb | awk '{print $4}'`
b=`echo $rgb | awk '{print $5}'`

echo "$i        $h      $s      $v      $r      $g      $b" >> $OUT

done

Data visualisation


Then we use a gnuplot script to get pretty plots:

set pointsize 2
# Line style for axes
set style line 80 lt rgb "#808080"
# Line style for grid
set style line 81 lt 0  # dashed
set style line 81 lt rgb "#808080"  # grey
set grid back linestyle 81
set border 3 back linestyle 80
set xtics nomirror
set ytics nomirror
set xlabel "frame number"
set terminal png nocrop enhanced font "Gill Sans,8" size 700,500

plik='webcam.dat'

set ylabel "HSB [%]"

set output "out/hsv-csplines.png"
plot plik using 1:2 smooth csplines title "Hue" lt rgb "red" lw 1,\
plik using 1:3 smooth csplines title "Saturation" lt rgb "green" lw 1,\
plik using 1:4 smooth csplines title "Brightness" lt rgb "blue" lw 1

set output "out/hsv-bezier.png"
plot plik using 1:2 smooth bezier title "Hue" lt rgb "red" lw 1,\
plik using 1:3 smooth bezier title "Saturation" lt rgb "green" lw 1,\
plik using 1:4 smooth bezier title "Brightness" lt rgb "blue" lw 1

set ylabel "RGB"

set output "out/rgb-csplines.png"
plot plik using 1:5 smooth csplines title "Red" lt rgb "red" lw 1,\
plik using 1:6 smooth csplines title "Green" lt rgb "green" lw 1,\
plik using 1:7 smooth csplines title "Blue" lt rgb "blue" lw 1

set output "out/rgb-bezier.png"
plot plik using 1:5 smooth bezier title "Red" lt rgb "red" lw 1,\
plik using 1:6 smooth bezier title "Green" lt rgb "green" lw 1,\
plik using 1:7 smooth bezier title "Blue" lt rgb "blue" lw 1


Output: plots


HSB, with different methods of curves smoothing

 

RGB, with different methods of curves smoothing

 

Live monitoring


To get the plots almost live, we can use rrdtool to gather, store and plot those data. First, create rrd database:


#!/bin/bash
rrdtool create hsb-rgb.rrd \
--step 300 \
DS:h:GAUGE:600:0:100 \
DS:s:GAUGE:600:0:100 \
DS:j:GAUGE:600:0:100 \
DS:r:GAUGE:600:0:255 \
DS:g:GAUGE:600:0:255 \
DS:b:GAUGE:600:0:255 \
RRA:AVERAGE:0.5:1:288                   \
RRA:AVERAGE:0.5:6:336                   \
RRA:AVERAGE:0.5:24:732                  \
RRA:AVERAGE:0.5:144:14600       \
RRA:MIN:0.5:1:288                       \
RRA:MIN:0.5:6:336                       \
RRA:MIN:0.5:24:732                      \
RRA:MIN:0.5:144:14600   \
RRA:MAX:0.5:1:288                       \
RRA:MAX:0.5:6:336                       \
RRA:MAX:0.5:24:732                      \
RRA:MAX:0.5:144:14600



Then add such lines to your webcam script (launched from cron every 5 minutes). $OUT_FILE is variable pointing to a just taken photo.

hsb=`convert "$OUT_FILE" -colorspace hsb  -resize 1x1  txt:- | sed 1d  | tr '()%,:' ' '`

h=`echo $hsb | awk '{print $8}'`
s=`echo $hsb | awk '{print $9}'`
v=`echo $hsb | awk '{print $10}'`

rgb=`convert "$OUT_FILE" -colorspace rgb  -resize 1x1  txt:- | sed 1d  | tr '()%,:' ' '`

r=`echo $rgb | awk '{print $3}'`
g=`echo $rgb | awk '{print $4}'`
b=`echo $rgb | awk '{print $5}'`

# update rrd database:
rrdtool update hsb-rgb.rrd "N:$h:$s:$v:$r:$g:$b"


Finally, after some data are collected, we can plot a nice graph (actually, we can plot it every five minutes, as new data arrives):

#!/bin/bash

BAZA=hsb-rgb.rrd

rrdtool graph hsb-24h.png  \
        --imgformat PNG \
        --start "now-24h"\
        --title="HSB [%]"       \
        --height 240    \
        --width 420    \
        --lazy \
        --alt-autoscale \
        --alt-y-grid \
        DEF:h=$BAZA:h:AVERAGE \
        DEF:s=$BAZA:s:AVERAGE \
        DEF:j=$BAZA:j:AVERAGE \
        LINE2:h#FF0000:"Hue"\
        LINE2:s#00FF00:"Saturation"\
        LINE2:j#0000FF:"Brightness"</pre>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFCZoyRIhEXYQEgSiiI3RGEZFtSfnJog6hNDprEf8JzjU7_1v8DCDAnxSUuxBcq3tqSYGJkfdr4UoZ2rHV-BBREHFU8KE4Flnob8XnD7_5PhWOzuxG9B8sidzWmhX8PGtMGFU0Rlo_l7XL/s1600/HSB-24h.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFCZoyRIhEXYQEgSiiI3RGEZFtSfnJog6hNDprEf8JzjU7_1v8DCDAnxSUuxBcq3tqSYGJkfdr4UoZ2rHV-BBREHFU8KE4Flnob8XnD7_5PhWOzuxG9B8sidzWmhX8PGtMGFU0Rlo_l7XL/s320/HSB-24h.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyGserUs6dUyqdudZA5_UagqTuZTj_weClykD_LKd_6ENAYSKw5tuqsugFVGbGH73rxJAH1kDhs2dCF_DBLB_H_dxZsD6k7XRRYFcdzP9uVgJKxIdA675z53hNIAeX0BXE2UsswMLBRK9D/s1600/RGB-24h.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyGserUs6dUyqdudZA5_UagqTuZTj_weClykD_LKd_6ENAYSKw5tuqsugFVGbGH73rxJAH1kDhs2dCF_DBLB_H_dxZsD6k7XRRYFcdzP9uVgJKxIdA675z53hNIAeX0BXE2UsswMLBRK9D/s320/RGB-24h.png" /></a>


Conclusions

I don't know, if this makes any sense, but looks cool :)

sobota, 20 kwietnia 2013

Bash script to generate knock sequences

Here is a simple bash script to generate random knock/knockd sequences with desired length, port ranges and types:

#!/bin/bash

# ---- config.
knocks_number=5
knocks_types=('udp' 'tcp');

port_min=1000
port_max=2000

# ---- config end

dport=$(( $port_max - $port_min ))

for i in `seq 1 $knocks_number`
do
    sequence=$sequence$[`shuf -i $port_min-$port_max -n 1` ]":"${knocks_types[ $[($RANDOM % 2)] ]}","
done

echo
echo "# A sequence for knockd.conf:"
echo "sequence = $sequence" | sed 's/,$//g'
echo "# A sequence to use with knock"
echo "knock \$ADDRESS $sequence" | sed 's/,/ /g'



The output may looks like:

# A sequence for knockd.conf:
sequence = 1460:udp,1344:tcp,1997:tcp,1712:udp,1304:udp
# A sequence to use with knock
knock $ADDRESS 1460:udp 1344:tcp 1997:tcp 1712:udp 1304:udp

poniedziałek, 15 kwietnia 2013

Control your computer with jabber/google talk and PHP

(I doubt this would be very useful, however maybe you will find it interesting)

Intro

When your linux box is hidden behind nat/firewall with no ssh/www/... access, you can use jabber protocol to control your computer (or do it for other purposes).

(hint: when you have access to ssh server you can use SSH tunneling for connection to your machine)


Prerequisites

You will need:

  • two jabber account (can be gmail account): one used only on remote computer, second can be your regular gmail account. Add each accounts to their contact lists.
  • on the linux box you want to connect with: php-cli installed
  • XMPPHP (download and unpack) with applied bug fix described here in XMLStream.php.
  • optionally: account on the server dedicated for this purpose (eg: adduser jabber). 

Script

here is simple script which connects to your jabber account (which is probably gmail/google talk), listen to your commands, executes it and return to you:


To run this script in a background, check periodically if the script is running (and client is connected to a jabber server) you would also need a wrapper


To execute above wrapper periodically, just add a line to your crontab:

*/5 * * * * /home/jabber/bin/jabber/wrapper.sh 1> /dev/null 2> /dev/null

here is a screenshot from session with my raspberry pi server:



sobota, 6 kwietnia 2013

Enhance night images from webcam

Night images taken by cheap webcams are usually blurry, pixelized and low quality:


But the quality of final image can be enhanced by averaging multiple subsequent expositions. Here is simple bash script. All we need is:program which takes photo by your webcam (here I use fswebcam) and convert from ImageMagick suite. Both available in repositories of main linux distributions.

    for i in {1..5}
    do
    echo -n "$i "
    fswebcam --save $1.jpg

    sleep 2s
    done

    echo
    echo "Averaging...."
    convert *.jpg -evaluate-sequence mean averaged.jpg

This script takes five photos, with two seconds pauses, a nd then "averages" those photos to eliminate artifacts. Here is "averaged" image:


As we can see, the quality is better than before. Good. But next problem is: we would like to perform such averaging only for dark images (images taken in good light conditions are acceptable quality).

Checking image brightness

Simple metod to asses brightness of image. By issuing command:

convert 0.jpg -colorspace hsb  -resize 1x1  txt:-

we gets some information, including "brightness":

# ImageMagick pixel enumeration: 1,1,255,hsb
0,0: (154,193,  1)  #9AC101  hsb(60.441%,75.8801%,0.488289%)

here 0.488289% is brightness.

To get the value, we use some bash tricks:

convert 0.jpg -colorspace hsb  -resize 1x1  txt:- | \
sed 1d | tr '()%' ' ' | awk -F "," '{print $6}'

then we can make some conditional if to check if image brightness is acceptable or low - in this case we will perform averaging. The script could look like:


#!/bin/bash

OUTPUT_DIR=/var/www/webcam/`date +"%Y/%m/%d"`
OUT_FILE=`date +"%Y-%m-%d_%H:%M:%S"`
TEMP=/var/www/webcam/temp

BRIGHTNESS_TRESHOLD_MIN=10      # for too dark images
BRIGHTNESS_TRESHOLD_MAX=60      # for too bright images


mkdir -p $OUTPUT_DIR
mkdir -p $TEMP



function take_photo {
fswebcam  -S 32 --jpeg 65 \
        -r 640x480 \
        --banner-colour "#35000000" --line-colour "#35000000" \
        --timestamp "%Y-%m-%d %H:%M (%Z)" \
        --set lights=off \
        --set "White Balance Temperature, Auto"=True \
        --set brightness=$2  \
        -q \
        --set "Saturation"=58 \
        --save $TEMP/$1.jpg
}


echo ------------------------ first photo -------------

take_photo 0 "50%"

jasnosc=`convert $TEMP/0.jpg -colorspace hsb  -resize 1x1  txt:- | \
sed 1d | tr '()%' ' ' | awk -F "," '{print $6}'`

#echo `echo "$jasnosc <= $BRIGHTNESS_TRESHOLD" | bc`
#exit

if [ `echo "$jasnosc <= $BRIGHTNESS_TRESHOLD_MIN" | bc` -eq 1 ]
then
    echo -n "Brightness: $jasnosc ... too low - averaging more photos ....: "
    for i in {1..5}
    do
    echo -n "$i "
    take_photo $i "66%"
        sleep 2s
    done

    echo

   echo "Averaging...."
    cd $TEMP
    convert *.jpg -evaluate-sequence mean averaged.jpg
    mv averaged.jpg $OUTPUT_DIR/"$OUT_FILE".jpg
    rm *

elif [ `echo "$jasnosc >= $BRIGHTNESS_TRESHOLD_MAX" | bc` -eq 1 ]
then
    echo "Brightness: $jasnosc ... too high - reducing brightness  ..."
    take_photo 0 "30%"
    mv $TEMP/0.jpg $OUTPUT_DIR/"$OUT_FILE".jpg

else
    echo "Brightness: $jasnosc :  one photo is enoguh ...."
    mv $TEMP/0.jpg $OUTPUT_DIR/"$OUT_FILE".jpg
fi

tehere is another elseif: if image is too bright, we are taking another one with reduced brightness.


Additional features


We can log the calculated brightness and then make a nice plot of this relative brightness during days:


Overexposed images from cheap webcam

It sometimes happens with cheap webcams, that as images taken inside (in the room) are well exposed, the images taken outside (eg from window) are far too bright. They are overexposed. This is because those webcams have fixed aperture, which is set for low light environment (like room). When we want to take photo outside, especially during sunny day, too much light gets into the sensor and image is too bright:

The solution could be disassemble the camera and install new aperture, with smaller hole (like here). But there is simpler (and probably better) solution.

Our cheap webcam needs sunglasses. Yes:






After this operation, the images are better exposed:



Of course, the best solution would be obtaining camera with adjustable aperture :)

piątek, 5 kwietnia 2013

Simple SSH tunnel with auto resuming

Intro 

There is a lot of tutorials about SSH tunnels. Also there is a number of tutorials about keeping/resuming/checking SSH tunnels. For me all those solutions has a big drawback: either they didn't work when system was rebooted, or they required root privilegs (or both, or simply they didn't work for me).

Here is a simple description how to set up such tunnel with quite smart resuming.

Idea


Nothing new here. We want be able to SSH from compA to compB, but compB is behind firewall/NAT, so it is not possible in a normal way (ie ssh compB - red arrow). Thus we have to set up SSH tunnel. Still: nothing new:

On compB:
ssh -R 19999:localhost:22 username@compA

Then on compA we can ssh to compB this way:
ssh localhost -p 19999 -v -l username 
(where 19999 is a port, can be another big number)

But what if SSH connection from compB to compA is lost? We will not be able to connect from compA to compB. Thus, we have to make sure, that this connection (B to A) will be resumed in case of connection lost or system reboot.

How to

On compB:
  1. Enable ssh logging from compB to compA without password (there is a lot of howtos: google it)
  2. Create auto connection script
    #!/bin/bash
    
    REMOTEUSER=username
    REMOTEHOST=compA   #compA IP
    
    SSH_REMOTEPORT=22
    SSH_LOCALPORT=19999
    
    COMMAND="$SSH_LOCALPORT:localhost:22 $REMOTEUSER@$REMOTEHOST"
    
    a=`ps -fe | grep "$COMMAND" | grep -v grep`
    if [ ! "$a" ]; then
        echo "No connection"
        ssh -o TCPKeepAlive=yes -fN -R $COMMAND
        date >> tunnel.log
    else
        echo "Connected"
    fi
  3. Add it to crontab, eg:
    */5 * * * * ssh_check.sh 1> /dev/null 2> /dev/null
that's it. Now you probably will be able to connect from compA to compB. Even when compB was restarted or connection was lost, as soon as the cron is triggered, the connection will be resumed.