Sunday, December 28, 2008
Text Processing with Perl
I was trying to use OpenOffice for spreadsheet purposes and I quickly came to the conclusion it was too slow and bloated. Then I remembered that awk could be used for text processing. Someone on IRC suggested to simply use awk and vim. All I needed was an easy way to add up some numbers in a text file and this is what I came up with:
awk '/\// { sum1 += $3 }; END { print sum1 }' /hd1/text/ptc.txt | while read a
do
echo "ptc total: " $a > /hd1/text/current-earnings.txt
done
awk ' { sum2 += $2 }; END { print sum2 }' /hd1/text/surveys.txt \
| while read b
do
echo "survey total: " $b >> /hd1/text/current-earnings.txt
done
awk ' { sum3 += $2 }; END { print sum3 }' /hd1/text/ads.txt \
| while read c
do
echo "ad total : " $c >> /hd1/text/current-earnings.txt
done
Not very elegant is it? Then another IRC person suggested to use Perl. The big problem with the awk script above is the awk task gets spawned in a child shell and can't set any environment variables in bash. Let's try again and use Perl:
sum1=`perl -e 'foreach (<>){@a = split(/\s+/); $s1 += $a[2];}; print $s1;' /hd1/text/ptc.txt`
echo "ptc total: $" $sum1
sum2=`perl -e 'foreach (<>){@a = split(/\s+/); $s2 += $a[1];}; print $s2;' /hd1/text/surveys.txt`
echo "survey total: $" $sum2
sum3=`perl -e 'foreach (<>){@a = split(/\s+/); $s3 += $a[1];}; print $s3;' /hd1/text/ads.txt`
echo "ads total: $" $sum3
total=`perl -e '$e += $_ foreach(@ARGV); print $e' $sum1 $sum2 $sum3`
echo "Grand total: $" $total
That's better. Now I can put this into a file called 'earnings' and make it executable and I'm off to the races. Paid surveys make the most money for me followed by PTC and the ads make the least. One can easily adjust things if you need to add numbers in a different field. The variable $a[2] refers to the 3rd field in the text file, as an example:
12/26/08 clixsense 0.70 (min $10)
The last part of the script adds the sums and gives the total.
awk '/\// { sum1 += $3 }; END { print sum1 }' /hd1/text/ptc.txt | while read a
do
echo "ptc total: " $a > /hd1/text/current-earnings.txt
done
awk ' { sum2 += $2 }; END { print sum2 }' /hd1/text/surveys.txt \
| while read b
do
echo "survey total: " $b >> /hd1/text/current-earnings.txt
done
awk ' { sum3 += $2 }; END { print sum3 }' /hd1/text/ads.txt \
| while read c
do
echo "ad total : " $c >> /hd1/text/current-earnings.txt
done
Not very elegant is it? Then another IRC person suggested to use Perl. The big problem with the awk script above is the awk task gets spawned in a child shell and can't set any environment variables in bash. Let's try again and use Perl:
sum1=`perl -e 'foreach (<>){@a = split(/\s+/); $s1 += $a[2];}; print $s1;' /hd1/text/ptc.txt`
echo "ptc total: $" $sum1
sum2=`perl -e 'foreach (<>){@a = split(/\s+/); $s2 += $a[1];}; print $s2;' /hd1/text/surveys.txt`
echo "survey total: $" $sum2
sum3=`perl -e 'foreach (<>){@a = split(/\s+/); $s3 += $a[1];}; print $s3;' /hd1/text/ads.txt`
echo "ads total: $" $sum3
total=`perl -e '$e += $_ foreach(@ARGV); print $e' $sum1 $sum2 $sum3`
echo "Grand total: $" $total
That's better. Now I can put this into a file called 'earnings' and make it executable and I'm off to the races. Paid surveys make the most money for me followed by PTC and the ads make the least. One can easily adjust things if you need to add numbers in a different field. The variable $a[2] refers to the 3rd field in the text file, as an example:
12/26/08 clixsense 0.70 (min $10)
The last part of the script adds the sums and gives the total.
Friday, December 12, 2008
Hacker Scripts
One of the things bad hackers like to do is run their code on other people's computers. Some of the ways this is done is via Perl or PHP scripts. Here is an example of a simple information gathering script written in PHP:
v7id.txt
It looks innocuous, but what does it do exactly?
It returns the UID, GID and Groups numbers of the account the script was executed on as well as all the group names. After that it does a "uname -a" and returns some system information (e.g. the exact version of the kernel you are running). Finally it returns the amount of free disk space and the total amount of disk space.
This is still bad because it means that there is some vulnerability in your system even though the script doesn't do anything destructive. After scouting out your system they are in a better position to compromise it.
Now if someone is always on the server they can keep an eye out for certain tell-tale signs of hacker activity. Sometimes you'll see things that look like hacker activity but aren't, e.g. Reddit or some other popular site linking to one of your web pages.
Some of the danger signs:
Security of your server is an on-going job and this blog entry is only scratching the surface. One simple thing you can do is to lock down important executables with "chmod 700" so only root can read, write or execute them.
Here is a list of executables that should be locked down, assuming they're on your system:
wget, elinks, curl, lynx, vsftpd, ftp
Basically any command that can "phone home" should be locked down. Even if you think your security is pretty good some new exploit could allow the bad hackers a way to execute scripts on your system.
v7id.txt
It looks innocuous, but what does it do exactly?
It returns the UID, GID and Groups numbers of the account the script was executed on as well as all the group names. After that it does a "uname -a" and returns some system information (e.g. the exact version of the kernel you are running). Finally it returns the amount of free disk space and the total amount of disk space.
This is still bad because it means that there is some vulnerability in your system even though the script doesn't do anything destructive. After scouting out your system they are in a better position to compromise it.
Now if someone is always on the server they can keep an eye out for certain tell-tale signs of hacker activity. Sometimes you'll see things that look like hacker activity but aren't, e.g. Reddit or some other popular site linking to one of your web pages.
Some of the danger signs:
- Much higher than normal disk activity
- Too many instances of sendmail
- Instances of lwp-download or wget running when they shouldn't be
- More than one process running klogd or acpid
Security of your server is an on-going job and this blog entry is only scratching the surface. One simple thing you can do is to lock down important executables with "chmod 700" so only root can read, write or execute them.
Here is a list of executables that should be locked down, assuming they're on your system:
wget, elinks, curl, lynx, vsftpd, ftp
Basically any command that can "phone home" should be locked down. Even if you think your security is pretty good some new exploit could allow the bad hackers a way to execute scripts on your system.
Subscribe to Posts [Atom]