Posts Tagged admin

netcat as a logging tcp proxy

I felt I needed to write an article about netcat, so here is it !
Netcat is an incredibly usefull tool, that allows you to play with tcp connection easily from the shell.
Basically, as it name implies, it’s just cat over the network, but what its name doesn’t tell you is that it also can act as a socket listener.
So let’s play with pipes, here is one of my favourite use of netcat:

mkfifo proxypipe
cat proxypipe | nc -l -p 80 | tee -a inflow | nc localhost 81 | tee -a outflow 1>proxypipe

This command will redirect traffic from localhost:80 to localhost:81, in the inflow file you while find the incoming http request, in the outfile, you will find the http response from the server.
Similarly, you can do this:

cat proxypipe | nc -l 80 | tee -a inflow | sed 's/^Host.*/Host:' |  nc 80 | tee -a outflow >proxypipe

This will allow your browser to point to google using http://localhost .
Anyway, this is my favourite but netcat has thounds of other uses, have a look at it !
It can be usefull for file transfers (gzip|nc) , performance measurement (dd|gzip), protocol debugging (replaying requests), security testing (nc does port scan) …


Comments (3)

Installing redmine 0.8 on intrepid (ubuntu 8.10)

I’ve successfully insalled redmine pretty much easily but I needed to find out what packages to install with apt, which one with gem, which version …
Here is my magic receipe to install it all:

apt-get update 
apt-get install subversion mysql-server rubygems rake pwgen
# next line generates a password for the database
export PASSWORD=`pwgen -nc 8 1`
gem install -v=2.1.2 rails
cd /opt/
svn export redmine-0.8
cd redmine-0.8/
cat <<EOF >> config/database.yml
  adapter: mysql
  socket: /var/run/mysqld/mysqld.sock 
  database: redmine
  host: localhost
  username: redmine
  password: $PASSWORD
  encoding: utf8

rake db:migrate RAILS_ENV="production"
rake redmine:load_default_data RAILS_ENV="production"
apt-get remove pwgen subversion
RAILS_ENV="production" ./script/server  

And that’s it ! Redmine is running on port 3000.
I did this on an EC2 instance and it works like a charm (ami-7cfd1a15).
Maybe next article will discuss running redmine in mongrel or apache, and creating an init script for having redmine running on boot !

Comments (2)

Hostname and underscore

RFC 952 and RFC 1123 explains the rules for choosing a hostname. I noticed recently that a lot of admins (including me) are using underscores in hostnames, but this doesn’t follow RFCs. This can lead to strange behaviours, such as mail not delivered with an RFC compliant mail server to an MX that have underscores in its name …
I noticed that because the “hostname” command on linux can set the hostname of a system, but the command doesn’t accept underscores. So guys, don’t use underscores !

Comments (1)

watch your process !

I just discovered the watch command, it can be useful !
If you don’t know watch, it does what you would do like this:
while true ; do "your command" ; sleep 1 ; clear ; done
that is, it executes in a while loop the same command , with a sleep so that it doesn’t overkill your cpu.
It also has nice parameters, for exemple --differences that can only show the differences between current and last run.
“your command” could be a du or a df , --differences could be useful when used with an ls to monitor a directory …
Read the manpage and have fun ! 🙂

Leave a Comment

web based bind zone generator

There a some web based bind zone generator, but searching for “zone generator” in google, I found a lot that aren’t working, refining my research didn’t helped me. I finally found one that does the stuff. It’s not optimal , but it work , and it’s there
Please, if you know of a better one, just let me know !

Comments (1)

Database (mysql) backup script

Don’t need to say much, everything is in the title of this post 🙂 .
Here is my mysql database backup script:


function backup_db {

BACKUPFILESUFFIX="`date +%m%d`.bz2"
DBLIST=`echo "show databases" | mysql -u backup -p$PASS -h $HOST`
NUMDB=`echo $DBLIST |wc -w`

if [ ! -d ${dir} ]; then
  mkdir ${dir}
if [ ! -e $DIR/count.$HOST ]; then
  echo $NUMDB > $DIR/count.$HOST

COUNT=`cat $DIR/count.$HOST`

if [ "$COUNT" -lt "$NUMDB" ]; then
  echo -e "Databases list: $DBLIST" | mail -s "New database, maybe new backups needed!" $MAIL
  echo $NUMDB > $DIR/count.$HOST

/usr/bin/mysqldump -u $USER -p$PASS -h $HOST --routines $DB|bzip2 > $DIR/$DB.$BACKUPFILESUFFIX

if [ $? != 0 ] ; then
  echo -e "Return code is : $? and log file contains:\n `cat $LOGFILE`" | mail -s "Backup MySQL $HOST: $DB Error" $MAIL

SIZE=`du  -sk $DIR/$DB.$BACKUPFILESUFFIX| cut -f1`
if [ "$SIZE" -lt $MINSIZE ]; then
  echo -e "File is smaller than $MINSIZE k, printing an ls output:\n `ls -l $DIR`" | mail -s "Backup MySQL $HOST potential error" $MAIL

# Cleaning up old files, or disks won't fill
find $DIR -ctime +7 -name "*bz2" -exec rm {} \; -print

backup_db "" "user" "password" "database" "9999"

To backup your databases add at the end of script, one line per database, based on this format:
backup_db "host" "user" "password" "database name" "min_size"

You also need to create a backup user on your database, I use this script:

CREATE USER 'backup'@ 'backup-host' IDENTIFIED BY 'a-password';

GRANT SHOW DATABASES ON * . * TO 'backup'@ 'backup-host' IDENTIFIED BY 'a-password ;

SHOW VIEW ON `a_database_to_backup` . * TO 'backup'@ 'backup-host';

And also add a crontab entry:

cat <<EOF >/etc/cron.d/dbbackup
0 5 * * * root /var/backups/db/ 2>&1 >/var/backups/db/backupdb.log
chmod +x /etc/cron.d/dbbackup

This script have some features I have implemented that I find usefull:

  • It mails you when a new databases is created (devs sometimes create a database but don’t inform me, they need backup of it, in case)
  • It’s easy to add new databases to backup
  • It checks for a minimum size, you know that some databases won’t be less than a fixed size, if it happens, there is probably a problem with the backup script or within the database
  • It also backups stored procedures (we use the --routines option of mysqldump)
  • It has a 7 days rotation mechanism, so the disk don’t fill
  • The databases are compressed

Some improvement that can be done to this script:

  • Better error handling, I’m not really sure how it works, I made this script pretty much fast for my daily needs
  • Use mk-parallel-dump from the maatkit
  • Use .my.cnf, and don’t display password in the script, is it better ?
  • please comment to give me some ideas 😉

Leave a Comment