Posts Tagged admin
May 13, 2009 at 9:18 am
· Filed under Uncategorized ·Tagged security, shell, system, network, script, scripting, networking, admin, scripts, sysadmin, system administration, shell script, reverse engineeering, debug, socket, networks
I felt I needed to write an article about netcat, so here is it !
Netcat is an incredibly usefull tool, that allows you to play with tcp connection easily from the shell.
Basically, as it name implies, it’s just cat over the network, but what its name doesn’t tell you is that it also can act as a socket listener.
So let’s play with pipes, here is one of my favourite use of netcat:
mkfifo proxypipe
cat proxypipe | nc -l -p 80 | tee -a inflow | nc localhost 81 | tee -a outflow 1>proxypipe
This command will redirect traffic from localhost:80 to localhost:81, in the inflow file you while find the incoming http request, in the outfile, you will find the http response from the server.
Similarly, you can do this:
cat proxypipe | nc -l 80 | tee -a inflow | sed 's/^Host.*/Host: www.google.fr/' | nc www.google.fr 80 | tee -a outflow >proxypipe
This will allow your browser to point to google using http://localhost .
Anyway, this is my favourite but netcat has thounds of other uses, have a look at it !
It can be usefull for file transfers (gzip|nc) , performance measurement (dd|gzip), protocol debugging (replaying requests), security testing (nc does port scan) …
Permalink
April 16, 2009 at 4:05 pm
· Filed under Uncategorized ·Tagged linux, ubuntu, system, script, scripting, admin, tips, tip, scripts, tips and tricks, sysadmin, system administration, shell script, project management, pm, tools, install, deployement, redmine
I’ve successfully insalled redmine pretty much easily but I needed to find out what packages to install with apt, which one with gem, which version …
Here is my magic receipe to install it all:
apt-get update
apt-get install subversion mysql-server rubygems rake pwgen
# next line generates a password for the database
export PASSWORD=`pwgen -nc 8 1`
gem install -v=2.1.2 rails
echo "CREATE DATABASE redmine DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci ; GRANT ALL PRIVILEGES ON redmine.* TO 'redmine'@'localhost' IDENTIFIED BY '$PASSWORD' WITH GRANT OPTION; FLUSH PRIVILEGES" | mysql
cd /opt/
svn export http://redmine.rubyforge.org/svn/branches/0.8-stable redmine-0.8
cd redmine-0.8/
cat <<EOF >> config/database.yml
production:
adapter: mysql
socket: /var/run/mysqld/mysqld.sock
database: redmine
host: localhost
username: redmine
password: $PASSWORD
encoding: utf8
EOF
rake db:migrate RAILS_ENV="production"
rake redmine:load_default_data RAILS_ENV="production"
apt-get remove pwgen subversion
RAILS_ENV="production" ./script/server
And that’s it ! Redmine is running on port 3000.
I did this on an EC2 instance and it works like a charm (ami-7cfd1a15).
Maybe next article will discuss running redmine in mongrel or apache, and creating an init script for having redmine running on boot !
Permalink
April 14, 2009 at 3:20 pm
· Filed under Uncategorized ·Tagged admin, dns, sys, sysadmin, system, system administration, tips, tips and tricks
RFC 952 and RFC 1123 explains the rules for choosing a hostname. I noticed recently that a lot of admins (including me) are using underscores in hostnames, but this doesn’t follow RFCs. This can lead to strange behaviours, such as mail not delivered with an RFC compliant mail server to an MX that have underscores in its name …
I noticed that because the “hostname” command on linux can set the hostname of a system, but the command doesn’t accept underscores. So guys, don’t use underscores !
Permalink
October 16, 2008 at 9:53 am
· Filed under Uncategorized ·Tagged admin, commands, linux, process, shell, sysadmin, system, system administration, unix
I just discovered the watch command, it can be useful !
If you don’t know watch, it does what you would do like this:
while true ; do "your command" ; sleep 1 ; clear ; done
that is, it executes in a while loop the same command , with a sleep so that it doesn’t overkill your cpu.
It also has nice parameters, for exemple --differences that can only show the differences between current and last run.
“your command” could be a du or a df , --differences could be useful when used with an ls to monitor a directory …
Read the manpage and have fun !
Permalink
September 10, 2008 at 12:28 pm
· Filed under Uncategorized ·Tagged admin, administration, bind, dns, network, sysadmin, system, tips, tips and tricks, tricks, unix, web
There a some web based bind zone generator, but searching for “zone generator” in google, I found a lot that aren’t working, refining my research didn’t helped me. I finally found one that does the stuff. It’s not optimal , but it work , and it’s there
Please, if you know of a better one, just let me know !
Permalink
August 4, 2008 at 10:44 am
· Filed under Uncategorized ·Tagged admin, backup, database, db, mysql, script, scripting, system
Don’t need to say much, everything is in the title of this post
.
Here is my mysql database backup script:
DIR="/var/backups/db/"
MAIL="your@mail.address"
LOGFILE=$DIR/backupdb.log
function backup_db {
HOST="$1"
USER="$2"
PASS="$3"
DB="$4"
MINSIZE="$5"
BACKUPFILESUFFIX="`date +%m%d`.bz2"
DBLIST=`echo "show databases" | mysql -u backup -p$PASS -h $HOST`
NUMDB=`echo $DBLIST |wc -w`
if [ ! -d ${dir} ]; then
mkdir ${dir}
fi
if [ ! -e $DIR/count.$HOST ]; then
echo $NUMDB > $DIR/count.$HOST
fi
COUNT=`cat $DIR/count.$HOST`
if [ "$COUNT" -lt "$NUMDB" ]; then
echo -e "Databases list: $DBLIST" | mail -s "New database, maybe new backups needed!" $MAIL
echo $NUMDB > $DIR/count.$HOST
fi
/usr/bin/mysqldump -u $USER -p$PASS -h $HOST --routines $DB|bzip2 > $DIR/$DB.$BACKUPFILESUFFIX
if [ $? != 0 ] ; then
echo -e "Return code is : $? and log file contains:\n `cat $LOGFILE`" | mail -s "Backup MySQL $HOST: $DB Error" $MAIL
fi
SIZE=`du -sk $DIR/$DB.$BACKUPFILESUFFIX| cut -f1`
if [ "$SIZE" -lt $MINSIZE ]; then
echo -e "File is smaller than $MINSIZE k, printing an ls output:\n `ls -l $DIR`" | mail -s "Backup MySQL $HOST potential error" $MAIL
fi
}
# Cleaning up old files, or disks won't fill
DIR="/var/backups/db/"
find $DIR -ctime +7 -name "*bz2" -exec rm {} \; -print
backup_db "192.168.0.1" "user" "password" "database" "9999"
To backup your databases add at the end of script, one line per database, based on this format:
backup_db "host" "user" "password" "database name" "min_size"
You also need to create a backup user on your database, I use this script:
CREATE USER 'backup'@ 'backup-host' IDENTIFIED BY 'a-password';
GRANT SHOW DATABASES ON * . * TO 'backup'@ 'backup-host' IDENTIFIED BY 'a-password ;
GRANT SELECT ,
LOCK TABLES ,
SHOW VIEW ON `a_database_to_backup` . * TO 'backup'@ 'backup-host';
And also add a crontab entry:
cat <<EOF >/etc/cron.d/dbbackup
MAILTO=root
0 5 * * * root /var/backups/db/backupdb.sh 2>&1 >/var/backups/db/backupdb.log
EOF
chmod +x /etc/cron.d/dbbackup
This script have some features I have implemented that I find usefull:
- It mails you when a new databases is created (devs sometimes create a database but don’t inform me, they need backup of it, in case)
- It’s easy to add new databases to backup
- It checks for a minimum size, you know that some databases won’t be less than a fixed size, if it happens, there is probably a problem with the backup script or within the database
- It also backups stored procedures (we use the
--routines option of mysqldump)
- It has a 7 days rotation mechanism, so the disk don’t fill
- The databases are compressed
Some improvement that can be done to this script:
- Better error handling, I’m not really sure how it works, I made this script pretty much fast for my daily needs
- Use mk-parallel-dump from the maatkit
- Use .my.cnf, and don’t display password in the script, is it better ?
- please comment to give me some ideas
Permalink