Posts Tagged system
May 13, 2009 at 9:18 am
· Filed under Uncategorized ·Tagged admin, debug, network, networking, networks, reverse engineeering, script, scripting, scripts, security, shell, shell script, socket, sysadmin, system, system administration
I felt I needed to write an article about netcat, so here is it !
Netcat is an incredibly usefull tool, that allows you to play with tcp connection easily from the shell.
Basically, as it name implies, it’s just cat over the network, but what its name doesn’t tell you is that it also can act as a socket listener.
So let’s play with pipes, here is one of my favourite use of netcat:
mkfifo proxypipe
cat proxypipe | nc -l -p 80 | tee -a inflow | nc localhost 81 | tee -a outflow 1>proxypipe
This command will redirect traffic from localhost:80 to localhost:81, in the inflow file you while find the incoming http request, in the outfile, you will find the http response from the server.
Similarly, you can do this:
cat proxypipe | nc -l 80 | tee -a inflow | sed 's/^Host.*/Host: www.google.fr/' | nc www.google.fr 80 | tee -a outflow >proxypipe
This will allow your browser to point to google using http://localhost .
Anyway, this is my favourite but netcat has thounds of other uses, have a look at it !
It can be usefull for file transfers (gzip|nc) , performance measurement (dd|gzip), protocol debugging (replaying requests), security testing (nc does port scan) …
Permalink
April 16, 2009 at 4:05 pm
· Filed under Uncategorized ·Tagged admin, deployement, install, linux, pm, project management, redmine, script, scripting, scripts, shell script, sysadmin, system, system administration, tip, tips, tips and tricks, tools, ubuntu
I’ve successfully insalled redmine pretty much easily but I needed to find out what packages to install with apt, which one with gem, which version …
Here is my magic receipe to install it all:
apt-get update
apt-get install subversion mysql-server rubygems rake pwgen
# next line generates a password for the database
export PASSWORD=`pwgen -nc 8 1`
gem install -v=2.1.2 rails
echo "CREATE DATABASE redmine DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci ; GRANT ALL PRIVILEGES ON redmine.* TO 'redmine'@'localhost' IDENTIFIED BY '$PASSWORD' WITH GRANT OPTION; FLUSH PRIVILEGES" | mysql
cd /opt/
svn export http://redmine.rubyforge.org/svn/branches/0.8-stable redmine-0.8
cd redmine-0.8/
cat <<EOF >> config/database.yml
production:
adapter: mysql
socket: /var/run/mysqld/mysqld.sock
database: redmine
host: localhost
username: redmine
password: $PASSWORD
encoding: utf8
EOF
rake db:migrate RAILS_ENV="production"
rake redmine:load_default_data RAILS_ENV="production"
apt-get remove pwgen subversion
RAILS_ENV="production" ./script/server
And that’s it ! Redmine is running on port 3000.
I did this on an EC2 instance and it works like a charm (ami-7cfd1a15).
Maybe next article will discuss running redmine in mongrel or apache, and creating an init script for having redmine running on boot !
Permalink
April 14, 2009 at 3:20 pm
· Filed under Uncategorized ·Tagged admin, dns, sys, sysadmin, system, system administration, tips, tips and tricks
RFC 952 and RFC 1123 explains the rules for choosing a hostname. I noticed recently that a lot of admins (including me) are using underscores in hostnames, but this doesn’t follow RFCs. This can lead to strange behaviours, such as mail not delivered with an RFC compliant mail server to an MX that have underscores in its name …
I noticed that because the “hostname” command on linux can set the hostname of a system, but the command doesn’t accept underscores. So guys, don’t use underscores !
Permalink
November 18, 2008 at 2:58 pm
· Filed under Uncategorized ·Tagged cluster, network, networking, servers, ssh, sys, sysadmin, system, system administration
With pssh (parallel-ssh) you can execute the same command on different hosts.
Pssh is a simple python script, the uses pretty much no python module, so it’s simple to install (it’s also packaged at least in ubuntu).
To use pssh, you need to create a hosts file which contains a list of hosts (one by line) followed by a username to use on that host, then just execute this command parallel-ssh -h hosts-file "command", it will execute “command” on all the hosts that are in the given hosts-file. I copied my ssh public-key so I don’t need to type my password on any server, if you don’t have your key, pssh will prompt for a password.
Pssh has a --print option that prints the output of the command execution, host by host, on the shell you’re launching pssh from, if you don’t use that option , it creates 1 file per host with the result.
Pssh is really nice, but, would be better if I could use the aliases I use in my .ssh/config for hostnames in my hosts-file. Maybe one day, I’ll make a patch to pssh so it uses your .ssh/config to recognize hosts and users in your hosts-file. Nice tool, anyway !
Permalink
October 28, 2008 at 2:21 pm
· Filed under Uncategorized ·Tagged backup, database, databases, dba, mysql, mysqldump, script, scripting, shell script, shell scripting, sql, sysadmin, system, system administration, tips and tricks
Today I encountered a problem: I needed to restore a single table from a database mysqldump.
Usually you cat the-mysqldump.sql |mysql the_database so you’re only able to restore the full database. I didn’t find any mysqldump option to extract a single table from a full database dump, so I’ve come up with this (minimal) shell script:
#!/bin/sh
extract_table(){
TABLE=$1
DUMPFILE=$2
grepstr="/*!40000 ALTER TABLE \`$TABLE\`"
lines=`grep -n "$grepstr" $DUMPFILE |cut -d":" -f1`
lines=`echo $lines|sed 's/\ /,/' `
echo "LOCK TABLES \`$TABLE\` WRITE;"
sed -n "$lines p" $DUMPFILE
echo "UNLOCK TABLES;"
}
extract_table $1 $2
Use it like this:
./this-script.sh table-to-extract dumfile-for-extract |mysql the_database (use the |mysql after you have checked the content).
Be carefull, this script is minimalistic:
- It doesn’t check if the file exist and is really a mysqldump file
- It doesn’t check if the table to extract exists
- It doesn’t work if disable-keys is set to false in mysqldump
- It doesn’t have a usage() function
If some people request it, I’ll write all these features, but as usual, I wanted to come up with a solution I could already use one hour a ago, and I’m spending time to write this script, let’s do it the faster I can ! 
Permalink
October 22, 2008 at 10:54 am
· Filed under Uncategorized ·Tagged linux, mail, network, networking, smtp, ssmtp, system, unix
Unix systems often needs a local mailer, but configuring and maintaining a mailer on each system is a timeloss.
You might have a gmail or google apps account. If it’s the case, you can easily configure a mailer on your systems which uses your gmail or google apps. To do so, I’ve used ssmtp and put this in /etc/ssmtp/ssmtp.conf:
root=postmaster
mailhub=smtp.gmail.com:587
AuthUser=your-mail@yourdomain.com
AuthPass=aStr4angeP45s
UseSTARTTLS=YES
hostname=the-hostname
That’s it, simple, effective, working …
To improve the things, maybe, we can use an IP address of the smtp server, so that if our DNS server doesn’t work, we still have mail on the system, but this has a drawback, if the server for which you gave an ip address changes or temporarly doesn’t work, you don’t have mail anymore.
ssmtp doesn’t seem to be able to have several mailhubs !
Permalink
October 16, 2008 at 9:53 am
· Filed under Uncategorized ·Tagged admin, commands, linux, process, shell, sysadmin, system, system administration, unix
I just discovered the watch command, it can be useful !
If you don’t know watch, it does what you would do like this:
while true ; do "your command" ; sleep 1 ; clear ; done
that is, it executes in a while loop the same command , with a sleep so that it doesn’t overkill your cpu.
It also has nice parameters, for exemple --differences that can only show the differences between current and last run.
“your command” could be a du or a df , --differences could be useful when used with an ls to monitor a directory …
Read the manpage and have fun !
Permalink
October 13, 2008 at 5:08 pm
· Filed under Uncategorized ·Tagged backup, dd, disk, dump, network, networking, pipe, ssh, system
If you want to dump a disk to a disk image you will use for example:
dd if=/dev/hdx1 of=/tmp/disk.img
and then, you will probably copy this disk image to another machine. The thing is, if you have low disk space than the size of /dev/hdx1 on your machine, you won’t be able to dump the disk to transfer it to the other machine.
There is a solution that I use, as usually with, ssh and pipe:
ssh hostname "dd if=/dev/hdx1" |dd of=/tmp/disk.img
on the machine receiving the image or
dd if=/dev/hdx1 |ssh hostname "dd of=/tmp/disk.img"
on the machine sending the image, so the content of the disk is directly transmitted through ssh !
That’s it !
Maybe you can tune the blocksize of the dd command so the troughput is better, maybe a futur article on that
Permalink
October 11, 2008 at 11:22 am
· Filed under Uncategorized ·Tagged browser, ip, ipv6, network, networking, scp, ssh, system, url, urls, web
URLs are written like this: protocol://host-or-address:port/path-or-function
What happens with ipv6 is that addresses contains colons (“:”) , so how do you specify the port number in your web browser ? The same happens when you do an scp: you usually do scp user@host:path/to/file /local/path, how can you differenciate the host part and the path which are also seperated with a colon ?
The answer is: USE BRACKETS !
an ipv6 url can be written like this:
http://[fe80::abcd:abcd:abcd:abcd]:8080/index.html
Also, a scp command with ipv6 addresses can be like this:
scp user@[fe80::abcd:abcd:abcd:abcd]:/etc/resolv.conf /tmp
I hope it’s usefull !
Permalink
October 6, 2008 at 4:12 pm
· Filed under Uncategorized ·Tagged icmp, linux, mac, network, osx, ping, scan, script, scripting, scripts, shell, subnet, system
Today I logged in a machine I don’t want to install anything on it, but I wanted to find a machine in its network.
I came up with the little shell script that scans the subnet:
CURR=1
SUBNET="192.168.0"
while [ $CURR -lt 255 ] ; do
ping -c1 -t1 $SUBNET.$CURR 2>&1 >/dev/null
if [ "$?" -eq "0" ]; then
echo "$SUBNET.$CURR"
fi
let CURR=$CURR+1
done
This script is suboptimal but it does the stuff: It uses ping with a timeout of 1 sec, so If no machine is up, the script takes around 255 seconds to scan the subnet, it doesn’t list the machines that doesn’t reply to ping and so on … but as I said it , it does the stuff.
I tested this script in Linux and OSX.
Permalink
Older Posts »
