limits , open files, pam, daemons and scalability

Be carefull, some daemons have some extensive activity !
It depends on the daemon (its goal, its activity) but some opens a lot of files, some uses a lot a network connections, some both of them (and some takes a lot of cpu , others a lot of ram) …
Anyway, on unix any process is limited by the system with the ulimit mechanism, and one of these “annoying” limits is the maximum number of file descriptor , that is the maximum number of open files a single process can use including network connections (on unix, a network connections has a file descriptor associated to it).

Depending on your daemon and your case, you sometimes need to get rid of these limits (be sure you need to get rid a of these limits). To do so , on a ubuntu , I have:

  • Added the following to /etc/security/limits.conf:
    nobody soft nofile 4096
    nobody hard nofile 63536
  • Added the following to /etc/pam.d/su:
    session required pam_limits.so

Here, I have added to the user nobody the right to open 63536 files PER PROCESS (yeah, per process, not for the user).
I added this thing to /etc/pam.d/su because when you run a daemon from an init script, it uses the su mecanism to run as the user it’s configured for.
I have added the user nobody because unfortunately , my running daemon is memcached which runs as nobody (should probably run as memcache user).

Be carefull, sometime you think you need this, but you don’t: I wanted to increase the number of files apache user can open, but it wasn’t needed: my apache was running with forks (because php needs fork), any new connection spawns a new process which has it’s 1024 file descriptor, the whole apache on the system was opening lot more than 1024 files, as I had a limit of 1100 forks running which I reached several times, but a single process wasn’t opening more than 1024 files.
You can check how much a process of files opened with lsof (which stands for LiSt Open Files):
lsof -p 3423 -n |wc -l where 3423 is the PID of the process you want to inscpect ( “-n” means don’t do dns resolution, lsof lists sockets and tries to put names to every address, a server process generally accepts a lot of connection from a lot of different hosts. Without “-n” , lsof is really slow )

That’s it, I hope yo uwill make a good use of these tips ! 🙂

Advertisements

1 Comment »

  1. carlos said

    great, thank for your help 😀
    very thank brother

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: