open files (was Re: Virtual Hosting)

Top Page
Attachments:
Message as email
+ (text/plain)
Delete this message
Reply to this message
Author: Kevin Buettner
Date:  
Subject: open files (was Re: Virtual Hosting)
On Jan 8, 8:21pm, George Toft wrote:

> > For your application, it sounds to me like you need to increase your
> > per-process open file limit. (You may need to increase the overall
> > system limit too.) I notice that as root, I'm able to do
> > ``ulimit -n 2048'' and then ``ulimit -a'' actually shows that the limit
> > has been increased.
>
> But if you exit that shell and open another, you will find your change
> is gone - ulimit only affects the current and child shells.


True. This means that you either need to change this ulimit value
on a global scale or else find a way to change it in the shell that
invokes apache.

On a Red Hat like system, you may want to take a look at the
invocation of ulimit in /etc/rc.d/init.d/functions. I haven't tried
it, but I think if you change the ulimit command at this point to have
a "-n 2048" (or some such) argument, you'll effectively change the
ulimit for most of the processes on the system.

Of course, there's likely a more subtle way of doing this too.

> > I've done some tests (using calls to pipe()) and have determined that
> > I'm able to create more than 1024 open files when I do this. However,
> > it seems to me that not all functions implemented by glibc will
> > support more than 1024 files. In particular, the select() function
> > relies on the fd_set data structure to know which file descriptors to
> > wait for. It appears to me that you can't use this function for file
> > descriptor numbers past 1023. In /usr/include/bits/types.h (which is
> > included by /usr/include/sys/types.h), I see the following:
> >
> >     /* Number of descriptors that can fit in an `fd_set'.  */
> >     #define __FD_SETSIZE        1024

> >
> > I think you're okay if your application uses poll() instead of
> > select() though.
>
> I'm not into rewriting Apache :)


Hmmm... I just took a look at the Apache source and am quite surprised
to see that it doesn't contain any code to use poll() for those systems
which support it. I would've thought that this would be at least
be a configuration option.

The other approach that you could take is to rebuild glibc with a
different __FD_SETSIZE value. Then rebuild Apache with the new header
files and linked against your rebuilt glibc...

(Yes, this would be a bit of work too.)

Kevin