The default file descriptor limit (ulimit -n 1024) is too low

[Scott Ritchie]

On 09/20/2010 05:09 PM, Kees Cook wrote:
> On Mon, Sep 20, 2010 at 03:21:29AM -0700, Scott Ritchie wrote:
>> Would there be any harm in raising this?
>>
>> I ask because I've seen a real world application hit this limit.  The
>> application in question is multithreaded and opens separate threads and
>> files to work on for each core; on a 4 core machine it stays under the
>> limit, while on an 8 core machine it hits it and runs into strange errors.
>>
>> I feel that, as we get more and more cores on machines applications like
>> this are going to increasingly be a problem with a ulimit of only 1024.
> 
> There are some unexpected things that can happen when open fd limit
> (RLIMIT_NOFILE) is raised above 1024, and the main one is that the libc
> select() helper macros don't expect the vectors to ever be more than
> 1024[1] bits long. To quote "man select":
> 
>        An  fd_set is a fixed size buffer.  Executing FD_CLR() or FD_SET() with
>        a value of fd that is negative or is equal to or larger than FD_SETSIZE
>        will result in undefined behavior.
> 
> "Undefined behavior" here mean, "will write all over your stack/heap,
> etc".  While there are defensive workarounds (see Apache, SSH, etc),
> some applications using select can be made to crash if the system
> RLIMIT_NOFILE is larger than 1024.
> 
> So, while it can certainly make sense to raise it for individual processes
> that have been validated to do the right thing with the select macros,
> I would strongly recommend against raising it system-wide until glibc
> fixes this bug[2] and everything that uses the macros has been recompiled
> (or changed to not use select() any more).
> 
> -Kees
> 
> [1] FD_SETSIZE on Linux is 1024:
>     /usr/include/sys/select.h:#define   FD_SETSIZE      __FD_SETSIZE
>     /usr/include/bits/typesizes.h:#define   __FD_SETSIZE        1024
> 
> [2] http://sourceware.org/bugzilla/show_bug.cgi?id=10352
>     But since this is "won't fix", I guess I really just mean "never".
> 

Upon investigation, it seems that there is support for a "soft"
(default) and a "hard" (apps-can-increase-themselves) limit.  Would it
be ok if we raised the hard limit?

-Scott