read() wont allow me to read files larger than 2 gig (on a 64bit)

Andy Whitcroft apw at canonical.com
Wed Dec 30 09:39:07 GMT 2009


On Mon, Dec 28, 2009 at 11:19:08AM +0100, glide creme wrote:
> Thanks for all your replies,
> but it seems that its not possible to use posix read to read file
> chunks larger than 2.1 gig.
> This is according to the linus
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=e28cc715
> 
> At somepoint I guess it was possible but the kernel has regressed to
> only allow smaller chunks being read.
> 
> kernelsourcetree/fs/read_write.c
> ---------------------------------------------------------
> /*
>  * rw_verify_area doesn't like huge counts. We limit
>  * them to something that fits in "int" so that others
>  * won't have to do range checks all the time.
>  */
> #define MAX_RW_COUNT (INT_MAX & PAGE_CACHE_MASK)
> int rw_verify_area(int read_write, struct file *file, loff_t *ppos,
> size_t count);
> ---------------------------------------------------------
> This was rather annoying, it would have been nice if this had been
> documented outside of the kernelsource and kernel git.
> 
> I'm aware that the returnvalue from the 'read' function can be smaller
> than the number of bytes expected.
> 
> But I think it should be clarified in the documentation, that the
> function can't return a value larger than 2147479552.

The problem with that is that the standard says those values are valid.
The standard says that read will return between 0 and the number of bytes
you asked for.  That means that in the future it could return more than
2GB, to document that it could not would allow you to make assumptions
about the return value which may not be correct in the future.  Bad.

-apw



More information about the kernel-team mailing list