Best filesystem to use for a specific type of application

[Steve Flynn]

On 13 February 2012 16:20, Rashkae <ubuntu at tigershaunt.com> wrote:
> On 02/13/2012 11:08 AM, Steve Flynn wrote:

> ReiserFS is deceptive.  It was advertised as being very high performance
> with small files.  But that was only taking into account benchmarks that
> favor it.  In my experience with this kind of workload, one very important
> 'benchmark' is the speed in which you can read all files in the order they
> are returned by Readdir.  (The function that lists all files in a
> directory.)  For the reading of small files to be efficient, they have to be
> read in the order they are laid out on the hard drive.  Otherwise, the head
> has to thrash with lots of random seeks, and that will drop your read
> performance to only a few MB/s.

Yup - we've seen that kind of pain already with much smaller datasets
(just over a million files). So much so that we ended up  splitting
the directory structure up into "no more than 1000 files in a
directory" and effectively building a tree structure of directories
within directories to make sure we actually spend time reading the
contents of the files rather than reading the contents of the
directories. Worked OK for the smaller stuff but now I'm looking at 40
times more files so we need a better method.

> I haven't tested it much for this, but I think EXT4 has overcome this
> limitation.  XFS was by far the fastest filesystem I tested for this
> workload (before EXT4 was released.)  Reiser, and EXT3 were practically
> unusable.

Perfect. I'll move those two up my list a bit!

Cheers.

-- 
Steve

When one person suffers from a delusion it is insanity. When many
people suffer from a delusion it is called religion.