Best filesystem to use for a specific type of application

[Steve Flynn]

Afternoon all,

I'm looking at a situation where I need to read a lot of small files.
Roughly 40,000,000 files averaging around 35KB each. Some will be
larger and some will be smaller as they are TIFF scans. Not sure of
overall size of the dataset yet but 3 TB feels about right (not got
the data yet so can't tell exactly)

As you're probably aware, very small files are a nightmare for
throughput. Currently, we've been using encrypted external USB
drivesto move this data between ourselves and my clients but now that
the size of the dataset is increasing, it's time to move to something
a bit more robust. I've been looking at a couple of NAS drives to
press into action, some of which give us the option of changing the
filesystem from to something more suitable.

Can anyone point me to some stats for how differing filesystems
(Reiser, XFS, JFS, Ext3, Ext4, BTRfs, etc) stack up against each other
when dealing with a lot of very small files. I have a little bell
tinkling away at the back of my mind that Reiser was particularly good
for small files, but I could well be making that up... plus I don't
know how well that stacks up these days against the advances made in
other filesystems.

Anyone got any empirical evidence or horror/success stories with any
of the available filesystems?

-- 
Steve

When one person suffers from a delusion it is insanity. When many
people suffer from a delusion it is called religion.