[RFC] Removing hash prefix in storage vastly improves performance

[Robert Collins]

On Fri, 2006-08-18 at 10:10 -0500, Matthew D. Fuller wrote:

> Now, for some more useful info.  On a memory-backed filesystem, on a
> FreeBSD -CURRENT system, I can touch 5000 files into existence in an
> empty dir in 31 seconds, and a basic `ls >> /dev/null` takes around
> .05 seconds.  rm -rf'ing it takes less than a second.
> 
> Doing it with 50,000 files takes 5:42 (while fork()ing like a monkey
> on speed, so it's not really a pure filesystem test by any stretch),
> which is roughly linear to the above.  The ls takes around .44
> seconds, so that's linear again.  An ls -l (so it has to stat() 50k+2
> times) burns a whopping ~3.5s.  cat'ing a random file from there with
> a full path takes almost unmeasurable time (.01, the limit of tcsh's
> 'time' precision).

Hmm, I wrote a 'blat-speed-tester' a while back, now where did I put
it ?

(It writes 20K files in a predefined pattern, with subdirs). No forking,
and using standard python calls. Its kind of a 'maximum lower limit'
tester.

It could be nice to write one of those for all the different layouts
we're talking about over the next period, and get people to run them as
a os-analying benchmark.

Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060819/67d967bb/attachment.pgp