[RFC] Removing hash prefix in storage vastly improves performance

[John Arbash Meinel]

Matthew D. Fuller wrote:
> On Fri, Aug 18, 2006 at 09:31:34AM -0500 I heard the voice of
> John Arbash Meinel, and lo! it spake thus:
>

Thanks for the benchmarks, it really is informative.

...

> So, you're fine with a 50k dir on FreeBSD, and the scaling should keep
> you going just fine to 100k and beyond unless it hits a big knee.
> Still, I'd be a little nervous about saying "Every filesystem works
> fine with 100k files in a dir, so we'll do with it".

Well, we can say "If you are working on a tree with 50K files, use a
filesystem that handles 100K files in a dir". :)

I don't really expect people to check out Kernel trees (or the FreeBSD
ports tree) onto a FAT16 or FAT32 filesystem.

But yes, we would want to scale reasonable well to large trees on
mediocre filesystems.

But we already have that ability by just simply having 2 repository
formats. If you use filesystem X use repository format Y.

Your tmpfs tests reminded me that you can use tmpfs, and create a large
empty file in it, and remount that in loopback mode as another
filesystem. (yeah, sure its ugly, but it should work).

So I'll setup some tests for large numbers of files in one directory,
and report the results.

John
=:->


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060818/358a89a8/attachment.pgp