RFC: CachingLocalTransport plugin

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I just put together a quick plugin, available from:
  lp:~jameinel/+junk/mmap_local_transport

Basically, it just installs a handler for "file://" urls.

It only overrides readv(), and can either just cache the files after
opening them, or can go the next step and mmap the files for returning
readv() requests through mmap[start:end] rather than f.seek(start),
f.read(length).

It does have a noticeable effect on 'bzr log' performance, though we
should be fixing that in different ways. But basically, when you are
accessing the index for lots of keys, it has to re-open the .rix file
for each readv() request.

The times are basically:

  bzr log --limit 10 --short
   6.883s
  Caching open() objects
   6.462s
  mmap()
   6.306s

So you get about 400ms saved for caching open() calls, and another 160ms
for using mmap.

I'm not suggesting we include that into core, though caching just open()
calls is probably safe.

The big thing is that if you want to use mmap, you can just install this
plugin... and there you go.

I mostly wrote it because I was wondering what actual overhead we were
encountering, which seems to be a bit, but not enormously bad. At least
not versus the other costs just associated with traversing too much of
the graph.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFH4CquJdeBCYSNAAMRAnJNAJ9MOF+/iPq89/rl2+ImCb4HZbDGygCXYuV7
nHN4WbrKtxrR6/tUPp8F+g==
=fZAj
-----END PGP SIGNATURE-----