[MERGE] Use a LRUCache in LocalTransport.clone

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Bennetts wrote:
> This is a simple change to use a LRUCache in LocalTransport.clone.  It seems
> that calculating an abspath can be quite expensive.

Yes, I've also observed that abspath takes a long time for the
find_branches API.  It doesn't seem really reasonable.

> By keeping the LRUCache I
> reduce the time to pull a 300 revisions from a knit repo into a pack repo from
> ~32 seconds to ~26 seconds!

Nice.

> Specifically, I'm testing with a Twisted trunk import made with bzr-svn which is
> in “Bazaar Knit Repository Format 3 (bzr 0.15)” format, and timing using this
> simple-minded command:
> 
>     time (bzr init --rich-root-pack foo && cd foo && bzr pull -r 300 /tmp/twisted-trunk)
> 
> (So it's also building the working tree, so the 32s vs 26s difference slightly
> understates the relative improvement).

For benchmarking purposes, it might make sense to create a treeless
branch, e.g. by sticking a no-trees shared repo above foo.

> Or perhaps clone should just be faster; by
> calling self.abspath it invokes local_path_to_url, and in constructing a new
> LocalTransport it also invokes local_path_from_url... it seems like one of those
> ought to be unnecessary in an ideal world.

I think clone ought to be much, much faster.  We shouldn't pay a huge
performance penalty for doing things the "right" way.

> +            try:
> +                t = _local_transport_lru[key]
> +            except KeyError:
> +                abspath = self.abspath(offset)
> +                if abspath == 'file://':
> +                    # fix upwalk for UNC path
> +                    # when clone from //HOST/path updir recursively
> +                    # we should stop at least at //HOST part
> +                    abspath = self.base
> +                t = LocalTransport(abspath)
> +                _local_transport_lru[key] = t
> +            return t

It seems to me like our use cases for LRU caching are going to be pretty
redundant and boring.  So what about a decorator?  e.g.

@with_lru_cache(max_size=6000)
def _abspath():

Also, depending on the hit/miss ratio, dict.get may be a better choice
than dict[] and catching exceptions.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHfnZY0F+nu1YWqI0RAnvRAJ9GHGCqpfIU1Gi2Kd/kK+TM4MC/KACcC+hh
PuyKBR6O6CdWlPCQYROnycw=
=wRTL
-----END PGP SIGNATURE-----