[BRANCH] Transport enables lazier updating

John A Meinel john at arbash-meinel.com
Tue Jul 12 10:15:46 BST 2005


So, now that my Transport layer is up and running:
http://bzr.arbash-meinel.com/bzr-split-storage/

I went ahead and started realizing that bulk updates/generators need to
be propagated further up the chain.

I mean, what good is having a transport layer that allows you to place a
request for 100 files, if the storage layer doesn't know about it, or
the Branch object doesn't do anything either.

Anyway, I'm currently on revision number 946, and I have managed to get
bzr branch sped up by a pretty reasonable amount.
Naturally my --revision namespace code accidentally broke "bzr branch"
(which I have applied fixes for) but using bzr mainline version 896
(just before things broke) it takes real 1m4s user 30s sys 10.1s to copy
944 revisions (1587 texts).

With my current code it takes real 34s user 18s sys 6.3s

Now, these numbers are all only for local filesystem copies. For remote
operations (copying the first 100 revisions from bzr-mainline) my branch
takes real 1m20s user 8s sys 4s, while bzr mainline takes real 1m30s
user 8.4s sys 4.2s

Admittedly over a remote connection Abentley's Caching code has the most
help. *But* my HttpTransport does not do asynchronous downloading
(though in theory urlgrabber at least does keepalive).
But still, there is hope that it can be improved.

Also, I would like to announce that I have implemented at least the
start of an RsyncTransport protocol.
Basically, it keeps a local /tmp directory, and fills it with the remote
entries. Which can then be returned as normal.
It makes the "bzr branch -r 100 rsync://bazaar-ng.org/bzr/bzr.dev" take
20s instead of 1m25s over http.

It is about 50% slower than the LocalTransport I assume that is mostly
because it copies everything out of the way, to copy it back in again.
It doesn't seem to work quite yet as a push target, but pulling seems to
work quite well.
The only downside right now, is that for a lot of the time, I am unable
to give a progress. Since I am batch processing the rsync. I suppose I
could enable verbose rsync, and then parse the output, but that is kind
of a pain.

But it does seem that bzr pull and bzr branch both work with my
RsyncTransport.

John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20050712/bcfc924d/attachment.pgp 


More information about the bazaar mailing list