status update on the high performance server and dirstate

[Robert Collins]

So, I thought I'd let everyone know how things are progressing with
these two projects.

Firstly the high performance server. We've broken this into two phases -
a RPC based transport, which allows us to get the infrastructure we need
working well. This is coming up for merge-for-review tomorrow, and all
the pre-requisites for that should be sent in as [MERGE] requests
tonight by Andrew, who has been slaving away at this for a while now.

The first cut at the RPC based transport already supports operation over
ssh, via inetd, and as a standalone server. Hopefully we'll have HTTP
integration done, and documentation for running it with apache in place.

The RPC transport is no faster than HTTP or SFTP transports on its own,
but its directly extensible allowing us to add methods that perform
higher level actions, such as pushing a bundle onto the end of a branch.
It is also structured to support user-space chrooting, so that only a
limited portion of the filesystem is exposed via the server.

So this transport brings in key infrastructure for a smart server: an
extensible protocol that can be invoked by users over ssh with no
configuration needed on the server, or as a direct internet facing
service, or tunneled fairly efficiently via http. 

Everything going well this will be in 0.11 and we can have solid user
feedback on it for 0.12. We're hoping to deploy this onto the Launchpad
supermirror - bazaar.launchpad.net - shortly after 0.11 is released.
This will be an important milestone, because it will prove the internal
architecture of the protocol is compatible with the various systems that
make up the supermirror - and that means that when the higher level
server is merged into bzr, it can be deployed on the supermirror too.

Feedback from anyone that would like to have this running now - it
passes all the transport tests so should be safe to run *but not as an
internet facing service*. That is, run it via tcpwrappers, or via ssh
(using bzr+ssh urls), or protected by iptables rules. Before the merge
to mainline we'll be doing an audit of the key code paths to be sure
there are no gotchas or loopholes that have been missed during
development, but we have not done that audit *yet*, so this is a
sensible precaution.

The branch with the fully operational RPC transport is:
http://people.ubuntu.com/~andrew/bzr/add-get_smart_client/

The higher level operations branch, which is targeted at 0.12 at this
point is available too, though its not finished yet - you can see the
shape of the code changes that are being prepared by diffing it against
the RPC transport branch:
http://people.ubuntu.com/~andrew/bzr/remote-bzrdir/

Again, all tests are passing in this branch. Over the next month we'll
be converting all the transport based operations of RemoteBzrDir,
RemoteRepository etc into semantic operations via the RPC transport.
This will start to deliver the speed improvements we desire for branch,
checkout, merge etc.

If anyone wants to jump in and start helping out, please say so, and
Andrew and I will break up the task list - the remaining work is
probably quite parallelisable now that the core infrastructure is done.

And what of dirstate? Dirstate is coming along nicely. There has been a
lot of refactoring of other code paths in bzr to prepare for it, which
is still ongoing, but the bulk is complete. dirstate wont be ready for
0.11 but hopefully will be for 0.12. Right now Andrew and I are engaged
in a mini-sprint focusing on delivering the RPC transport for 0.11, as
the remote-bzrdir branches improvements will scratch a much more visible
itch for most users than local performance, so validating the substrate
works, and getting the delta between bzr.dev and the remote-bzrdir
branch under control is very important.

Cheers,
Rob


-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060913/7fdfd1d6/attachment.pgp