Imminent MERGE request: 12 steps towards a high performance server

[Andrew Bennetts]

As Robert recently posted, we've got a fair few infrastructure changes in
support of the work on the high performance server.  There are lots of mostly
independent changes, so to ease the reviewing load I've split them into seperate
branches.  These branches are essentially a sequence of patches to be applied;
I think I'll post bundles for review in individual mails (so that each change
gets its own thread for discussion), but first here's an overview of what's they
all are, in order:

   http://people.ubuntu.com/~andrew/bzr/cosmetic/
     purely cosmetic changes -- docstring tweaks and the like.  Hopefully
     nothing controversial :)

   http://people.ubuntu.com/~andrew/bzr/sort-os.listdir-in-add.py/
     a one-line change to smart_add_tree to ensure a consistent ordering -- I
     recently enabled the 'dir_index' option on my ext3 filesystem, and it broke
     the test suite without this fix.

   http://people.ubuntu.com/~andrew/bzr/open_from_transport/
     add an open_from_transport method to bzrdir.py

   http://people.ubuntu.com/~andrew/bzr/BzrNewError.__str__/
     make BzrNewError.__str__'s error handling a bit more robust and bit more
     informative.

   http://people.ubuntu.com/~andrew/bzr/preserve-transport-when-creating-bzrdir/
     fix, with test, to ensure make_bzrdir preserves the transport

   http://people.ubuntu.com/~andrew/bzr/workingtree-creation-bug-fix/
     fix, with test, to how make_branch_and_tree makes workingtrees

   http://people.ubuntu.com/~andrew/bzr/extra-transport-tests/
     this adds a bunch of tests for transports, particularly about how the path
     handling at the root of a url works -- and fixes the bugs they reveal.

   http://people.ubuntu.com/~andrew/bzr/lock_read-lock_write-raise-TransportNotPossible/
     allow lock_read and lock_write to raise TransportNotPossible.  This branch
     accidentally has some refactoring of url handling in transports in it
     (possibly really belonging to extra-transport-tests?) too.

   http://people.ubuntu.com/~andrew/bzr/TestCaseWithTransport.get_url-fix/
     a small fix to TestCaseWithTransport.get_url

   http://people.ubuntu.com/~andrew/bzr/start-and-stop_bzr_subprocess/
     Add start_bzr_subprocess and stop_bzr_subprocess to allow test code to
     continue running concurrently with a subprocess of bzr.  This is useful for
     some tests in the next branch...

   http://people.ubuntu.com/~andrew/bzr/add-get_smart_client/
     Adds support for bzr:// and bzr+ssh:// urls, adds a new 'bzr serve'
     command, and adds a Transport.get_smart_client method.  This includes a
     SmartTransport, that performs file operations over the RPC added in this
     branch. 

   http://people.ubuntu.com/~andrew/bzr/remote-bzrdir/
     Adds a RemoteBzrDir class, etc, that simply proxies methods through to a
     real underlying bzrdir, and hooks the RemoteBzrDir and SmartTransport into
     the test suite.

Remember, each branch builds on the previous branch -- e.g. if you diff
remote-bzrdir against bzr.dev, you'll see *all* these changes combined, so if
you review these, you probably want to just review the diff against the previous
branch in this list.

It's likely that some of these changes aren't perfectly seperated, but it should
be easier to review than one big lump anyway.  It's possible that a couple of
the intermediate branches don't pass tests, although I have tried to check that
each branch makes a reasonable amount of sense on its own, it's very
time-consuming to do thoroughly.

Please feel free to start reviewing these branches.  I'd like to get as much of
this as possible into 0.11.

Seeking +12 ;)

-Andrew.