Notes on smart server network performance: what to do next

Andrew Bennetts andrew at canonical.com
Fri Dec 7 02:08:53 GMT 2007


One area we're trying to improve is the performance over the network when using
the smart server.  Here's a braindump composed from notes I made during a
conversation with Robert recently:

Part of this is reducing roundtrips by adding appropriate verbs to the protocol
(e.g. a single request for "give me the data for this set of revisions").
There's still a lot of room to improve here.  Any place where we're still
issuing VFS (i.e. file-level) requests is a particularly obvious candidate.  For
instance, a verb for reconcile would make reconciling a remote repo much faster
(although at the cost of server load, and also we'd need to figure out how to
report progress back to the client).

So adding more verbs cut down on VFS operations should still be an on-going
goal.

Another thing we can do is stream large transfers, rather than buffer them.
There's some work already landed in bzr.dev to support this for responses from
the server (e.g. when pulling revisions), and I have some work in progress for
the other direction (e.g. when pushing revisions).

An issue here is reducing friction between what's on disk and what's transferred
over the wire.  For instance, when streaming revisions from a pack repository,
the client ideally should not need to do any buffering, it should be able to
just insert the data the from the wire as it arrives.  There's a few parts to
this:

  * transmitting revisions in the "right" order (so that we don't send
    compressed texts before the parent they are compressed against, for
    instance).
  * transmitting revision data in the "right" order: if our wire format assumes
    that we should transfer whole versioned files, then we can't efficiently
    write packs from that stream.
  * keeping the wire streaming format as close to the disk format as possible
    (also to avoid duplicating code).

Finally, we discussed graph operations over the network.  When pushing and
pulling a branch, we first need to figure out the revisions to transfer; i.e.
the graph difference.  Currently we transfer the entire graph so that one side
can do the graph difference locally (and incidentally we give bad progress
reporting in the UI while this happens, and it's one of the first things that
happens in a push/pull!).  Clearly this isn't optimal, and we've already
discussed various interesting ways we can improve this.  A good incremental step
to improving this would be some infrastructure for transmitting graph
"descriptions" over the wire.  For example we probably want to add some smart
verbs roughly like (not carefully thought through):

  * "Give me the graph/revisions from A to X"
  * "I have heads {A, B, C, D}, what do I need to get to X?"

So a standard way to talk about "The graph from A to X" or "The graph from {A,
B, C, D} to X" on the wire could be a useful building block.  An immediate use
would be to make the current streaming pull request much shorter, by avoiding
the need to explicitly list every revision to be fetched.

Another building block that could be useful would be a way to incrementally
retreive data about a graph, for use when walking parents.  To do this
efficiently you'd want to do something like is already implemented in the pack
index bisection code (I believe): when the client/caller asks for "what are the
parents of X, Y and Z?", return that data but also more data for nearby
revisions (the parents' parents, recursively?), up to (say) 64KB worth of data,
and then transmit that.  You can pack a fair bit of "revision A has parents
{B,C}" into 64k, so you can avoid a large number of round trips at low cost with
this approach.  The downside to working on this is that pack repositories
already implement something similiar at the file-level, so the immediate win
here would be quite modest.

-Andrew.




More information about the bazaar mailing list