Mercurial and Bazaar SSH Speed

[A. S. Budden]

Note: copying to the Bazaar mailing list for information.

I've been doing some more extensive testing of the differences between
Mercurial and Bazaar for SSH repository access.  For those that didn't
read the original thread (titled "Pushing over SSH" on the Mercurial
mailing list), the background to this is that I've been investigating
Mercurial as an alternative to Bazaar (because of the FishEye/JIRA
support) and one of the things I'm personally interestedin , as opposed to
interested in for work reasons, is performance when sending and
receiving from a remote server over SSH.  The only access to the server
is SSH, no web server or anything like that.  The server runs OpenFSG
and is based on an ARM core.  There is no native compiler, so if the
application version isn't in the package manager, it's difficult to
upgrade.  I am a user who likes command line access to applications, so
I tend to use cygwin for most things.  Most of my colleagues (who might
also be using the VCS, but probably not over SSH, so these results are
irrelevant to them) like GUIs.  At work, we all use Windows (currently
XP, but soon to change to 7); at home, I use Ubuntu.

I was hoping to include Git in these tests, but since the package server
(ipkg2.nslu2-linux.org) appears to be down at the moment, I can't
install Git on the server and thus it isn't possible to test it.  This
isn't too much of a hardship for me as it is unlikely that I'd choose
Git due to the flaky operation I've experienced on Windows and the lousy
GUIs (which my colleagues like and for which Bazaar has the upper hand
on, at least until the new PyQt version of TortoiseHg comes out).

Having done some quick tests with Mercurial and Bazaar, it was
noticeable that Bazaar was substantially quicker than Mercurial when
doing network operations.  This may be something to do with Bazaar using
SFTP and Mercurial using SSH, but from my point of view that's an
implementation detail and my criteria are that it's an SSH-based
connection and that it works, so SSH or SFTP are fine.  Mercurial
doesn't support SFTP, so I used SSH instead.

Tests were carried out with a repository containing 413 revisions
(containing my ~/.vim directory, including various plugins and a couple
of py2exe'd applications, the Windows versions of ctags and cscope,
etc).  The repository has been revisioned using bzr for most of its
history, with the first 50 or so revisions from its subversion days.  I
can't remember the details of the svn->bzr import, but I created an
equivalent Mercurial repository using "hg convert vimfiles hgvimfiles".
The relative sizes of the various directories are:

           Local Copy (Repo + Working Copy)  Server Copy (no Working Copy)
Bazaar     79MB                              34MB
Mercurial  73MB                              27MB
Git        116MB                             69MB

(Git line just for information and in this case 'Server Copy' is
actually a local
copy without a working copy due to the server issues mentioned above).

Detailed results below, but the executive summary is that Mercurial is
MUCH slower than Bazaar to push this repository over SSH.  It is also
substantially slower than Bazaar to clone this repository over SSH,
although with the Cygwin version and compressed SSH, it takes 'only' 25%
longer.  With the Tortoise-Hg version retrieving over SSH, it takes over
7 times longer than Cygwin Bzr!

All tests are to the same server, with no other server activity going on
at the same time and no significant local host activity.

Results rounded to nearest second:

Name:                       Send Time   Get Time
Cygwin Hg Default:          7m 9s       13m 50s
Cygwin Hg Cygwin-SSH:       7m 11s      14m 3s
Cygwin Hg Compressed-SSH:   5m 37s      5m 25s
Windows Hg:                 7m 11s      13m 31s
Tortoise-Hg:                6m 59s      30m 48s
Tortoise-Hgtk:              7m 6s       29m 54s
Cygwin Bzr:                 1m 37s      4m 15s
Windows Bzr:                1m 38s      4m 27s

Key:

* "Send Time" is time to push/clone a repository from the local machine
  to the server.  "Get Time" is the time to branch/clone a repository
  from the server to the local machine.

* Cygwin Hg Default is the Cygwin version of Mercurial with no
  ssh-related configuration options.

* Cygwin Hg Cygwin-SSH is the Cygwin version of Mercurial forced to use
  the Cygwin version of ssh.exe (times appear close enough to assume
  that this is the same as the default one).

* Cygwin Hg Compressed-SSH is the Cygwin version of Mercurial with the
  (cygwin) ssh command set to "/usr/bin/ssh -C".

* Windows Hg is the version of hg.exe that comes as part of the Windows
  installer for Mercurial.

* Tortoise-Hg is the version of hg.exe that comes with the TortoiseHg
  installer.

* Tortoise-Hgtk is the version of hgtk.exe that comes with the
  TortoiseHg installer.  The button is clicked as soon as possible after
  the dialog box pops up (maybe a few seconds).  It is run with --nofork
  so that the timing works correctly.

* Cygwin Bzr is the version of Bazaar installed by the Cygwin package
  manager.

* Windows Bzr is the version of Bazaar from the Bazaar Windows
  standalone installer.

Full session log (including version numbers) is here for information:
http://pastebin.com/DSsAsAfF

Al