Some unscientific timing results (on the Python source tree)

[raindog at macrohmasheen.com]

Were you using the fast smart server for this test?
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: "Paul Moore" <p.f.moore at gmail.com>

Date: Sun, 23 Mar 2008 20:31:27 
To:"Bazaar mailing list" <bazaar at lists.canonical.com>
Subject: Some unscientific timing results (on the Python source tree)


There are now experimental Bazaar and Mercurial repositories for the
Python source tree, so I thought I'd have a look at some timings.
These are completely unscientific, as I do not have sight of how the
repositories were created - however, I believe that both contain the
full Python history (in time at least - the bzr repository contains
trunk, py3k and 2.5, where the hg one only contains trunk).

The Bazaar repository is described at
http://www.python.org/dev/bazaar, and the Mercurial repository is at
http://dev.pitrou.net:8000/cpython/trunk/.

The figures are on my PC (Windows XP Home, 2.4GHz AMD Athlon 64 X2
Dual Core, with 2G of RAM. Mercurial is the latest Windows binary
(version 434139080ed4+20080225) and Bazaar is the 1.2.0 Windows
binary.

Creating a local repository:

On Bazaar, I followed the described procedure:
- Download the precreated repository, 13 min
- Untar the file, 1 min
- Create a local branch, 2 min

On Mercurial, I did a straight "hg clone", which took 5 min.

Trying some basic tasks:

bzr log | head - 24 sec
hg log | head - less than 1 sec

bzr status - less than 1 sec
hg status - less than 0.5 sec

bzr pull (no changes) - 3 sec
hg pull (no changes) - 2 sec

bzr pull (about a day's worth of changes) - varies between 30 sec and
1 min 30 sec
hg pull (about a day's worth of changes) - about 10 sec

bzr branch (within the repository) - 32 sec (branched 38690 revision(s))
bzr branch (to a local branch outside the repository) - 8 min 31 sec !!!!!!
bzr branch (from http to non-repository target, trunk only) - 22 min
51 sec (branched 38683 revision(s))
hg clone (local) - 1 min 42 sec (3750 files updated)
hg clone (from http) - 4 min 15 sec (added 38626 changesets with 80965
changes to 7934 files)

Looking at the branch timings, it's very obvious why the Python Bazaar
repository was supplied as a downloadable archive of a shared
repository. But it does seem to imply a failure in the basic tool if
it cannot be practically used to clone a large project over the
network directly :-(

Size comparisons aren't completely fair, as bzr has a shared repo with
both the trunk and py3k branches, where hg just has the trunk. But to
get a fairer comparison, I took a local bzr branch outside the
repository, and checked its size. The results were

bzr - 262Mb
hg - 140Mb
Subversion checkout - 333Mb

It's pretty clear from this that Bazaar still has a pretty long way to
go to perform as fast as Mercurial. In many ways, I prefer Bazaar, but
the speed difference is serious enough that I don't want to make that
decision just yet. In my view, one key aspect of any VCS is that it
should be very low overhead (fast, and as little additional space as
practical) in order to ensure that there is no question about using it
(yeah, I know I should create a branch, but it's just a quick hack and
taking a backup of the file I'm changing is so much faster...)

I know there's been a lot of work on speed in the run up to 1.0 and
beyond, but there's still more that needs to be done, IMHO.

I hope this is useful. I'm not trying to criticise anyone here, I just
don't want to see progress on speed slow down under a mistaken
assumption that "we're fast enough now".

Paul.