Fresh results for Benchmarking page

[Kannan Goundan]

Please disregard the "slowness of 'bzr diff'" part of my e-mail.  For
some reason the machine I used is unusually slow (it claims to have a
3GHz Xeon processor; maybe the hard drive is really slow?)  I tried
the same thing on another machine and bzr finished within a second on
the largest repository.

I'm still interested in updating the Benchmarks page with the new size
results.  Please let me know how I should go about doing this.

-- Kannan

On Fri, Apr 16, 2010 at 15:33, Kannan Goundan <kannan at cakoose.com> wrote:
> The benchmarking page is a bit out of date
> (http://wiki.bazaar.canonical.com/Benchmarks).  It uses slightly old
> version of bzr, hg, and git.  I have a script that runs the tests from
> the first table (hg repos).  I also added in a git run for those
> tests.  The results:
>
>  Bazaar (bzr) 2.1.1
>  git version 1.7.0.5
>  Mercurial Distributed SCM (version 1.5.1)
>
>  Repository Sizes
>  | hg    | git   | bzr   | bzr/s |
>  --------------------------------------------------
>  |   11M |  3.9M |  4.2M |  4.1M | PIDA IDE (trunk)
>  |   56M |   25M |   27M |   25M | Thunderbird 3.0
>  |   86M |   51M |   64M |   64M | MoinMoin 1.8
>  |  128M |   48M |   69M |   72M | Xen 3.4-testing
>  |  316M |  122M |  158M |  194M | Firefox 3.5
>
> Unrelatedly, the script also runs "<scm> diff" to see how long it
> takes on the various repos.  I ran these tests on a machine that's a
> couple  years old, but not that old.  Bazaar is pretty slow here.
> Arguably slower than "fast enough".
>
>  Diff Command Running Time (seconds; best of 10 runs)
>  | hg      | git     | bzr     |
>  ------------------------------------------------
>  |  13.338 |   8.678 |  53.928 | Firefox 3.5
>  |   0.757 |   0.359 |   3.402 | MoinMoin 1.8
>  |   0.530 |   0.086 |   2.652 | PIDA IDE (trunk)
>  |   2.723 |   1.407 |  10.167 | Thunderbird 3.0
>  |   1.278 |   0.581 |   4.955 | Xen 3.4-testing
>
> Any ideas with 'diff' is so much slower?  Mercurial is written in
> Python too so that gives me some hope, though maybe they implement
> this part in C.  I'd like to look into this a bit.  Any hints?
>
> Anyway, my benchmarking script is attached, if anyone is interested.
> Run "bench get" to get all the repositories (takes a while), "bench
> size" to display their sizes, and "bench diff" to display how long it
> takes to run 'diff'.
>
> Unfortunately it's a Bash script.  I just found out about the
> bzr-usertest plugin.  Had I known about it when I started, I probably
> would have used that.
>
> -- Kannan
>