Bazaar 1.6 released - some benchmarks.

[Nicholas Allen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
That is disappointing! I wonder why these performance regressions were
not noticed automatically by performance tests - I thought that
automatic performance tests were being run on Bazaar to prevent this
kind of thing from happening....

Nick

David Ingamells wrote:
> I have to say that my first experience with bzr 1.6 performance is
> very disappointing. I observe times more that 300% slower than bzr
> version 1.5 (which itself showed no improvements over bzr 1.2).
> Checkout --lightweight is, in some cases, even worse.
>
> Working with a repos with only a few revisions (33) but LOTS of
> files:
>
> cmsadmin at mscvs01:/home/CmsRoot/repos/BSW/12.1$ bzr info Repository
> branch (format: pack-0.92)
>
> I timed a bzr+ssh:// branch with versions 1.2, 1.5 and 1.6. Both
> source and destination disks were local to their machines.
>
> bzr_1.2: ===== david.ingamells at msdes002: time bzr branch
> bzr+ssh://mscvs01/home/CmsRoot/repos/BSW/12.1 branch_1.2 Branched
> 33 revision(s).
>
>
>
> real    2m45.985s user    1m14.725s sys     0m55.023s
>
> bzr 1.5: ===== david.ingamells at msdes002: time /data/id/bin/bzr
> branch bzr+ssh://mscvs01/home/CmsRoot/repos/BSW/12.1 branch_1.5
> Branched 33 revision(s).
>
>
>
> real    2m48.375s user    1m13.581s sys     0m56.372s
>
>
> bzr 1.6: ===== david.ingamells at msdes002: time bzr branch
> bzr+ssh://mscvs01/home/CmsRoot/repos/BSW/12.1 branch_1.6 Branched
> 33 revision(s).
>
>
>
> real    10m31.409s user    5m2.215s sys     5m16.720s
>
> bzr 1.6 stacked ========== david.ingamells at msdes002: time bzr
> branch --stacked bzr+ssh://mscvs01/home/CmsRoot/repos/BSW/12.1
> branch_1.6_stack using
> <bzrlib.repofmt.pack_repo.RepositoryFormatKnitPack5 object at
> 0x876decc> for stacking Created new stacked branch referring to
> bzr+ssh://mscvs01/home/CmsRoot/repos/BSW/12.1/.
>
>
>
> real    17m21.220s user    8m12.383s sys     8m56.226s
>
> Similar timings with the file:// protocol and checkout
> --lightweight gave:
>
> pack-0.92 branch bzr+ssh://   branch file://   checkout
> --lightweight bzr+ssh:// =================   ==============
> ================================= bzr 1.2            2m45.985s
> 1m6.795s         3m0.301s bzr 1.5            2m48.375s
> 1m3.309s         3m7.716s bzr 1.6            10m31.409s
> 1m33.950s        17m44.029s bzr 1.6 stacked    17m21.220s
> 1m20.099s With both protocols bzr 1.6 is for my repositories a
> retrograde release as far as branch performance is concerned.
>
> I repeated the same tests with another repository having lots of
> history (3644 revisions) but fewer files and with dirstate-tags
> format:
>
> dirstate-tags branch bzr+ssh://   branch file://   checkout
> --lightweight bzr+ssh:// =================   ==============
> ================================= bzr 1.2            1m0.376s
> 1m1.186s         0m37.084s bzr 1.5            1m52.103s
> 1m58.407s        0m37.400s bzr 1.6            4m8.584s
> 2m19.070s        1m4.605s bzr 1.6 stacked    1m11.704s
> 1m2.370s The branch sizes are:
>
> $ du -sk * 513384  branch_1.2 513380  branch_1.5 513380  branch_1.6
>  296960  branch_1.6_stack 285328  toto_1.2 285324  toto_1.5 244948
> toto_1.6 97140   toto_1.6-stack
>
>
> Given the performance of bzr 1.6 with pack-0.92 I feel very cynical
>  about the advice produced when I ran the bzr 1.6 branch command on
>  the dirstate-tags repos - it was so important it had to be
> repeated! (with the --stacked option there were 5 messages such
> produced).
>
> Format <RepositoryFormatKnit1> for
> chroot-140730668:///home/CmsRoot/repos/CVTOOL/.bzr/ is deprecated -
>  please use 'bzr upgrade' to get better performance Format
> <RepositoryFormatKnit1> for
> bzr+ssh://mscvs01/home/CmsRoot/repos/CVTOOL/.bzr/ is deprecated -
> please use 'bzr upgrade' to get better performance
>
> From my measurements "better performance"is only notable by its
> absence. Admittedly it looks like 1.6 branch --stacked will be
> faster provided there is lots of history in the repos that can be
> left behind. Also --stacked with such a repos saves lots of disk
> space.
>
> However the bottom line is that version 1.6's branch with pack-0.92
>  and bzr+ssh:// protocol is about 375% slower than in version 1.5
> and 382% slower than version 1.2. With file:// protocol the figure
> is 221% compared to bzr 1.5.
>
> I ran these tests with lots of hope for improvements, and really
> wanted to be able to send good news. I would love it if someone can
>  point out a stupid error I've made which will turn the figures
> around. Maybe I've misunderstood the intentions of stacked
> branches. I have shown here that they do save lots of disk space,
> but I had also expected performance similar to bzr 1.5's checkout
> --lightweight. I have shown that branch --stacked and checkout
> --lightweight are similar in version 1.6 (which makes sense), but
> why are all the timings much slower than their equivalents in bzr
> 1.5???
>
>
>
> John Arbash Meinel wrote:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>
>> Finally, the long awaited bzr 1.6 has been released.
>>
>>
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
iD8DBQFItXIIbpmWsXfOU58RAoAGAJ9/lUVF+RfbSDX3lOGa7BEvdHqpLwCfdkPw
eMJXjzKHxqqe7MOu/VrrVWI=
=678A
-----END PGP SIGNATURE-----