bzr performance (blitz speed on GNU/Linux compared to M$)

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

dhruva wrote:
> Hi,
> 
> 
> ----- Original Message ----
>>   time bzr co --lightweight bzr.dev test
>>
>> And the resulting times were:
>> 43.2, 5.1, 4.5, 2.7, 2.4, 1.95, 2.6, 12.9, 1.7, 8.0, 2.3, 1.6, 1.6, 1.7
>>
>> So we start with a pretty abysmal time of 43s to create ~900 files. It
>> then drops dramatically down to 5.1s (8x speedup). However, the
>> improvement doesn't stop there, as eventually it gets down to under 2
>> seconds to build all 900 files.
> 
> For any serious timing on windows, we could either use perfmon or timeit (from windows resource kit).
> I will make a separate post on my findings once I get enough samples.
>  To a considerable extent, it appears like an OS specific issue and need OS specific solution. The other post of using
>  a single file (bzr.exe) would improve when anti virus ON-ACCESS scanner is in action. It will start scanning the bzr
> scripts every it is accessed.
> 
> -dhruva

I don't think that using a single "bzr.exe" is going to fix the issue
that I brought up.

Specifically, I always run from source, which means that all the files
are unpacked on disk. And the *startup* overhead for this configuration
is about 200-400ms.

Note that the python interpreter startup overhead on windows is about
100ms. (time python -c '').

Note that on another machine running linux, this is down to about 10ms.
(about 10x faster, just to start the python interpreter).

I don't fully understand the reasons for the startup issues, and
certainly an on-access scanner would make the specific startup time even
more of an issue. However, even if the startup overhead was 1second,
there is still a very large difference between 42s and 1.7s.

Note that on this other machine the "bzr co --lightweight" time starts
at 2.0s and goes down to 0.6s. Some of this is because it is a desktop
versus a laptop, with a better hard-drive, etc.

Regardless, I've certainly still seen a big difference just rebooting my
machine. At this point, the difference isn't enough for me to be
bothered (on a tree the size of bzr, at least).

Maybe something else really weird is going on, but at least trying to
compare the machines:

$ time python -c "y = 0
for x in xrange(1000000):
 y += x
"

On the linux machine, this takes 400ms, on this machine it takes 500ms.
However, this is pretty much just the 100ms startup overhead showing up.
As if I go another order of magnitude I get 3.5s and 4.0s. So while the
other machine is slightly faster at executing additions in python, it is
only just so. (2.4GHz versus 2.2GHz)

So the bulk of the time is spent in the Windows I/O subsystem (for the
slow 'bzr co' times), and I wouldn't mind if someone wanted to spend
some time and try to work out any hints, etc, we could be giving to
improve things.

I do know that we open-and-read the same file multiple times, I don't
think we ever open-and-*write* multiple times.

Also, things slow down at some really weird times. When we are building
a working tree, we stage it in a limbo area. And add entries to that as
we go. When we have finished, for most entries we rename their
containing directory into the final location in the workingtree. I've
seen it where the "creating new file" portion goes fast, but doing "mv
.bzr/checkout/limbo/dir working-dir" my system actually seems to just
hang for about 5-10s not doing anything (that I can see).

Also in general, the CPU time during all of this is practically 0.

Now perhaps my disk is a bit full (the partition is about 80-90% full).
So maybe with multiple runs Windows is able to page in some free-blocks
that get to be re-used over-and-over again. At least until I do
something else, like send an email, and then all of those blocks get
messed up.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmMe58ACgkQJdeBCYSNAAM+oQCgiKObfQlY1AbEqDtJlV8OCLtP
wj8An3K0sWQGkdlr/2fFOzGEaE0vnlX/
=BBSQ
-----END PGP SIGNATURE-----