bzr performance (blitz speed on GNU/Linux compared to M$)

[Alexander Belchenko]

John Arbash Meinel пишет:
> dhruva wrote:
>> Hi,
> 
> 
>> ----- Original Message ----
>>>   time bzr co --lightweight bzr.dev test
>>>
>>> And the resulting times were:
>>> 43.2, 5.1, 4.5, 2.7, 2.4, 1.95, 2.6, 12.9, 1.7, 8.0, 2.3, 1.6, 1.6, 1.7
>>>
>>> So we start with a pretty abysmal time of 43s to create ~900 files. It
>>> then drops dramatically down to 5.1s (8x speedup). However, the
>>> improvement doesn't stop there, as eventually it gets down to under 2
>>> seconds to build all 900 files.
>> For any serious timing on windows, we could either use perfmon or timeit (from windows resource kit).
>> I will make a separate post on my findings once I get enough samples.
>>  To a considerable extent, it appears like an OS specific issue and need OS specific solution. The other post of using
>>  a single file (bzr.exe) would improve when anti virus ON-ACCESS scanner is in action. It will start scanning the bzr
>> scripts every it is accessed.
> 
>> -dhruva
> 
> I don't think that using a single "bzr.exe" is going to fix the issue
> that I brought up.
> 
> Specifically, I always run from source, which means that all the files
> are unpacked on disk. And the *startup* overhead for this configuration
> is about 200-400ms.
> 
> Note that the python interpreter startup overhead on windows is about
> 100ms. (time python -c '').
> 
> Note that on another machine running linux, this is down to about 10ms.
> (about 10x faster, just to start the python interpreter).
> 
> I don't fully understand the reasons for the startup issues, and
> certainly an on-access scanner would make the specific startup time even
> more of an issue. However, even if the startup overhead was 1second,
> there is still a very large difference between 42s and 1.7s.
> 
> Note that on this other machine the "bzr co --lightweight" time starts
> at 2.0s and goes down to 0.6s. Some of this is because it is a desktop
> versus a laptop, with a better hard-drive, etc.
> 
> Regardless, I've certainly still seen a big difference just rebooting my
> machine. At this point, the difference isn't enough for me to be
> bothered (on a tree the size of bzr, at least).
> 
> Maybe something else really weird is going on, but at least trying to
> compare the machines:
> 
> $ time python -c "y = 0
> for x in xrange(1000000):
>  y += x
> "
> 
> On the linux machine, this takes 400ms, on this machine it takes 500ms.
> However, this is pretty much just the 100ms startup overhead showing up.
> As if I go another order of magnitude I get 3.5s and 4.0s. So while the
> other machine is slightly faster at executing additions in python, it is
> only just so. (2.4GHz versus 2.2GHz)
> 
> So the bulk of the time is spent in the Windows I/O subsystem (for the
> slow 'bzr co' times), and I wouldn't mind if someone wanted to spend
> some time and try to work out any hints, etc, we could be giving to
> improve things.
> 
> I do know that we open-and-read the same file multiple times, I don't
> think we ever open-and-*write* multiple times.
> 
> Also, things slow down at some really weird times. When we are building
> a working tree, we stage it in a limbo area. And add entries to that as
> we go. When we have finished, for most entries we rename their
> containing directory into the final location in the workingtree. I've
> seen it where the "creating new file" portion goes fast, but doing "mv
> ..bzr/checkout/limbo/dir working-dir" my system actually seems to just
> hang for about 5-10s not doing anything (that I can see).

Don't forget: for mv on windows there is fancy_rename trick is used:
old file renamed to temp name, new file renamed to old name and then
old file with temp name is deleted. Actually 3 IO operations instead of 1
atomic on Linux.

> Also in general, the CPU time during all of this is practically 0.
> 
> Now perhaps my disk is a bit full (the partition is about 80-90% full).
> So maybe with multiple runs Windows is able to page in some free-blocks
> that get to be re-used over-and-over again. At least until I do
> something else, like send an email, and then all of those blocks get
> messed up.
> 
> John
> =:->