RFC: startup time - again

[Russel Winder]

Alexander,

On Thu, 2008-09-11 at 19:37 +0300, Alexander Belchenko wrote:

> Here is the numbers for 1.6.1 with and without tests.
> 
> Full bzrlib with tests:
> 
> C:\Temp\bzr-1.6.1\win32_bzr.exe>timeit bzr.exe --no-plugins --no-aliases rocks
> It sure does!
> 
> time: 0.297
> 
> Without tests and benchmarks:
> 
> C:\Temp\bzr-1.6.1-wo-test\win32_bzr.exe>timeit bzr.exe --no-plugins --no-aliases rocks
> It sure does!
> 
> time: 0.281
> 
> So I get about 16 ms win, i.e. >5% speedup. Does it big or small? For me it's enough big.
> There is another 3-5 places where I can get additional 16ms speedup on each.
> So in sum I could get about 80ms speedup. Does it big or small?

I would be strongly tempted to not make any deductions at all from two
data points.  I find that I easily get ±10% variation for the same job
on the same machine just running at different times.   Even if you run
the same job on the same machine just one after the other, I bet the
times are easily going to be ±5%.

> In the same time library.zip without tests/benchmarks is thinner: 11MB -> 7MB
> (in python sources from 9MB of bzrlib about 4.9MB weights tests and benchmarks).
> 
> PS: It seems like my timeit utitlity has the same bug as time.time() on Windows:
> it has precision of 16 ms. I need to rewrite it to have more precise results.
> I'll try to find implementation of time.clock() in Python sources. Anybody could
> give me a hint?

On Ubuntu with Parallel Python or using the processing package (renamed
and made standard as multiprocessing in Python 2.6) I find the
time.clock function generally returns the same value in almost all cases
whereas time.time returns a reasonable value.  So despite the comments
in the manual that time.clock should be used for benchmarking, this is
fundamentally not the case for Python on parallel systems.

-- 
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080911/3feb360c/attachment-0001.pgp