RFC: startup time - again

[Robert Collins]

On Tue, 2008-09-09 at 21:25 +0900, David Cournapeau wrote:
> Robert Collins wrote:
> > So, Martin and I just had an interesting chat. We were talking about
> > startup time and lazy_import etc.
> >
> > startup time, such as 'bzr rocks' isn't a very useful metric for 'how
> > slow is bzr to get going'. Much more useful is running a command with
> > little work, and the same command with lots of work.
> >
> >   
> 
> I don't understand why bzr rocks is not a very useful metric: on my
> workstation, it takes 0.3 s for bzr rocks, to do nothing. This seems to
> be the core issue for short time commands: what bzr rocks loads is a
> subset of what is needed by 'normal' bzr commands, right ? If bzr rocks
> was say 0.05 seconds instead (on my workstation, a doing nothing python
> script is 0.02s), all the other commands would also be faster, no ?

No :). bzr rocks establishes a lower bound on all commands - yes thats
true. But the lower bound is established by not loading any of the code
that other commands use. So all its showing is the time to:
import bzrlib.commands
import bzrlib.builtins
run bzrlib.builtins.cmd_rocks

tuning bzr rocks to be fast is not as useful as tuning bzr st to be fast
on an empty tree.

> I am not familiar with a lot of bzr code, but I am a bit surprised that
> most of the time is in bzr code itself. For example, we recently boosted
> import time performance for numpy, which is a big piece of code (with a
> lot of C too; I think we have much more files and code than bzr), and it
> loaded faster than bzr even before the boost (we went from 0.2 -> 0.1
> seconds, again on the same machine). The problem was that some modules
> in the stdlib are extremely slow to load (for example, I remember
> inspect to be really slow: importing inspect take twice as much time as
> launching python itself).

Yah, we used to monkey patch re to create a default lazy-compile, so
that expensive 'static' compiled re's in the std lib wouldn't penalise
us. John seems to be sayings that thats horked on 2.5.

> Do you have a list of the modules from the stdblib loaded by bzr (for
> example, by bzr rocks) ? By quickly grepping into the bzr sources, I
> could get a python script which does nothing but importing the modules I
> got, and it already takes 1/3 of the time taken by bzr rocks.

As John mentions, the --profile-imports global option is very useful for
examining this.

-Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080910/e1477947/attachment.pgp