bzr-explorer performance question

[Maritza Mendez]

Gary:

I was just starting to reduce my somewhat scattered data to concise notes,
when I read your post.  There is a lot of information, and I want to make
sure whatever I add actually helps.  The qbzr bundled in the stand-alone
2.2.1 installer for Windows is 0.18.4.  You've made improvements since
then.  And a lot of what I have done is wander around looking for patterns.
Now that we think we see some patterns, I think the right thing for me to do
is repeat the init/add/stat+/commit/stat+ test in these ways:

  1. bzr builtin commands
  2. bzr qbzr comands
  3. bzr explorer 1.0.1

under the conditions:

  A. fixed version of bzr (currently 2.2.1)
  B. two versions of qbr: 0.18.4
  C. latest from lp:qbzr

and collecting the callgrind profiles for each.

The bzr builtin test is a one-off, not affected by qbzr.  And there
So there are tests 2 and 3 under conditions B and C, which gives four more
test runs.

I will probably start on this late today.  Any suggestions to improve the
data collection are welcome.

[FYI although I appreciate (and often use!) the ability to show&add
unversioned files in qcommit (great feature!) I actually am very careful
about who I show.  I have some non-developers using bzr to control mark-up
documents.  They don't have a deep understanding of version control (as if I
do!) so I have to give them simple recipes to follow.   So thanks and
thanks.]

[I think your comment about collapsing the qbrowse tree for sparse
directories is significant.  Whether this helps or hurts may depend on
whether the tree is created in the collapsed form or if it is a "tree
rewrite/copy" of a proper tree already allocated.  I also notice that the
commit and add operations use a lot of memory -- typically 250 to 280 MB.
The language translation and compiler folks deal with this sort of thing, as
you probably know, where way to limit peak memory demand is to process the
tree in small sub-trees on demand.  For a context-free "grammar" like a
recursive directory without symlinks, this may help.  Symlinks have the
potential to add context bridging sub-trees, so some care would be needed.
(Sub)Trees are great UI for selecting files for building a flat list, but
then maybe the (sub)tree can be discarded.  I suspect Nautilus and Windows
Explorer do something like this, because browsing the same directories is
almost instantaneous.]

Thanks for the clue on kcachegrind.  I get the idea, so I just need to learn
the particulars.  This helps.

Obviously I think these are important usability issues, and I appreciate the
interest here.  But I also want to emphasize that qbzr and bzr-explorer are
already excellent tools.

Thanks
~M



On Wed, Jun 30, 2010 at 1:09 AM, Gary van der Merwe <garyvdm at gmail.com>wrote:

>
> Sorry I did not mention this before (I actually forgot about it.) In
> qbzr trunk revisions 1220-1224, I made some performance improvements
> to qadd and qcommit. You mentioned elsewhere that you are using
> 0.18.4, in which case, you wont have these improvements.
>
> Note that you can do the add and commit steps from the qcommit dialog.
> Tick "Show non-versioned files" and then either tick "Select /
> deselect all" or manually choose file to add. Any files that are not
> versioned will be added before committing.
>
> I'm glad to see that qbrowse is not much slower than bzr status. I'm
> sad to see that qcommit and qadd are much slower. The tree widgets in
> qcommit and qadd share alot of code with qbrowse. Thing that happen in
> qcommit and qadd that don't happen in qbrowse:
>
> 1.) Folders are merged together with their sub item if they contain a
> small number items. For example in this screenshot:
> http://qbzr.googlegroups.com/web/gtk_qadd.png we don't show:
> docs
>    gettext_usage (copy).txt
> but rather:
> docs/gettext_usage (copy).txt
>
> 2.) All folders are loaded at startup. (In qbrowse only the root is
> loaded at startup, and sub folders are loaded when you expand them.)
>
> Currently, when you press Ok in qadd/qcommit, it runs a bzr
> subprocess. (You can see the command that it ran.) Because of this,
> there can't be any sharing of data. I'm currently working on
> refactoring qbzr so that we have 2 threads, one for the gui, and one
> for bzrlib. Once this is done, I plan on removing the subprocess, i.e.
> the command will be run in the same process. This will mean the caches
> will be shared, and hopefully this will improve the performance of
> pressing Ok.
>
>
> On Wed, Jun 30, 2010 at 1:23 AM, Maritza Mendez <martitzam at gmail.com>
> wrote:
> > I'm afraid I don't read lsprof output -- although it looks plain enough
> --lsprof-file test.callgrind
> > but I can run it as you suggest.
>
> .callgrind files typically open with kcachegrind [1]. You can also do
> --lsprof-file test.txt which will output a csv file which you can open
> with OO Calc. But if you are going to send it to us, please send a
> .callgrind
>
> [1] http://kcachegrind.sourceforge.net/
>
> > When using lsprof, should I capture the
> > whole session in a single callgrind file, or would it be more appropriate
> to
> > launch and profile a separate session for each action (qadd, qbrowse,
> > qcommit, qbrowse, et cetera)?
>
> As qcommit, qadd, etc are launched as a subprocess from bzr explorer,
> these would not be included in the output of bzr explorer
> --lsprof-file test.callgrind.
>
> I would like to take a look at the output of:
>
> bzr explorer --lsprof-file explorer.callgrind
> bzr qcommit --lsprof-file qcommit.callgrind
> bzr qbrowse --lsprof-file qbrowse.callgrind
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/bazaar/attachments/20100630/01c28d0e/attachment.htm