bzr-explorer performance question

[Gary van der Merwe]

On Tue, Jun 29, 2010 at 9:41 PM, Maritza Mendez <martitzam at gmail.com> wrote:
> Ok.  Thanks for all the great ideas.  I will try to get to them between
> other tasks.
>
> I will summarize some of my thoughts as additions to Issue#582838 and open
> others if appropriate.
>
> As has been said before, there appears to be both a bzrlib component to this
> and bzr-explorer component.
>
> To identify where the focus really belongs, I did the experiment again with
> the same files, but this time with qbzr on the command line, without
> bzr-explorer.  Here are the results:
>
> bzr qinit SHARED_REPO  # pick shared repo & format 2a
> cd SHARED_REPO
> bzr qinit trunk  # pick stand-alone branch & format 2a -- uses shared repo
> cd trunk
> [copy all 300+ directories with 500+ files to trunk]
> bzr qbrowse  # <1 second
> bzr qadd
> # immediate response: QTreeView::rowsInserted internal representation of the
> model has been corrupted, resetting.
> # 76 seconds to build list of files to add
> # click "OK" to start the actual add
> # 34 seconds to actually add files
> # total time 76+34=110 seconds
> bzr qbrowse  # < 1 second
> bzr qcommit
> # immediate response: QTreeView::rowsInserted internal representation of the
> model has been corrupted, resetting.
> #  37 seconds to build list of files to commit
> #  click "OK" to start the actual commit
> #  154 seconds to actually commit
> # total time 37+154=191 seconds
> bzr qbrowse  # 33 seconds
> bzr qbrowse  # <2 seconds
> bzr qbrowse  # <2 seconds
>
> Three things deserve comment:
>
> 1. The QTreeView errors are interesting.
> 2. The first qbrowse is slower than subsequent runs, consistent with what we
> see with 'bzr stat'
> 3. The qadd and qcommit seem to take longer than their pure bzrlib
> counterparts
>
> There's more to be learned from the details.  As I said, I need to think
> just a little before I log this at launchpad.
>
> ~M

Sorry I did not mention this before (I actually forgot about it.) In
qbzr trunk revisions 1220-1224, I made some performance improvements
to qadd and qcommit. You mentioned elsewhere that you are using
0.18.4, in which case, you wont have these improvements.

Note that you can do the add and commit steps from the qcommit dialog.
Tick "Show non-versioned files" and then either tick "Select /
deselect all" or manually choose file to add. Any files that are not
versioned will be added before committing.

I'm glad to see that qbrowse is not much slower than bzr status. I'm
sad to see that qcommit and qadd are much slower. The tree widgets in
qcommit and qadd share alot of code with qbrowse. Thing that happen in
qcommit and qadd that don't happen in qbrowse:

1.) Folders are merged together with their sub item if they contain a
small number items. For example in this screenshot:
http://qbzr.googlegroups.com/web/gtk_qadd.png we don't show:
docs
    gettext_usage (copy).txt
but rather:
docs/gettext_usage (copy).txt

2.) All folders are loaded at startup. (In qbrowse only the root is
loaded at startup, and sub folders are loaded when you expand them.)

Currently, when you press Ok in qadd/qcommit, it runs a bzr
subprocess. (You can see the command that it ran.) Because of this,
there can't be any sharing of data. I'm currently working on
refactoring qbzr so that we have 2 threads, one for the gui, and one
for bzrlib. Once this is done, I plan on removing the subprocess, i.e.
the command will be run in the same process. This will mean the caches
will be shared, and hopefully this will improve the performance of
pressing Ok.


On Wed, Jun 30, 2010 at 1:23 AM, Maritza Mendez <martitzam at gmail.com> wrote:
> I'm afraid I don't read lsprof output -- although it looks plain enough --lsprof-file test.callgrind
> but I can run it as you suggest.

.callgrind files typically open with kcachegrind [1]. You can also do
--lsprof-file test.txt which will output a csv file which you can open
with OO Calc. But if you are going to send it to us, please send a
.callgrind

[1] http://kcachegrind.sourceforge.net/

> When using lsprof, should I capture the
> whole session in a single callgrind file, or would it be more appropriate to
> launch and profile a separate session for each action (qadd, qbrowse,
> qcommit, qbrowse, et cetera)?

As qcommit, qadd, etc are launched as a subprocess from bzr explorer,
these would not be included in the output of bzr explorer
--lsprof-file test.callgrind.

I would like to take a look at the output of:

bzr explorer --lsprof-file explorer.callgrind
bzr qcommit --lsprof-file qcommit.callgrind
bzr qbrowse --lsprof-file qbrowse.callgrind