[RFH] Rework Bazaar Explorer + QBzr to use separate process for bzr operations and main process only as GUI presenter

[Alexander Belchenko]

Hi all,

During many last months I'm thinking about reworking Bazaar Explorer 
(and QBzr as well) to new architecture. Currently we have enough bug 
reports about stability (sometimes related to Qt itself), about issues 
with refresh feature of explorer, plus we have performance issues with 
big trees, plus responsiveness of UI. I'm going to mark such reports 
with some distinctive tag so I can provide better overview of them.

My plan for attacking this (very important and very hard) problem:

For all bzr operations, the most often used by explorer is getting 
status of working tree, -- start new separate process for each bzr 
location. That process works as server and can talk with any number of 
clients via IPC mechanism (I'm thinking about using cross-platform 
QLocalSocket/QLocalServer from PyQt4 if that matters).

Using one process per bzr location is inspired by Google Chrome 
architecture (as I understand it). This approach has several pros and 
some cons. This process might fail/segfault/whatever without affecting 
the GUI process. Therefore we need another global process to watch 
worker processes and if needed restart the crashed ones. I think it will 
be nice to adapt ideas from bzr-service plugin for such watcher. Or 
maybe add that process to AutoRun as TBZR does (does Ubuntu/Linux have 
some sort of autorun?).

One worker process per bzr location will keep up-to-date cache of WT 
status and will help reduce the refresh time for explorer. It also could 
be proactive and notify his clients about need for refresh.

This architecture will be a bit similar to one used in TortoiseBzr: 
separate background process crawls bzr trees and watches status of 
files, Windows Explorer shell extension talks with that background 
process. I don't know how much we can share the code there with TBzr, 
maybe we can't at all, because I need cross-platform solution.

Last year at bzr sprint (during UDS-M) Gary has started working on 
different approach for improving QBzr responsiveness: extract bzr 
operations into separate thread to make sure main thread is GUI only.
That's much simpler in the terms of implementation and it terms of 
exchanging data between layers. But crash on one thread will terminate 
GUI thread as well. That's why I think separate process is more robust 
solution here.

As you can see such plan is not very easy one. I need to write down many 
details, including communication protocol. Maybe I need to try to use 
processing Python library (standard in Python 2.6+ IIRC) to implement 
communication protocol. Also I can't predict how long it will take to 
implement new architecture. Maybe I can reuse some code from TortoiseBzr?

Hence RFH -- request for help. I fear I won't be able to finish this 
alone. I need some help to look over this plan and make sure it doesn't 
have major pitfalls. Then I need to decide what communication protocol 
will be. First minimal alpha version should work as `bzr status` worker 
and cache (again, the very first code from TortoiseBzr may help here to 
start) and we need to try adapt bzr-explorer + treewidget from qbzr to 
use that worker processes instead of running iter_changes() in GUI code.

Your comments are welcome, but please consider whether you are 
interested to help, so please avoid bikeshedding if possible.

Please note, that I'm really not sure is it worth to consider bzr-gtk as 
  possible client for such architecture. As I can see it now, such major 
change to use separate process will unite qbzr and bzr-explorer 
codebases much more than now.

I regret a bit that I won't be able to attend this May bzr sprint, I had 
hope to talk on this topic, maybe some of sprinters will be able to 
spend some time on this topic and maybe I can participate via IRC or 
other chat (skype is also OK, but text channel is slightly easier for me).

Alexander
-- 
All the dude wanted was his rug back