Bazaar spontaneously unusable

[Ian Clatworthy]

On 26/03/10 16:50, Martin Pool wrote:

> The obvious short term thing is just to update all the plugins but it
> would be nice to avoid going through this again in six months when we
> bump to 2.3.

Agreed. Let's try and address this in the current (2.2) cycle please.

> These messages are a bit silly because probably all of these plugins
> were working with bzr.dev yesterday and they still would be today, but
> now you get enormous warnings about it.

And in some cases, key things (e.g. qlog) stop working altogether with 
error messages that don't shed any light on the cause. :-( For example, 
see https://bugs.edge.launchpad.net/qbzr/0.18/+bug/546843.

> I believe Robert's concept for api warnings is that having a version on
> all of bzrlib was just a first step, and eventually we would get
> separate versions on bzrlib.tree, bzrlib.ui, etc, and plugins would
> declare their dependencies on just the parts they need. We could pursue
> this, but I'm a bit skeptical about this working well if we rely on
> developers to manually keep these things accurate on both sides. (There
> is also the issue of versioning submodules being added or removed but
> that could potentially be handled within the framework.)

I don't think there's much value in partitioning bzrlib into lots of 
bits. There may be value though in identifying 3-5 layers within bzrlib, 
e.g. util, domain, application ...

1. "util". bzrlib has a core set of really useful stuff. I'm thinking of 
modules like:

* options
* trace
* osutils
* progress
* cmdline
* urlutils
* ui

I suspect 95% of plugins use APIs from these modules. Many 
non-bzr-specific projects could benefit from them too. Perhaps we should 
be very strict about only extending, "never" deprecating APIs within 
this layer.

2. "domain". This layer has all the DVCS objects: repositories, 
branches, trees, revisions, etc. It's the guts of what make bzr tick and 
it's why bzrlib is so valuable. We should expect this to evolve, 
including deprecations, but it should become relatively stable over time.

3. "applications". This layer is builtins.py and all the modules 
explicitly created to support our UI commands. Being closest to the 
user, this layer has the highest rate of change and ought to remain dynamic.

If we felt it was worth it, we could also split the domain layer into 3:

* generic networking stuff, i.e. transports
* generic data storage APIs, e.g. btrees, chk_maps
* the rest.

In the context of plugin API tracking though, 3 layers is enough I suspect.

> We could go to a stricter definition of interfaces, including tagging
> them with versions,

> It tends to move us to more of a "whatever is
> not permitted is denied" approach, which if we could do it might be
> better for interface stability but might impede plugin development quite
> a bit.

I'd need a fair amount of convincing that this would be a step forward. 
Right now, it doesn't sound like a win.

> So the problem is when bzrlib is newer than the plugin and therefore the
> plugin code can't really have any good idea whether it will work with it
> or not. It has to just assume

Right. But *I* feel our plugins authors *ought* to assume that future 
versions of bzrlib will work, not fail. Having being burnt in the past, 
I can appreciate why they are being pessimistic, but lots of plugins 
suddenly spewing out scary messages - or breaking altogether - every 6 
months isn't the right end result. Our job as core developers is to give 
plugin authors every reason to be optimistic that, if their plugin works 
now, it will also work with future versions of bzrlib.

> Some concrete suggestions then:
>
> * Put into bzrlib a blacklist of plugin versions known _not_ to work
> with this version of bzr;

> we only need to add to this list when a problem is discovered; after the
> plugin fixes the problem they need only bump their minor number.

+1 from me on this idea.

> * If a plugin fails to load, don't print a message every time we start,
> but instead only inside 'bzr plugins'. It would make it less obtrusive,
> but perhaps mysterious if a single plugin stops working.

-1 from me. A plugin failing to load is bad. Let's stop that happening 
rather than play with when we tell users about it.

Here's another idea to consider ...

We run some static analysis scripts over the set of registered plugins 
and get some hard data about exactly what modules and APIs within bzrlib 
are used and by whom. Before we deprecate stuff in the code, we check 
that data/web-site and react accordingly. We might decide not to 
deprecate certain things. Or we might proactively communicate with the 
plugin maintainer about the change and synchronise releasing a new 
version of the plugin with landing the change into bzr's trunk. Either 
way, I feel it would show plugin authors that we care and, equally 
importantly, keep things running smoother for end users.

Ian C.