Bazaar spontaneously unusable

[Martin Pool]

On 25 March 2010 21:56, Russel Winder <russel.winder at concertant.com> wrote:
> I appreciate that using bzr.dev is bleeding edge but the recent change
> of bzrlib version has rendered a number of plugins unusable:

Russel, can you please use more specific and less hyperbolic mail 
subjects?  For example something like "many plugin api version 
warnings".  People will still pay attention even if you don't claim the 
sky is falling, I promise.

> Unable to load plugin 'svn'. It requested API version (2, 1, 0) of module <module 'bzrlib' from '/home/Checkouts/Bazaar/Bazaar/Mainline_Working/bzrlib/__init__.pyc'> but the minimum exported version is (2, 2, 0), and the maximum is (2, 2, 0)
> Unable to load plugin 'git'. It requested API version (2, 1, 0) of module <module 'bzrlib' from '/home/Checkouts/Bazaar/Bazaar/Mainline_Working/bzrlib/__init__.pyc'> but the minimum exported version is (2, 2, 0), and the maximum is (2, 2, 0)
> Unable to load plugin 'rewrite'. It requested API version (2, 1, 0) of module <module 'bzrlib' from '/home/Checkouts/Bazaar/Bazaar/Mainline_Working/bzrlib/__init__.pyc'> but the minimum exported version is (2, 2, 0), and the maximum is (2, 2, 0)
> Unable to load plugin 'loggerhead'. It requested API version (2, 1, 0) of module <module 'bzrlib' from '/home/Checkouts/Bazaar/Bazaar/Mainline_Working/bzrlib/__init__.pyc'> but the minimum exported version is (2, 2, 0), and the maximum is (2, 2, 0)

Right, this obviously isn't very good and I hope we can work out 
something better to do.

The obvious short term thing is just to update all the plugins but it 
would be nice to avoid going through this again in six months when we 
bump to 2.3.

The point of the API warnings is that if people have a plugin that is in 
fact out of sync with bzr, they get a clean understandable failure 
rather than (most likely) an internal exception or in the worst case 
incorrect behavior.  We know that does cause user pain.

These messages are a bit silly because probably all of these plugins 
were working with bzr.dev yesterday and they still would be today, but 
now you get enormous warnings about it.

I believe Robert's concept for api warnings is that having a version on 
all of bzrlib was just a first step, and eventually we would get 
separate versions on bzrlib.tree, bzrlib.ui, etc, and plugins would 
declare their dependencies on just the parts they need.  We could pursue 
this, but I'm a bit skeptical about this working well if we rely on 
developers to manually keep these things accurate on both sides.  (There 
is also the issue of versioning submodules being added or removed but 
that could potentially be handled within the framework.)

We could go to a stricter definition of interfaces, including tagging 
them with versions, through eg ZopeIterfaces 
<http://svn.zope.org/zope.interface/trunk/src/zope/interface/README.txt?view=markup>. 
  Last time this was raised there was some concern about its 
startup-time overhead.  For this to really work we would need some 
ongoing refactoring to make sure that every interface that plugins 
wanted to use was a specifically defined Interface, which would be 
interesting, but not accomplished overnight.  It tends to move us to 
more of a "whatever is not permitted is denied" approach, which if we 
could do it might be better for interface stability but might impede 
plugin development quite a bit.

But the main thing here is that we really want to be bothering the user 
only when they need to do something.  At the moment I think we often 
warn them when they don't need to be told, and sometimes (especially in 
development series) don't warn them when they do need to be told.  It's 
poor that the first api change in a series in bzrlib requires updating 
many plugins, most of which probably won't be actually affected by that 
change.

The cases seem to be:

0- The plugin is known to work with a previously released bzrlib; 
trivial; obviously it should not complain.

1- The plugin is known not to work with an older bzrlib; if this is true 
it's reasonable for it to refuse to load.

1- The plugin is run with a later version of bzrlib and actually does 
work; it should not complain.  Current behaviour depends on whether the 
plugin checks for require_api or require_minimum_api.  The variation is 
not necessarily bad because as Jelmer says, some plugins really do have 
tighter coupling than others.  If it did require_api and it actually did 
work, we need an update to the plugin to relax the check -- this is the 
case Russel is hitting here.

2- The plugin's run with a later version and actually doesn't work. 
Again if the checks were strict we'll catch this, and if they aren't, we 
won't.

So the problem is when bzrlib is newer than the plugin and therefore the 
plugin code can't really have any good idea whether it will work with it 
or not.  It has to just assume and will sometimes be wrong one way or 
the other.

Some concrete suggestions then:

  * Put into bzrlib a blacklist of plugin versions known _not_ to work 
with this version of bzr; this should reduce the need for plugins to 
make pessimistic assumptions about which bzr releases they support, and 
it will let us catch problems with plugins that are optimistic.

incompatible_plugins = [('svn', None, (0, 90))] # meaning <= 0.90

we only need to add to this list when a problem is discovered; after the 
plugin fixes the problem they need only bump their minor number.

Eventually this could be populated from the plugin info but it would be 
useful even if it was manual.

  * If a plugin fails to load, don't print a message every time we 
start, but instead only inside 'bzr plugins'.  It would make it less 
obtrusive, but perhaps mysterious if a single plugin stops working.

-- 
Martin <http://launchpad.net/~mbp/>