bzr 2.1 retrospective

[Martin Pool]

2.1 is out, it's a good product, and I'd like to now have a bit of a
retrospective around the release and the whole cycle, looking for both
good and bad things that we can build on next time.  I'm going to
nominate some things to change etc but you can disagree (as if I have
to tell you :-).  I want this to be about the experience of
development, not nominations of particular features or bugfixes to do
next, but feel free to start one separate thread about that.

2.1 has been mostly an incremental cycle, with some useful
small-medium features (per-file merge hooks, reduced memory, etc) and
quite a few bug fixes.  A lot of the innovation has happened in
plugins including huge progress in loggerhead, qbzr, bzr-explorer,
bzr-git, and bzr-hg, and that seems like a sign of maturity.

Still, in the next cycle it would be good to have one or two bigger
bangs.  The Canonical staff are going to be helping Ubuntu a fair bit
so we shouldn't overcommit, but perhaps we could pick one.  If we are
going to do large changes, it works much better if many people are
committed.

* change: pick one or two focus features for 2.2?

We did two candidates over a three week period leading up to the final
release.  This felt about right.  Most of the changes in them were
fixes for pre-existing bugs that would have qualified to go onto a
stable branch.  The main things that were regressions in rc1 are (I
think) things for the per file merge hook, which landed just before
rc1.  However, I don't think we especially had a problem with this
being rushed in, it's just something that happened to land close to
rc1.

* affirmed: release on schedule; don't rush features in and don't slip
the release to add a feature; land when it's ready

2.1.0 final is in Lucid with some time to spare before their beta
freezes, so we have the chance to land any additional bugfixes, and
they're shipping something supportable and with the latest features.
This is very good.

* affirmed: target final releases for a few weeks before Ubuntu feature freeze

We had some very substantial memory usage improvements from John,
including spinning off memory profilers.  My impression is that this
worked well because of having some uninterrupted time to get into it.
What more could we do to replicate this?

Vincent has made some plans for better conflict handling but little
has landed into 2.1.  What if anything should we do differently?
Maybe he was interrupted too much by other things?

We changed selftest to depend on <https://launchpad.net/testtools>,
which caused some amount of fallout, but I think is ultimately a step
in the right direction towards making nice things we've added within
selftest be generally reusable, and hopefully also getting nice things
from other projects.  I think subunit output has not yet really
reached its full potential, in that at least half the time I use it
there is some problem that sends me back to using plain text output.
(For example, that subunit-filter by default shows you a lot of output
other than just failures.)

We have Babune now on Hudson and telling us about cross-platform
failures, and these seem to be being cleaned up?  It seems like it's
mostly Vincent that checks it and fixes things, but that may not be so
bad.

I think I, at least, have had a bit of a problem with not finishing
off fallout/regression from landings, such as
<https://bugs.edge.launchpad.net/bzr/+bug/456077>.

* change: look harder at +assigned or +inprogress before starting
something new; get this down to zero and then there'll be more freedom
to choose

We merged some new plugins into the Windows installer between rc2 and
final; this caused some problems in the Windows installer (see thread
"Win 2.1.0-1 installer bzr-explorer toolbar problem" and
<https://bugs.edge.launchpad.net/bzr/+bug/524162>).  This was caused
by a late update to bzr-explorer to move it from 0.11 to 1.0.  We had
planned to do an additional rc with that change but because of
personal factors we didn't actually do it.  This is not catastrophic,
but we may have to ship a new Windows installer or do a 2.1.1 soon.
The bug also points to some unnecessary coupling between bzr core and
the windows installer metadata.

* change: Plugins must have bugfixes only, relative to the version
shipped in rc1.
* change: Clearer reminders to plugin authors that this deadline is
coming up, and of what it means to them.
* change: Insist on zero changes between the last rc and the final
release?  This may be waste though; making the release takes some work
and it's not necessarily worth doing another cycle to squash the
probability to absolutely zero.

Packaging for Windows continues to be a headache, with it being
somewhat labour intensive and specialized.

The stable branch approach seems to be going well.  We have what feels
like about the right number of bug reports on betas vs stable.  We
haven't merged any changes to 2.0 that caused regressions or made
users complain.  I think targeting bugs and patches to 2.0 is not
proving to be too much of a burden, though it does tend to need to be
explained to new users.
<http://doc.bazaar.canonical.com/bzr.dev/developers/bug-handling.html#backports>
explains a bit about the bugtracker.

* affirmed: stable branches yay

One nice outcome of stable branches is that people can remove old APIs
more easily, and plugins within a series will not be broken.  It's
very good to see for example Jelmer recently deleting unnecessary
APIs.

* affirmed: API stability by code branch
* affirmed: delete crufty code!

We got 2.0.2 into karmic-updates, which validates that our stable
branches are a good way to get bugfixes into the distro bugfix
channel.  (In this particular case Ubuntu, but we hope others will
take them too.)  However, we've only so far done one of these, and it
was fairly slow.  I think we should try to get 2.0.4 (or 2.0.5) into
Karmic etc too, and
<https://bugs.edge.launchpad.net/ubuntu/+source/bzr/+bug/495023> could
be a figurehead bug for that.

* change: more SRUs, faster (but not too fast), communicate more with
Ubuntu packagers

We haven't yet got 2.0.4 backported to earlier releases
<https://bugs.edge.launchpad.net/ubuntu/+source/bzr/+bug/302987> and
we should close this gap.

* change: get more attention onto bzr backports?

Patch piloting also seems to be going well.  It is a fair amount of
work for people who aim to provide a high level of service, which I
think is worthwhile but not mandatory.  It is quite interrupt-y work
so I think it's good to have it separate from doing development
yourself.  It is intensely satisfying to see new contributors like (to
take just the most recent example) Parth coming back with updates and
new patches because they get quick helpful feedback.  It seems to have
also helped experienced developers get prompt decisive reviews.

* affirmed: patch piloting
* affirmed: explain how to solve +easy bugs

I think the way staff are managed, with a lot of selfdirection but
some focus is working well, but you tell me.

It would also be nice to find more time to make changes that make bzr
easier to develop on, and in particular easier to write tests against.
 But there has been some progress here, with shell-like tests,
ModuleAvailableFeature, better testing guide
<http://doc.bazaar.canonical.com/latest/developers/testing.html> and
some other things.

* affirmed: keep improving ease of development

We have made some progress on bugs with patches, but there are still
>30 needing to be dealt with.

* affirmed: push down +patches bugs

John and I (at least) have been working through the +new bugs and have
reduced this from somewhere over 260 to about 117.   So the end is in
sight.  Many of them can be disposed of as dupes; sometimes to a
closed bug (which is nice for the user) and if it goes to an open bug
it gives us an idea of how to prioritize it.  Some things like
Hydrazine's bugclient, sorting by number of dupes, and some patches I
have in train for Launchpad should make this process faster.  I'm
still not absolutely convinced that triaging bugs is a better use of
time than fixing bugs you already know about, but perhaps it can be
part of a balanced diet.

We created a bzr-qa team but have seen relatively little activity by
non-developers on bug gardening.  I'd like to know why.  I suspect it
is is either people don't see it as useful, or they don't feel
confident to get in and change things.

I really like the idea of a what's new document, as drafted by Ian,
etc, especially because it can talk about all the good stuff in
plugins.

* change: maintain a "What's new" document

GNU emacs and Bugzilla, amongst others, converted to bzr, and seem
fairly happy.  emacs generated a spate of good bug reports, including
documentation bugs.  Some have been fixed but some remain.  One
particularly important point is that there is no good user-oriented
documentation of bzr's model and we should add one.

In the Canonical ecosystem people are using Bazaar much more for
Ubuntu branches, and basically liking it (which is good) and filing
bugs (which is also good).  Launchpad's merge proposals provide a
pretty good workflow tool.  One could say much more but there should
probably be a separate thread onto their open developer list.

Personally, I've really enjoyed the last few months and feel like
we're in a pretty good productive zone.  I have certainly written a
lot more code than in the previous cycle.

Here's to a great 2.2 cycle,
-- 
Martin <http://launchpad.net/~mbp/>