Fwd: Ubuntu/bzr stories update

[Martin Pool]

This was previously private, due to a misunderstanding, but it should
be public - much of the precursors are already on wiki.ubuntu.com or
ubuntu-devel at .

I'll soon announce a more specific topic public list that can be a
quieter forum than either bazaar or ubuntu-devel.  We should still
discuss the impact for Bazaar, as far as either work scheduling or
technical changes, on the Bazaar list.

---------- Forwarded message ----------
From: Andrew Bennetts <andrew.bennetts at canonical.com>
Date: 2009/11/3
Subject: Ubuntu/bzr stories update



I had a call with James Westby on Friday about the stories mail I sent a while
back.  First, here's the highlights:

 * The key feature requirement, and design question, is bzr's support for a
  loom-like structure for packages.
 * Recurring “nice to have” features: parallel imports, nested trees.
 * The stories from my original mail are basically right.

It's becoming clear (at least to James and myself ;) that figuring out how
exactly we will represent the structure of packages is the big design unknown
that will drive a lot of the other decisions.  In turn, this means figuring out
what operations people will want to perform with these things.

Briefly, a source package isn't cleanly represented with a single regular bzr
branch.  There are clear notions of “upstream” and “collection of changes to
that upstream”.  The NoMoreSourcePackages and DistributedDevelopment wiki pages
mention this, but avoid going into much detail about exactly how this should
work in Bazaar.  An obvious possibility is something very much like a loom,
where the base thread is the upstream version, and then each patch added to the
package is thread on top of that (including a patch to add the debian/ directory
with the packaging rules).

Here's a representative scenario (this is roughly story 3 from my original
mail):

“Ubuntu has a foo-1.1 package of a project called Foo.  This was derived from
Debian's foo-1.0 package, a package of an earlier version of upstream Foo.
Debian has since uploaded a foo-1.2 package, which updates to the latest Foo
release and updates the packaging.  Ubuntu's maintainers wish to merge these
changes (both the update to the current release of Foo *and* the packaging
improvements).”

For figuring out the required operations, there are some workflows discussed on
the NoMoreSourcePackages and DistributedDevelopment wiki pages, and also
bzr-builddeb already has commands like “builddeb”, “merge_upstream” and
“merge_package”.  So we have some starting points to build on, and hopefully as
we dig into the stories I posted and discussed with James any others will become
evident.

So I think by the end of Mooloolaba we probably want to get a draft proposal of
the structure for packages-in-bzr, and maybe even a prototype?

Parallel imports and nested trees did come up a few times as nice to have, but
not clearly a necessity to provide a smooth experience for Ubuntu devs.

Moving on to some details, here are some annotations on the stories from my
original list.  (I'll shortly put up a wiki page with the stories written up
more completely, this mail is just giving an update.  I also have some other
ideas and observations from the call that are worth sharing, but this mail is
already becoming too long so I'll do that in a followup message.)

1. Mini-grumpy
--------------

This isn't a story, at least not in the same way as the others — but Canonical
is keen to do this because it enables stuff that isn't possible now.  James says
that nested trees would be helpful for this (to assemble packaging branches and
upstream trunks into one tree?  James, care to elaborate on this?).


2. Apply upstream fix
---------------------

This is important, Ubuntu developers do this a lot.

Existing patch systems are a big mess.  We get a patch from somewhere, maybe
upstream trunk, maybe bug trackers, whatever... then record that in the
packaging.

This story doesn't map neatly to bzr features yet, need to tackle the larger
problem before we can break it down.


3. Compare packages
-------------------

Having the diff is the main thing.

A neat report of patches added/removed/changed would be nice, not vital.

Simple “bzr merge” already found to be inadequate, there's some ad hoc smarter
operations in bzr-builddeb now.


4. Host packages where Ubuntu is upstream
-----------------------------------------

We can drop this story; it's not particularly important or interesting (it's a
degenerate case; if we take care of the rest then this case is taken care of
too).


5. Branch to a PPA
------------------

James is keen to get this one right, it's important for first-time
contributors.  That said, it's more a Launchpad issue than a bzr issue (how to
create & host a branch), and more an Ubuntu issue (how to edit a
source package).

Ideally there would be a command like “bzr add-a-patch” — taking a existing
package and adding one patch to it is a pretty common use case.

I know *I* would use this story if it were sufficiently simple, and other
Canonical folk have said similar things.


6. Review diffs, submit them upstream
-------------------------------------

Probably this story should be reworked to focus on the submission story.

“bzr diff -r upstream:0.1” already exists (in bzr-builddeb)!

It would be good to be able to record notes like “this patch has been forwarded
to bug X”.

Ideally submission to upstream would be as automated as “bzr send” is for
regular bzr patches: bzr knows where to send it, what base to compare against,
etc.

Patch submission isn't a fire-and-forget operation, it's the start of a
conversation (although venues vary), because it's so rare for a patch to be
accepted purely based on an initial submission.  The submission tool may need to
take this into account?  Ideally, as devs commit code into bzr they start the
conversation (i.e. submit it).

7. MOTU merging latest from Debian
----------------------------------

There is a lot of merging from Debian.  You do the merge, then review the
changes you want to bring in, then review what's left (so you can make sure they
are forwarded upstream).


-Andrew.