Bazaar Sprint report: Day one

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

I'm reporting from the Bazaar sprint in Mooloolaba, with a focus on
ubuntu distributed development.

Martin Pool asked me to report on the discussion we had on day one of
our sprint.  So far, there have been no final decisions, but there has
been a lot of clarification.

We discussed whether this would be a talking sprint, and Martin said
that at least the first two days should be focused on discussion.

Like Launchpad, the Bazaar project has been asked to focus on bridging
the gap with upstreams.  Our problem is that, as a group, we don't know
about packaging.

There seem to be two possible goals:
 1. increase the use of Bazaar by Ubuntu developers
 2. work on building from a branch into ppas

Whatever we choose, we should have *some* people who are enthusiastic
about Bazaar six months from now.

Jonathan Lange spoke about the view from Canonical's London office.
Launchpad is working on upstreaming bugs and doing whatever we can to
enable proto-Grumpy.

Proto-Grumpy is the concept of way of getting builds of the very latest
commit to a project's trunk.  Originally, it was conceived as a
pseudo-release like Debian's Sid, turned up to 11, called "Grumpy
Groundhog", where the entire distro was built from trunk.  Now it's more
about providing daily-build PPAs and integrating that into the distro.

If we were to focus on Grumpy, the result would be easier nightly PPAs,
descriptive (rather than imperative) ways of creating them.  PPAs would
need to be very discoverable.  We would promote PPAs built from
bzr-builder recipes.  Recipes could be combined and built on.

Jonathan continued discuss our strategy of making Ubuntu the best
possible conduit between upstream and users.  Proto-grumpy would make it
easier for users to try out the very latest version, so that upstreams
can get better bug info, and would in turn feel good about Ubuntu.  In
some ways, this is what Gentoo already provides.

As we understand it, proto-Grumpy won't directly help Ubuntu dev, but it
will be possible to extend the technology we develop from Grumpy to
serve Ubuntu development.

We decided we need a good "elevator pitch" for proto grumpy.  (An
elevator pitch is a short sales pitch you can give during an elevator
ride.)  "You can take a branch of any upstream that's in Ubuntu and
really easily start making packages from it."

We plan to have a "teach me packaging" session in the next few days, so
that everyone has a better understanding of packaging issues.

We discussed differences between Debian and Ubuntu approaches to
packaging.  In Debian, there's a focus on individual and small-team
ownership of individual packages, while in Ubuntu, people are encouraged
to work on a much broader set of packages.  This makes packaging
consistency across the whole set of packages more important to Ubuntu
than it is to Debian.

Martin Pool wants to reach a focus of either Grumpy or patch management.
 The Ubuntu devs have needs now and will be unhappy if these things
aren't fixed.  Martin suggested choosing Grumpy as a focus, but leaving
slack time to work on bugs, with priority to bugs affecting Ubuntu devs,
and this was generally considered reasonable.

Ian Clatworthy spoke about how it was too hard to find PPAs, and how we
need visibility into the things we care about.  There was discussion
about the difficulty of getting graphs for the things we care about.

Robert Collins spoke about packaging.  He mentioned that current
patch-based packaging met GPL requirements that modifications must be
clearly marked.  He described that difference between "normal" and
"native" packages (which have their packaging information built-in), and
explained that native packages are unpopular, partly because of
limitations in their implementation.  On Ubuntu, source packages are
uploaded, but on Debian, the developer uploads binary packages for their
architecture as well.  Robert noted that upstreams are discouraged from
doing packaging, and feels that this is wrong.

We discussed past failed attempts to transition from patch-based
packaging to branch-based packaging.  hct failed because
 - it was never open-sourced because it was not mature (they didn't want
   to release a bad prototype of a good idea).
 - it was a completely different patch representation that was hard to
   push to ubuntu.
 - it as slow
 - getting imports right was an issue

Robert described the several ways states that an Ubuntu package can be in:
 - unchanged packages are automatically synced.
 - when Ubuntu has patches, we publish them on a web site
 - when Ubuntu and Debian both have changes, we automatically merge,
   but we must QA -- we don't automatically sync anymore.

We should upstream as much as we can, but upstreaming stops at
packaging.  Ian suggests this hurts our ambition for "world domination".

Because of the popularity of Debian and Ubuntu, there can be an
antipattern where people work to get a pet application packaged, then
disappear.

We discussed enhancing the lp plugin, and making it depend on
launchpadlib.  We could make it an optional plugin, but we want to ship
with "batteries included".  Launchpadlib has an unfortunately long list
of dependencies.

We discussed componentizing bzr further, e.g. using testtools, splitting
Bazaar's command stuff out.  Ian said "partitioning is the most powerful
tool for driving quality up".  We also observed that we help others by
doing this, even if there is pain for us.

We discussed Launchpad's experience of Bazaar.
 - We like Bazaar
 - LP is slow, including branches in Bazaar
 - Bazaar is slow on checkout
   - recently improved
   - still a small fraction of tree creation time
 - We integrate virtually every bzr release into Launchpad, and it can
   be a chore.
   - We have enumerations of formats, so every new format affects lp.
 - It uses a lot of memory
 - It doesn't give us as much log info as we'd like
 - We wish we knew who is using what branches
 - stacking and rich roots interact very badly

Andrew Bennetts spoke about his discussions with James Westby.  There
were 5 stories:
 - proto-grumpy
   - best solution unclear: what exactly do we need to do to build debs
     from tip?
 - apply an upstream bugfix to ubuntu
   - (Why not a new version?  Often impractical.  Brings in other
     changes)
   - We discussed CVE patches
   - There are various patch systems, but the workflow is largely the
     same.
   - Hard for novices, but experts are happy and unmotivated to change
     things.
 - Compare a package with Debian
   - Sync requests were discussed
   - Sometimes we just smash the ubuntu changes
   - We should work to produce a lower-friction API for for
     sync-requests
   - merging is hard
 - New contributors
   - branch a package to a PPA
     - Make a trivial patch trivially
     - possible, but not easy enough
     - When you branch, you branch both the branch and the bzr-builder
       recipe
 - Submitting diffs upstream
   - Formats vary
   - Patch submissions are just the start of the conversation
   - Maybe loom threads need individual submit messages

We had an upcoming conference call with James Westby, so we discussed
what we wanted to ask him.
 - Does he use looms and/or pipelines?
 - Must we choose between patch management and proto-grumpy?
 - Who else should we be talking to on the packaging side?
 - Which stories does he feel are most important?
 - Why are recipes the way they are (esp. with arbitrary command
   execution)


We discussed some tricks and tips, like finding branches that can be
deleted.

This was all what we discussed on Monday, when I was the reporter.
There has been much discussion since, and other reports should follow
soon.  Please reply with any questions or comments.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr6XF0ACgkQ0F+nu1YWqI0vIQCfetrAv7aKbOyO22NMiC9DPAg0
9NEAn3p/mg7+oN5A+2Zhk5mjJVkyEGCP
=oIkI
-----END PGP SIGNATURE-----