[braindump] ppa update discontents

[James Westby]

On Fri, 20 Aug 2010 18:47:11 +1000, Martin Pool <mbp at canonical.com> wrote:
> Along with maxb and garyvdm I've been updating the bzr beta ppa for
> our 2.2 release.  This is not absolutely the most fun thing I've ever
> done; in fact it's pretty tedious.  Since we're supposed to be making
> tools to make packaging more productive and fun we could either find
> some more things here to improve, or perhaps that I'm doing it wrong.
> This mail has more problems than solutions but it's a start.

Thanks, complaints without solutions are useful anyway, so any proposed
solutions are a bonus.

> It's the kind of tedious work that doesn't require much creative input
> but is not very automated, seems to have a bunch of snags that make it
> not trivially automatable, and with long latency, so you can't just
> sit down and do it then know you're done.  It's about 10 minutes of
> actual thinking spread over a couple of days of spade work.

Are there ways that we could chain things for you, so that you could set
them up, and if all went well all steps would be done?

> Most of what we do is just add a rebuild line into the debian
> changelog, push, and upload, but things occasionally go wrong per
> package.  This should be very automatable.

Rebuild line to build for a different distroseries?

> It can be a bit hard to work out which packaging branch is used for
> the version in our ppa.  Launchpad has a concept of series branches
> for We can have a convention for plugins that we package, but it's
> just a convention and because it's evolved over time it's not always
> consistent.  For instance if Gary has uploaded bzr-gtk -
> 0.99.0-1~hardy4 but it's failing, I can get the source for that
> package and try to fix it, but I can't easily work out what branch
> contains that source.  (Well, I can guess it's one of the recently
> updated branches in <https://code.edge.launchpad.net/bzr-gtk> but it's
> not great.)

  https://bugs.launchpad.net/launchpad-code/+bug/395200

is one thing that might help that. As it would allow for a package in a
PPA to tell you a list of branches that contain the revision that built
it.

Also, recipes could help here, as they are pointers to branches, and I
think you could go last build->recipe and work from there.

Lastly, Launchpad may want to do something to have "official" branches
for PPAs, like we do for distributions.

> The lag seems to come in mostly through Soyuz delays: there's a delay
> before your package is accepted, then a delay that was running at
> several hours yesterday before it's actually built or rejected.  I do
> realize that there's a lot of demand for Soyuz build machines and that
> people are working on improving scheduling and performance.  But still
> it does mean there's this long period where you have mental
> work-in-progress, and can't wholeheartedly move on to something else.
> If there's a mistake either in your work, or the scripts, you have to
> page in the state you had a while ago.  The long lag is pretty
> unreasonable when bzr has barely any compilation steps and builds in
> well under a minute on a laptop.

You were test-building the packages locally? That's what I do, and means
that I'm not waiting on the builds to know if they worked or not. (They
sometimes fail, but it's rare enough that I assume that they won't.)

> One approach to the lag in soyuz builds is to test build every package
> locally, before uploading.  This seems like a bit of an unnecessary
> kludge, but perhaps it's the most sensible thing for now.

(Should read the email once before replying :-)

Why do you consider it a kludge?

> It seems like we have to keep track of which builds have passed or
> failed by collecting a set of nearly-identical mails.  There's a
> "latest updates" portlet in eg
> <https://edge.launchpad.net/~bzr/+archive/proposed> but it only shows
> the last 5 which is not really enough (http://pad.lv/620903).

You know about the "View package details" link?

You're looking at the user-focused page their, clicking on that link
will get you the developer-focused one.

> Perhaps we could standardize a way to do local test builds in a way as
> close as possible to what Launchpad does, including how to get
> bzr-builddeb to use it for a a test build before uploading.

Sounds like a plan.

> Our process at the moment is to upload everything into
> <https://launchpad.net/~bzr/+archive/+proposed> before then promoting
> it to the regular archive, so that we don't break dependencies between
> packages.  (Uploading a bzr newer than the plugins support might make
> them uninstallable, and unlike the main archive(?) ppas don't
> themselves check for this.)  I think the basic approach of using a
> staging archive is ok.  We ought to be able to script something that
> does both the check and the promotion if it succeeds.

The main archive does not check for this.

> Perhaps some or even a lot of this could be addressed by giving
> bzr-builder the ability to build from a tag into a ppa (or perhaps it
> already does that), and then we can drive most of this by just
> creating and running the recipes.

Yes, it can already do that.

The main irritation you will find right now is updating the version
numbers. However, that can be as little work as once per-package, and
then requesting builds in to each distroseries, and that could probably
be scripted.

Give it a go and let us know what you think. I'm sure there are many UI
improvements that could be made around that process, so I'd like your
input.

Thanks,

James