Upgrading pristine-xz on jubany

[Vincent Ladeuil]

>>>>> Max Bowsher <_ at maxb.eu> writes:

<snip/>

    > Right, I agree we should use stuff straight from Ubuntu when we
    > can, but I don't agree that it's necessarily wrong to customize
    > our deployment practices to get fixes faster.

Good, I'm glad we agree on the 'get fixes faster' value.

    > In particular, I've been pinged on IRC about whether we can fix a
    > couple of these packages soon,

Which ones ?

They may indeed be fixed by upgrading to precise (52 fixes) or quantal
(180 fixes), but may be not (still 147 failures).

Leaving urgency aside for a moment. Some packages fail to import for
months, I think it's fair to work on solutions requiring weeks.

    > and it seems to me that I could do just that - and do it this
    > weekend - by installing new xz and pristine-tar versions just for
    > the importer. So I'd quite like to do that.

    >> backporting pristine-tar is not enough, xz-utils is also needed,
    >> with its own dependencies.

    > It can't be *that* onerous, can it? :-)

That's not what I'm saying ;)

Being able to run quantal means we get that for free so it's a
side-effect benefit that makes the work on running quantal more
valuable.

And so far, the instructions to setup an lxc container is roughly 20
lines of commands to run on either the host or the guest:

[host] sudo lxc-create -t ubuntu -n pkgimporter-quantal -- -r quantal -a i386 -b $USER
[host] in /var/lib/lxc/pkgimporter-quantal/config :

lxc.network.link=br0
lxc.network.hwaddr = 00:16:3e:0d:4f:90 # needs to be DHCP registered

[host] add bound mount <where on host>/pkgimport pkgimport in /var/lib/lxc/pkgimporter-quantal/fstab
[host] sudo mkdir /var/lib/lxc/pkgimporter-quantal/rootfs/pkgimport
[host] sudo chown -R <user>:<user> /var/lib/lxc/pkgimporter-quantal/rootfs/pkgimport
[host] sudo lxc-start -n pkgimporter-quantal

[pkgimporter-quantal] sudo apt-get install python-software-properties
[pkgimporter-quantal] sudo add-apt-repository ppa:vila/pkgimport
[pkgimporter-quantal] sudo apt-get update

[pkgimporter-quantal] sudo apt-get install bzr-package-importer

[pkgimporter-quantal] sudo apt-get install language-pack-en
[pkgimporter-quantal] sudo apt-get install xauth
lpkgimporter-quantal] sudo apt-get upgrade

XXX: setup ssh

XXX: install lp:udd
XXX: set config for udd

for production:

- restore dbs,
- restore credentials.

[pkgimporter-quantal] sudo poweroff
[host] sudo lxc-start -n pkgimporter-quantal
[host] ssh -AY pkgimporter-quantal.local


Another key benefit of using a container is that it reduces the
differences between jubany and any test setup devs need to use before
deploying.

    >> Moreover, we want to hand over the deployment to Canonical ops so
    >> adding more manually installed dependencies doesn't seem to go in
    >> the right direction.

    > OK, this is going to be controversial,

Yup, this controversy has been going on for quite some time and I'm more
on your side. Yet, there are constraints that should be obeyed.

    > but.... I don't think we should be planning to hand the deployment
    > over to Canonical Ops any time soon.

I'm been tasked to make that happen sooner rather than later :)

    > The needs, and bugs, of the importer still seem to be evolving at
    > a pace where retaining full control in the hands of those with
    > domain specific knowledge seems to be a far far better option than
    > enforcing a Dev vs.  Ops divide.

Both points are valid but I took a different approach:

- ops don't want to run packages that are not under their control,

- devs (we here) want to run packages that address their needs with a
  minimal amount of work.

Using an lxc container means both can be achieved.

    >> Don't get me wrong, I *did* push to get to the point where we
    >> could use branches for bzr and bzr-builddeb (let's ignore
    >> distro-info...) but I'd prefer to draw the line there, these are
    >> packages we are upstream for and it makes sense to be able to
    >> quickly deploy fixes if only to test them before official
    >> releases are cut.

    > On the other hand, we're (one of?) the primary consumers of
    > pristine-tar, so it's not really much of a stretch.

It's unclear to me that we are *known* as a primary consumer.

It's clear that we need fixes faster. Being able to report better bugs
should helps us getting there and also make us known as a primary
consumer.

    > And it provides better service to Ubuntu developers if we can turn
    > around an upgrade of that sort of thing without communication
    > across a Dev/Ops boundary.

I agree with this goal too.

For the record, I have a trivial fix for pristine-xz that address 2
failures and a wip that could well fix the ~140 left on quantal.

  Vincent