Juju Delegated Team

[Nicholas Skaggs]

Given the feedback at today's DMB meeting, I am attempting to provide 
more detail on the background of the juju source packages within ubuntu. 
I'll also provide a little snapshot of the journey on improving these 
packages since my involvement in the team. I am happy to answer any 
questions anyone may have -- feel free to send them my way.

 From June 2014 to August 2014 the juju core and TB engaged in a 
discussion around a stable release exception for juju. I'll link you to 
the relevant conversations here:

https://lists.ubuntu.com/archives/technical-board/2014-June/001969.html
https://lists.ubuntu.com/archives/technical-board/2014-July/001972.html 
and replies
The approval 
https://lists.ubuntu.com/archives/technical-board/2014-August/001992.html

In summary, similar to landscape, juju is subject to many external 
influences, including external providers, as well as external machines 
and operating systems. The client server nature of juju demands that the 
agent and client stay in sync. The TB has defined several ways in which 
they acknowledge the stability of a package can be adversely affected, 
even which no change occurs to the package itself. For juju, this 
typically means server or providers changes. Quoting Alexis's initial email:
> Juju is built on cloud APIs from Microsoft, Joyent, HP, Openstack, and many
> others, and we have been required to push updates that fix juju when those
> APIs change in backwards incompatible ways. Likewise juju is subject to
> changes in real and virtual machine hardware deployed in the field and in
> particular is being used as a test bed for high density server systems with
> ARM and Power hardware which provide very rapid moving targets.

At some point that wiki page Martin created at that time was broken and 
I was asked to create a new page to be referenced for SRU's which 
outline our requirements. That page is here: 
https://wiki.ubuntu.com/JujuUpdates.

This exception grant predates my direct involvement with juju.

As to the history of the package, others can certainly provide much more 
detail than I. However, as an outside observer involved with QA and 
community testing, I saw juju struggling to meet the distro's 
requirements for development and landing. Often they wished to land late 
in the cycle, or even post release as an SRU. This caused tension and 
extra work for everyone.

This was the case when I joined the team in April and no 2.0 releases 
had been uploaded while the archive was open during the development 
cycle. My first work was involved in trying to land juju-2.0 into xenial 
and formulating this FFe: 
https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1545913. I 
discovered the security team's requests for breaking out the packaged 
dependencies 
(https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1568148) and 
set to work on solving the many packaging issues, adding adt tests to 
prevent LXD breakage, and including what I saw was an issue with the LTS 
upgrade story. So I advocated a solution for trusty users migrating with 
juju-1 installed. This led to the creation of the juju-1-default 
package. If you check the bottom of that bug I listed 3 things to 
improve on during the yakkety cycle. All three have been worked this 
cycle, including regular beta uploads, further package dependency work, 
and now this ppu application.

Since that initial work, keeping uploads and SRUs into xenial and 
yakkety on a regular basis has been challenging because of juju 
packaging defects and autopkgtest failures, archive changes, and things 
like 32-bit built failures. I believe juju has fallen down and simply 
avoided uploading once things got difficult in the past. I've worked to 
try and solve these issues, and in the cases of 32-bit failures, worked 
with the release and SRU teams to get working builds despite my concerns 
about the lack of upstream support and the precarious nature of 
unsupported builds.

For example, when 32-bit ppc failed to build recently, pitti was helpful 
in pointing out resources to help solve the problem. I added a patch to 
keep the build rather than remove it as upstream wished 
(https://launchpad.net/ubuntu/+source/juju-core/2.0~beta15-0ubuntu2.16.04.1).

When 32-bit builds failed again 
(https://launchpad.net/ubuntu/+source/juju-core/2.0~beta17-0ubuntu1.16.10.1), 
I sought resolution in some way on a best effort basis. My belief and 
initial discussions with others such as slangasek and pitti was that 
there was no good way forward. My primary concerns were for users on 
running a neutered or old version of juju and our inability to get a 
much needed fix into Xenial. This inability to push SRU's is a big deal 
and needs to be solved. Broken or outdated packages should not be left 
to rot. This led me to request removal of these builds from xenial; and 
the associated work to providing a package to transition 32-bit users 
off the old build 
(https://lists.ubuntu.com/archives/ubuntu-release/2016-September/003882.html). 
Based on a follow-up IRC conversation with pitti's and rbasak's concerns 
and feedback, among others, a consensus was reached to retain whatever 
support possible, on the basis a neutered package is better than no 
package at all. In agreement, I will again seek to put the distro first 
and find a solution to preserve a 32-bit package on xenial. I've 
requested for upstream to allow building without providers, so when 
external code drops support for 32-bit, it can be removed from the 
build. This upstream work I trust should guarantee I am not blocked on 
deploying critical releases, while still allowing a best effort for 
unsupported builds. It is work upstream will undertake that otherwise 
wouldn't have occurred.

Historically I've worked within the community and advocated for 
solutions that will help the distro as a whole, rather than focusing on 
specific upstream desires. I understand and agree with the rational of 
LTS releases, I've been on both sides of SRU's, and I'm no stranger to 
the discussing the release cycle 
(http://www.theorangenotebook.com/2013/03/staring-down-scarecrow-should-ubuntu.html). 


Again my goal is to continue to work with the security, foundations, 
release and SRU teams to make juju a better citizen within the distro. I 
believe the yakkety cycle has been a much better example of what juju 
could be, and I trust with ppu rights this could be further improved. 
These challenges are not going to go away, but I believe juju needs 
someone willing to engage and understand the challenges of a 
distribution. I trust this engagement will mean a better package for 
ubuntu and juju.

Thanks again for your consideration, and please don't hesitate to reach 
out with any questions or concerns.

Nicholas