Removing 32-bit architectures from juju-core source package

[Nicholas Skaggs]

On 09/19/2016 10:37 PM, Steve Langasek wrote:
> Hi Nicholas,
>
> On Mon, Sep 12, 2016 at 06:47:31PM -0400, Nicholas Skaggs wrote:
>> On 09/07/2016 04:57 PM, Nicholas Skaggs wrote:
>>> Hello all. I'd like to discuss the removal of 32-bit architectures from
>>> the juju-core source package. The current packaging in the xenial and
>>> yakkety archive for juju-core specifies it's architecture as 'All'. This
>>> was an oversight as we officially support only the following
>>> architectures:
>>> amd64 ppc64el arm64 s390x
>>> We don't test or support 32-bit builds of juju. This is in-line with the
>>> clouds upon which juju runs which don't support 32-bit servers, as well as
>>> our own support of xenial server and mitaka -- 64-bit only.
>>> With this in mind, I'd like to update the archive packaging in both xenial
>>> and yakkety to remove these unsupported builds. I realize removing
>>> previously published binaries from the xenial archive isn't ideal, however
>>> we cannot update the current packages in order to deal with changes in
>>> cloud providers.
>>> I am looking for feedback and help to accomplish this. I would propose the
>>> following, but am open to other ideas to best accomplish this task.
>>> 1) Upload a new conjure-up package to xenial and yakkety that changes the
>>> architecture to 'Any'
>  From my POV this is not required.  Anyway, changing its architecture to
> 'any' would probably not be what you want since that would still build the
> package on all architectures; if you were going to change it at all,
> presumably you would want to restrict the architecture list, or add the same
> kind of warning-message-on-upgrade dummy package.  But provided that we are
> handling the architectures appropriately in juju-core itself, I don't think
> you need to change anything in conjure-up.
>
>>> 2) Upload a new juju-source package to xenial and yakkety that:
>>>         -- specifies the architecture as 'amd64 ppc64el arm64 s390x'
>>>         -- provides a second binary package for 32-bit users that ensures
>>> they upgrade to a message informing them the package isn't supported.
> This seems like the best available option.  Ideally we would not get
> ourselves in the situation in the first place that a package we can't
> support has made it into a release.  But seeing that we're here, I think
> it's better to bite the bullet now rather than let it drag on further.
>
>>> I want to ensure a smooth experience for anyone who installed a 32-bit
>>> version of juju on xenial. It is not found on any images, and juju itself
>>> is not yet final. Production deployments should still be utilizing juju-1.
>>> I would like to remove this package before wider adoption as juju2 enters
>>> RC and final release stages. I would especially appreciate ideas about
>>> ensuring a good upgrade story for current users. I don't suspect there are
>>> many at all, but I don't want to leave unsupported and abandoned packages
>>> in the archive.
>> I'd like to update this mailing to reflect a series of discussions today on
>> #ubuntu-devel surrounding this topic. It has been proposed that a solution
>> be found to provide as much support for the existing 32-bit packages as
>> possible. I've requested the juju-core team to provide compile-time support
>> for disabling providers as a means to mitigate issues where upstream code
>> stops support for 32-bit builds. For 32-bit users, they will retain support
>> to the extent possible for providers on a best effort basis.
>> Most importantly, I trust this relieves my concerns about the inability to
>> landed needed SRU's and regular updates in the development version of ubuntu
>> due to a failing build in an unsupported architecture. I'd like to thank
>> everyone involved for there feedback and discussions. For now, consider this
>> request canceled.
> This implies that the package will be available on 32-bit now, but that you
> are not committing to keep that package free of functional regressions in
> SRU on 32-bit architectures.  That seems much worse, to me, than disabling
> the package altogether on 32-bit now, or immediately disabling all the
> providers that upstream has not committed to providing 32-bit support for.
I agree it's better to cut this off now rather than later. That was the 
initial point of my mail, and my continuing concern with shipping 
packages that don't have upstream support. The counter-arguments given 
in response to my initial mail pointed out that juju is not unique in 
this regard and that efforts should be made to support 32-bit. My 
concerns haven't gone away; I think removing the package and being clear 
to users about what is supported is a "better" user experience, but I am 
open to feedback that would seek alternative solutions. My concern 
either way is that we're blocked on fixing issues because of 32-bit 
failures later on, which I consider a big risk. The proposal of making 
it so we could simply disable the broken bits on 32-bit and retain as 
much as we can was a reasonable request, and one I wanted to try and 
fulfill.
> One of the arguments made for why it's ok to drop the 32-bit package right
> now in xenial is that juju 2 is not yet at its final release and isn't yet
> used in production.  Tabling this request "for now" just means that the next
> time it comes up, it will have much worse impact on the end user.  Let's
> please sort out now what the juju team can commit to support on 32-bit for
> the lifetime of the LTS and do that, rather than pulling the rug out from
> under the users later.
I also agree that now is the time to decide, hence my mails. I felt the 
risk of "dropping" a 32-bit package, while never an ideal thing, was 
best done now early in the cycle and before adoption. If we think there 
is any chance the package would have to be removed during the course of 
the cycle, I would be firmly in the camp of removing it now. I'm also 
concerned about the value and overhead that will be placed on keeping 
the 32-bit package useful over 5 years. Speaking as a user personally, I 
really don't like neutered packages. They are like landmines sometimes; 
I've personally been bitten by this on armhf. I laid out a worst case 
scenario of having a juju package that works only with the local 
provider. Others felt would this would retain value even if that came to 
pass (which I agreed wasn't the most likely outcome at the time). I 
think if there is value even in the worst case scenario, it makes sense 
to try and support it. So I sent my follow-up mail to do just that.

Fast-forwarding to today, I made an effort to get a sustainable 32-bit 
package. I also wanted to gain reassurance I could easily deal with 
further issues should they crop up to truly alleviate my concerns about 
being blocked, hence "sustainable". This was not the first 32-bit build 
issue, and xenial is quite young. Thus I requested the juju core team 
provide a way of compiling without various provider support or other 
things that may trip up builds in the future. In addition, I worked 
again on trying to get a working 32-bit build myself with the idea of 
breaking providers, tests, or anything else as needed. With the caveat 
in mind that I am not a go developer, I discovered that as I tore parts 
away, more things started breaking. The build failures were not unique 
to a specific provider as we had thought, and making things work was not 
an easy task. I also began to seriously wonder how well juju would 
function with the amount of edits I was doing. I was ultimately not 
successful in getting a build, nor was I confident that the build would 
work reasonably. At the same time, the core team investigated breaking 
out provider by provider support and found it was going to be onerous, 
and represent a continual burden due to the intrinsic nature of go. 
Armed with the knowledge that this was not going to be easily solved by 
disabling providers, I don't think this failure changes the outcome.

Given the lack of results, I'm not confident about the 32-bit builds 
going forward. After my investigation and attempts to produce a current 
working build, I think the safest option remains the package removal. 
I'm even less convinced now that 32-bit builds can be supported in the 
future. Sadly, the window for discussion and exploring alternatives on 
this is closing. Juju 2 is releasing an RC today, and adoption will 
begin. I would like to resolve this matter asap. I want to get the RC 
into the archive for our LTS users especially, as well as yakkety for 
our QA and stabilization efforts there.

Nicholas