Re-evaluating Ubuntu's Milestones

Tim tim at feathertop.org
Sun Apr 22 07:50:56 UTC 2018


Hi Simon,


On 22/04/18 17:08, Simon Quigley wrote:
>> First my gut feeling is your going to see less community participation
>> because there is no tangible outcome, obviously this will vary depending
>> on the flavours community, some my do better, some may do worse.
> The goal is to have established testing weeks increase the number for
> *everyone* while lessening the burden for everyone.
I really hope is does work out that way for all you flavours
>> From a technical perspective having the archive frozen was quite useful,
>> it allows you to focus on a fixed target, rather than getting distracted
>> by a moving target that may well introduce further bugs.
> Actually, I have typically seen more bugs *fixed* immediately after a
> milestone release than introduced. Also, if we didn't have to spend our
> time working hard to release a product (Alphas and Beta 1), we could
> focus on improving and refining QA (both automated and manual) and spend
> time on that instead.
I think that has largely been because the freeze's have been a little to
restrictive, Unless it was a installer bug or super critical flavour
bug, it just would get put on hold. That is why I suggested relaxing
somehow the restrictions during freeze, to allow flavours to control
movement of packages during the archive freeze, The images wont
necessarily be stale, because your team will upload what you need and
then re-spin, without having to worry about random universe uploads etc.
>
>> LIkewise for
>> giving the flavour leads control over re-spins rather than depending on
>> daily builds.
> Flavor leads do have control over respinning images, but they don't have
> control over stopping them (maybe just pressing the "disable" button?);
> we have yet to hear back from the Ubuntu Release Team on that.
I am fully aware of that, more meant as an example, even if the
resulting image does not become an official milestone release, there are
many advantages to your dev teams, having the archive semi-frozen so you
have control (mostly, there is obviously some overlap between flavours)
of the resulting images. Flavours could leave there cron jobs running or
choose to stop them and manually re-spin. Reducing that moving target to
something more controlled is only going to benifet the testing week.
>> I think
>> traditionally there have been too many milestones in a cycle (perhaps
>> somewhat biased by GNOME's late release cycle), however I still think
>> the milestones serve an important purpose. If Ubuntu GNOME were still
>> spinning ISO's, I'd probably advocate for a more hybrid model, use the
>> more informal testing 'weeks' early in the cycle, then one beta and the
>> RC Milestones.
> This is precisely what I'm proposing. Get rid of Alpha 1, Alpha 2, and
> Beta 1, then rename "Final Beta" to just "Beta". Then we have the RCs,
> which aren't really a milestone.
Good!
>
>> As for the automated testing, I think is important, however my
>> recollections of so many milestone releases dealing with somewhat corner
>> case installer bugs, wonder how you will get 100% test coverage. Also
>> for some flavours the work maintaining these test cases may end up being
>> as much work as co-ordinating milestone releases. I would probably
>> recommend getting the automated testing in place before changing things
>> too drastically.
> I disagree; while putting in automated testing is definitely something
> that I believe in making a priority, we don't need to block on change
> for that. The goal is to "set it and forget it" (which, with the nature
> of how these tests are designed, can be done in that way), so while it
> may involve some initial investment, I think the return far exceeds the
> time spent.
>
Please do not underestimate that, it will never be set and forget, as
things evolve there are going to be test regressions, which won't
necessarily be bugs the software but bugs in the assumptions made by
tests. Just like we see now in test suites and autopkgtests across every
cycle.




More information about the Ubuntu-release mailing list