release cadence and channels for the ubuntu-core snap

[Gustavo Niemeyer]

On Mon, Mar 7, 2016 at 9:24 PM, Leo Arias <leo.arias at canonical.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 2016-03-02 08:56, Gustavo Niemeyer wrote:
> > On Wed, Mar 2, 2016 at 11:46 AM, Leo Arias
> > <leo.arias at canonical.com We shouldn't be forced to use Beta just
> > because the Snappy infrastructure supports it. If we do use it, the
> > difference should be more meaningful.
>
> Of course we are not using a channel just because. The idea is to gate
> master/edge on pull requests, so we need to take good care about the
> time it takes to get a green result to allow the pr to land. We also
> need to care a little about the resources we consume, and about the
> likelihood of misleading errors due to infrastructure problems. So the
> first gate will be a subset of the second, with a balance that makes
> the devs and us happy. The second will be the whole enchilada, testing
> as many platforms as we can. As long as this second one takes less
> than 23 hours to complete, we are good.
>

If we have automated tests that can run daily to release to the beta
channel, why are we not running those for the edge channel instead?
Because people said they were fine with more risk? Huh?

> Stable:
> >
> > Once we are happy with the exploratory testing on RC, we will
> > promote to Stable. After the promotion, we will trigger the
> > automated tests again and verify updates and rollbacks.
> >
> >
> > Why?  Anything relevant happening here should have been done on the
> > RC. If we think we need to run tests twice because it's not quite
> > safe to run only once, we should do it twice on the RC itself.
> > After it reaches stable, it's too late. We'll have a massive number
> > of people "testing" it for us.
>
> Mainly, because it doesn't hurt much to re-run the tests. If they fail

it would be late, but hopefully not as late as if we wait for a user
> to report the bug. Other than that, the rc channel is not exactly the
> same as the stable channel. They are handled by almost the same code,
> but there must be a tiny "if" somewhere that's different. I have seen
> it all, so if a safety net is easy to get, why not?
>

Have you ever seen a safety net which was under the ground? That's why. :-)

The release candidate is precisely the point where we test snaps that are
about to go stable and be released. Any relevant tests should be done
there, and in these tests it does not matter that the snap came from one
channel or another. It's the exact same snap.

If you expect different results for different runs of the test, the test is
not a very good one, but even when that's the case we should run the test
multiple times before it reaches stable.

And after today's meeting with iahmad and jibel I was thinking that we
> can generate the image only if this last group of tests succeed. Then
> a problem would hit all the people who have an image already installed
> and autoupdate enabled, but it won't hit people who are just
> downloading the image.
>

Just think about that: this is basically saying we have automated tests
which could fail, and we're releasing snaps to final users at the same time
we are running those tests. But hey, that's okay because we're only
destroying half of their systems.


gustavo @ http://niemeyer.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/snappy-devel/attachments/20160307/902c8b8e/attachment-0001.html>