Micro release Exception for Nova, Swift, Glance, and Keystone

[Clint Byrum]

Excerpts from Martin Pitt's message of Mon May 14 13:09:49 -0700 2012:
> Hello Clint,
> 
> Clint Byrum [2012-05-06 12:12 -0700]:
> > Excerpts from Martin Pitt's message of Sat May 05 12:52:34 -0700 2012:
> > > The current SRU has been in -proposed for 5 months now, without any
> > > feedback about formal testing, and has been supeseded by a security
> > > update months ago as well. So based on this I think we'll need a few
> > > more trial runs before I agree to a general microrelease exception.
> > > 
> > 
> > I'm not so sure I agree that there has been no feedback.
> 
> There are certainly verified bugs, but I meant feedback in the sense
> of "we ran these test suites and these regression tests with the
> proposed version and they all succeeded".
>  
> > The nova SRU fixed *38* bugs. Our usual SRU verification process is
> > mostly cumbersome red tape compared to the review process that the stable
> > updates team for OpenStack used to get these fixes in. 8 of the 38 bugs
> > were verified.
> 
> Exactly, MREs with new upstream versions are usually like that: We
> don't verify that every single of many bugs is actually fixed, but we
> have a plan for regression tests with good coverage (which is the more
> important part for these updates).
> 
> > The whole point of these types of exceptions is that we get more fixes
> > shipped to users when an upstream project takes sufficient steps to
> > reduce regression potential.
> 
> I agree. But I haven't seen any regression testing, so I was wondering
> what happened behind the scenes there.
> 

So I think I understand what happened here then. I don't think it was
at all clear that evidence of a large suite of test runs was needed or
asked for. Perhaps there was just a disconnect between expectations. It
appeared to me that the expectation was we'd finish a few SRU uploads
of the relevant packages *with the usual SRU process*. With 38 bugs and
very few explicit test cases, that just didn't happen.

The upstream policy is pretty strict and gated with a fair amount of
regression tests. We also have our own CI lab running against the dev
release of Ubuntu. Perhaps we need to have the CI lab pull from proposed
when there is an upload, and verify that it all works.

I'd be willing to consider that "verification-done" if you were. :)