Clarifying Feature Freeze Definition

[Scott Kitterman]

On Friday, August 05, 2011 03:50:22 PM Kate Stewart wrote:
> On Fri, 2011-08-05 at 14:24 -0400, Scott Kitterman wrote:
> > On Friday, August 05, 2011 12:17:22 PM Kate Stewart wrote:
> > > Dear Release Team members, and others interested,
> > > 
> > >    In asking around and after reading the documentation I'm hearing
> > >    some
> > > 
> > > varied opinions of what it actually means to feature freeze.  On
> > > reading https://wiki.ubuntu.com/FeatureFreeze,  it seems to not be as
> > > specific as needed.
> > > 
> > > "At this point we stop introducing new features, packages, and APIs,
> > > and concentrate on fixing bugs in the development release."
> > > 
> > > My expectation is that the freeze is for all main packages, and seeded
> > > universe packages.
> > 
> > As Micah mentioned, this has applied to the entire archive.  We've gone
> > through iterations with different Main/Universe freezes and separate
> > freezes for New versions/New packages/New features and after much
> > experimentation concluded that one "Feature Freeze" was the least
> > confusing approach.
> 
> Agreed.  Wasn't sure of the back story, and if this is the approach
> folks are comfortable with,  that's good.
> 
> > > In WIKI it states:  "For the purpose of the FeatureFreeze, the upload
> > > date matters, i. e. all packages which are in the NEW queue by that
> > > time will be processed without the need for an exception."
> > > 
> > > So my question areas are:
> > > 
> > > 1)  Is everyone comfortable with this comfortable with using upload
> > > date and NEW queue?   How long after it being in the NEW queue does it
> > > need to be processed by?   I think we need to aim for them all to be
> > > processed within the week, but would like to hear what's reasonable
> > > from the AAs.
> > 
> > I wouldn't worry about it.  With limited exceptions, the ability of New
> > pacakges to cause regressions in the archive is minimal.  The primary
> > purpose of freezing New package acceptance is to reduce AA workload so
> > the people tasked with this work have more time for other tasks as we
> > get closer to release.  I also think that if something drags out a
> > really long time and it gets to the point where it's problematic, AAs
> > are experienced enough to know this and discuss it before accepting.  I
> > don't think the release team needs to explicitly track this.
> 
> I thought I remembered some grumbles in the Natty timeframe, which is
> why I was wondering if it made sense to have a goal of all the NEWs
> processed within a week, but would have to dig to see if its a false
> memory or not.  Its more a question of setting reasonable expectations
> rather than something to track.   I'll monitor it a little bit closer
> this time around, and if there are things dragging on over a week, we
> can figure out what's reasonable for next cycle, discuss at UDS.
> 
> One thing I'm worried about is the workload on the AAs at this point in
> the cycle.
> 
> > > 2) if a package is rejected just before FF due to a technicality,  does
> > > it need a FFE exception after?   What consititutes a technicality?
> > 
> > If something is in queue at FF and is rejected after FF due to a serious
> > defect we have processed such packages when re-uploaded after FF without
> > FFe. I think this is reasonable.  I think it's reasonable to process
> > packages rejected shortly before FF the same way.
> 
> What is an example of a serious defect?  also how long after do you
> consider it ok to accept without a FFE.    I'm thinking that within a
> week of FFE is ok,  but the day before we freeze for beta seems a bit
> questionable to me.  :)

We don't reject for minor defects, but file a bug and accept.  So a serious 
defect is anything that got a package rejected.  In general, I agree with 
those timelines, but I don't think we need to have a hard rule.  If an AA has 
a question about if it's reasonable to process it, they can always ask the 
release team to review for FFe.

> > OTOH, is someone uploads 20 new
> > packages the day before FF and most of them have to be rejected, I'm
> > probably going to suggest they are too late and should wait for the next
> > cycle and do a better job.
> 
> agreed in general,  caveat would be if its pre-cleared and is considered
> a release priority.  I'm thinking about some of Daviey's comments today
> about what's landing for Server...

Agreed.  The new package freeze is mostly about workload management, so if 
it's a priority and the resources are available, then it ought to go in (it 
would benefit in special cases like this, I think, from having the decision 
documented, so an FFe would be a good idea).  New package freeze is not about 
stopping people from getting work done, it's about freeing the regular AAs 
from spending time on New.  If there's a package and someone has time to 
review it, I think the release team should be pretty open to approving FFes.

> > Once again, I don't think the release team needs to manage this
> > very closely.  If there's a question/problem any AA can bring it to the
> > release team.  IIRC, in the past, we've done FFe for packages rejected
> > before FF, but it was really easy to get an FFe in that situation (as
> > long as it was shortly after FF).
> > 
> > > 3) by feature freeze, all new images for the manifest should be
> > > building daily images?  (I consider a new hardware platform as a
> > > feature ;) ).
> > 
> > I think this makes sense.  Put slightly differently, any new images after
> > FF would need an FFe from the release team.
> 
> Yes that's a better way to phrase it.
> 
> > > 4) anything else others have been finding unclear?
> > 
> > The only thing that I'm aware of (which I still totally don't understand
> > how it was unclear) was last cycle someone thinking U/I changes didn't
> > count as features.  If we haven't made it really clear that all feature
> > changes need FFe and U/I Freeze is the time after which ALL U/I changes
> > need review (not the feature deadline for U/I) then we need to.  I
> > thought it was perfectly clear before, so I'm not the right person to
> > judge if it's clear enough now.
> 
> I'll go back and look at this, and see if I can figure out a way to
> improve the clarity.   Possible scenario is a feature is already present
> and has been since before FF, but the defaults affecting what's
> visible/behaviour change.  ie. config bit changes, but all features are
> present.

The thing to be clear about is that between FF and UIF you can bug fix the U/I 
without coordination.  Sometimes bug fixes have radical effects because they 
unblock some bit of functionality, but that's not the problem we ran into last 
time.

> > > Would like to get some concensus,  and then broadcast this so the
> > > expectations are clear with the development community too.
> > 
> > I would caution you against trying to make the rules too specific.  They
> > should be parameters to which people should apply judgement.  If you
> > make them too specific, then people will start asking like rules lawyers
> > and try to figure out a way to interpret the words to mean they can do
> > what they want.
> 
> Fair enough.
> 
> Reviewing my notes from Natty,  the other area that would be good to get
> input on is what is the last cutoff point for a compiler/interpretter
> toolchain (gcc, eglibc, binutils or python, java, etc.) bug fix.   We
> had a bit of churn and angst from that in Natty.  Is it reasonable to
> make sure any bug fix for the toolchain packages goes through a FFE
> after beta 1?

I think after FF.  That doesn't mean that there won't be FFes and they won't 
get approved, but part of what I would want to see in such a request is an 
estimated impact (e.g. 50 no change packag uploads) and identification of the 
engineering resources to do that work (for all of the archive - particularly 
after FF I think it's imperative that anyone trying to drive change into the 
release own all the work associated with effects and related collateral 
damage).


Scott K