ARB Review

[cprofitt]

On Tue, 2012-04-10 at 16:04 -0700, Jono Bacon wrote:
> Hi Martin,
> 
> On 10 April 2012 15:10, Martin Albisetti <argentina at gmail.com> wrote:
> > On Tue, Apr 10, 2012 at 6:22 PM, Jono Bacon <jono at ubuntu.com> wrote:
> >>
> >> As such, I would like to propose that we have a discussion about these
> >> challenges to explore whether we can resolve these issues with the ARB
> >> or whether we need to explore another alternative to serve our app
> >> developers; I don't think solving this challenge is as necessarily as
> >> simple as recruiting more ARB members. This discussion can form the
> >> basis of a wider set of discussions at UDS in May in Oakland.
> >
> > I agree it's critical for us as a project to get this right, and new
> > apps flowing smoothly. Now rather than later.
> > I do wonder, though, why you think it's not as simple as recruiting
> > more members to the ARB, could you expand a bit on that?
> > I assume that there's some guesses you have that you may of not
> > expressed, so I'm interested in understanding those, without having a
> > ton of context, it seems like a classic lack of people/time.

I agree this is something we have to get right and soon.

> Happy to expand, Martin. I do think an expansion of the membership
> would help (and we should consider it), but I think that there are
> other areas in which we could drive other efficiencies with it's
> resources. A few examples:
> 
>  * Application Prioritization - the board does not have any kind of
> prioritization facility in MyApps that indicates that App A may be of
> more interest/more important/more valuable (or however else we define
> priority) than App B. I think given there are more apps than time,
> some kind of prioritization would be helpful to ensure that the apps
> of most interest to Ubuntu users get to the top of the list. One
> possible idea I had here was making the queue open and allowing the
> community to go in and upvote/downvote and add comments and for the
> ARB to take this content into considerable. Of course it could be
> gamed, but I suspect it would provide at least indications of interest
> and popularity.

I think there is some merit to this line of thinking. We do similar
things with bugs currently -- heat ratings -- 'it affects me too' --
etc.

>  * Regular Queue Review - from what I have seen there is no regular
> cadence around queue reviews. This has meant that the ARB have been
> taking a more ad-hoc approach of picking apps whenever each member has
> time as opposed to regular meetings and reviewing and moving the
> submissions along. I think a regular queue review (either on a mailing
> list or on IRC) could be useful.

I think cadence is important as well. Just as with OEM vendors we want
app developers to be able to depend on a cadence.

>  * Defining Scope - I didn't see this myself but a member of my team
> observed that some ARB members have fixed bugs in some of the apps in
> the interests of resolving the issues to get them through the queue.
> While an admirable contribution, time spent fixing bugs in one app
> takes time away from other apps getting a review so I think it could
> be useful to define the scope of app reviews better to focus just on
> reviews, not fixing bugs in the apps.

I strongly agree here. The review board should not divert energy away
from approvals while there are apps in the que.

>  * Being Stricter With Needs-Information - I have suggested this a
> number of times before, but I still see that applications that are
> waiting upon feedback from a developer languish in the queue. I think
> we should put a strong "if there is no response within a week, it gets
> rejected". This will put the responsibility in the developers hands to
> be timely, which I think is reasonable given that the ARB is a
> community team offering their spare time to provide these reviews.

I think with a que it would be better to push it to the rear of the que
after a period of time. If it is 'pushed' to the rear twice it should be
marked as abandoned and rejected.

>  * Review Assessment Criteria - I think it could be useful for us to
> discuss what kind of criteria the ARB are using to review apps in the
> queue. I suspect there are areas in which the ARB could provide a
> lighter-weight review while still providing the assurances they
> provide around quality and security. As an example, in my (humble)
> opinion, if an application installs, can be removed, and does not pose
> a security risk, it should get through. If the application is terrible
> quality or crashy, the ratings and reviews will tell that story to our
> users.

That is a double edged sword to me. Applications in the Ubuntu
repositories need to have a certain quality level or users will
invariably place some blame on Ubuntu for the resultant crashes.

I know when I add a repo, ppa or install from a .deb that I am taking
that risk, but when it comes from the repositories there is some
assumption that it has been tested and works well with Ubuntu.

> Hope this helps.

That expansion helped frame the discussion for me. This is a very
important issue and I look forward to the discussion so we can find
solutions that help us solve this challenge.

Thanks,

Charles