ARB Review

[Jono Bacon]

On 11 April 2012 08:33, Allison Randal <allison at ubuntu.com> wrote:
> On 04/11/2012 08:03 AM, Jonathan Carter (highvoltage) wrote:
>> For what it's worth, Andrew and I have set up a weekly timeslot where
>> we'll be looking at ARB packages together (starting next week Tuesday).
>> I'll probably set up another slot like this with another ARB member
>> (most likely Stéphane). That might help the current package flow
>> problems somewhat. As with anything, if you don't make the time, you
>> won't have it :)
>
> Sounds like a good idea.
>
> I did some workload experiments over the past couple of months and can
> say that 1 person working full-time (5 full days a week) on the queue
> has no trouble keeping on top of the incoming new submissions, and
> catching up a little more each week on old ones. That's not a
> sustainable pace for one person (who also has work/school on the side),
> but it tells me that 5 people each dedicating 8 hours a week, or 10
> people each dedicating 4 hours a week would have a similar success rate.

I agree, Allison. As I mentioned in my original mail to the ARB a few
months ago, I think the primary bottleneck we are facing right now is
a people one and getting enough ARB eyeballs in front of the apps.
>From what I have seen of the ARB in the last few months some members
of the board have been much more active than others (as is common with
any community board), and your contributions have been a good example
of an active member. While I think that getting more ARB bums on seats
will help open up potential capacity, I worry that we might just get
more people on board who will also get busy with real life.

> More people will require more coordination, to make sure all those
> volunteers aren't duplicating effort (working on the same packages).
> But, we can ramp up coordination as we increase the number of volunteers.

I agree; more members will need more coordination and good leadership
to ensure everyone is motivated and keeping on top of things.

>> These responses are really the most common responses to submissions:
>> https://wiki.ubuntu.com/AppReviewBoard/Review/Responses
>>
>> Now to get the queue down we *could* just reject all the packages that
>> doesn't meet those criteria, but I don't think that advances the goal of
>> this project. It's probably better to help the people who submitted
>> their apps to do a proper submission.
>
> Yes, I think we've kept requirements to a minimum, and the approval
> process is fast now that we've got it down to an email vote. The
> greatest time seems to be spent responding to developers, explaining
> what's wrong with their submission, and then re-reviewing when they
> resubmit (often looping several times, when their second, third, and
> fourth submissions also have problems).

This does indeed seem to be an area where things slow down the most.
On one hand, as I have suggested, for developers that take weeks to
respond if ever, I think we should have a fairly explicit date in
which a response is required.

I also wonder if there are any other ways in which we could automate
some of the manual reviewing and checks going on. Maybe we could
develop scripts or improvements to ease this?

> The weekend before last I revamped the submission guide, and started
> adding links to it in responses to developers. This is helpful in giving
> the developers more details, but in a standard way so the reviewer
> doesn't have to spend much time on any one reply to a developer:
>
> https://wiki.ubuntu.com/AppReviewBoard/Submissions

Nice work!

> I'd like to get this linked in from developer.ubuntu.com, so more
> developers see it *before* they submit.

Agreed. This is something David Planella in the 12.10 cycle.

  Jono


-- 
Jono Bacon
Ubuntu Community Manager
www.ubuntu.com / www.jonobacon.org
www.identi.ca/jonobacon www.twitter.com/jonobacon