[App-review-board] ARB queue status

[Allison Randal]

Trying again on a reply that seemed to go to dev/null this morning...

Hi Daniel,

Thanks for this. Unfortunately I'm not sure the numbers are helpful, as
they cover a period where we halted reviews waiting on tools, and two
change overs, first to MyApps+Launchpad, and now to not using Launchpad
anymore (the activity in the past two weeks has gone into MyApps). The
MyApps developers are working on building similar statistics into the
reviewer interface.

We just started a "patch pilot"-like schedule, to have reviewers
clearing the queue each week:

https://wiki.ubuntu.com/AppReviewBoard/ReviewShifts

I had the first shift, and in that time was able to package one app from
scratch, do a code review on another, get another submitted upstream to
Debian (which has now been accepted, huzzah!), and comment on a couple
of others. So, I'm hopeful that we'll catch up pretty quickly this way.

And, we've also invited others to help do reviewing and packaging, as
non-voting participants. I had one person asking about it on IRC during
my shift, and another emailed me this week. Given your experience
building up community teams, perhaps you have some suggestions on how to
organize the non-voting reviewers? i.e. should we create a launchpad
team for them, sign them up with us for review shifts, etc?

You may remember I was concerned at UDS that the submission rate had
hopped up to 10/week, but wasn't sure if that was a rate that would
continue, or drop off as the newness of developer.ubuntu.com wore off?
It has dropped off to about 1-2/week. I hope that rate will continue to
grow, and I know the two David's have some ideas about how to promote
the portal to developers, but for now that means we have time to catch
up, build our regular workflow, and bring in non-voting reviewers.

One thing that still remains a difficult issue is installing in /opt. I
can package an app in an hour, and then spend 3 more getting it to work
in the non-standard install location. If it takes me that long, I'm sure
it takes a new developer substantially longer. This might be improved
with clearer documentation, showing how to solve common problems with
installation in /opt. Again, I think your experience could be helpful here.

Another blocker is when packaging an app reveals a bug in a toolkit the
app is using. The one I packaged last week has a bug in python-pyglet
with either bamf or the Unity launcher, that is causing the Launcher to
think it's an application called 'panel' and to show multiple windows,
even though it only has one. I'm not even sure who to report the bug to.
You can find the app in my PPA, called 'crabhack':

https://launchpad.net/~allison/+archive/ppa/+packages

If it weren't for this bug, we could have voted and approved this app
this week. It's a pretty little retro scroller game.

Thanks,
Allison

On 11/24/2011 09:40 AM, Bhavani Shankar R wrote:
> On Thu, Nov 24, 2011 at 5:00 PM, Daniel Holbach
> <daniel.holbach at ubuntu.com> wrote:
>> Hello everybody,
>>
>> Jono talked to David and me and asked if we could help the ARB somewhat.
>>
>> As a result of a long delay of getting the tools in place and everybody
>> being busy in their other roles, it seems there's a big problem with
>> latency.
>>
> 
> Yes Daniel, I respectfully agree with you in this regard.
> 
>> To assess the situation, I had a look at [1] and found the following:
>>
>> +-----------------------------------------------------+
>> |   bug  | in queue | longest time (in d) |    who    |
>> |        | (in d)   | between comments    |           |
>> +-----------------------------------------------------+
>> | 644066 |   429    |   224               |   ARB     |
>> | 691347 |   342    |   285               |   ARB     |
>> | 730196 |   263    |   206               |   ARB     |
>> | 758020 |   227    |   168               |   ARB     |
>> | 763937 |   221    |   164               |   ARB     |
>> | 861471 |    57    |    55               |   ARB     |
>> | 861480 |    57    |    43               |   ARB     |
>> | 861540 |    57    |    57               |   ARB     |
>> | 866067 |    51    |    51               |   ARB     |
>> | 866086 |    51    |    51               |   ARB     |
>> | 867660 |    51    |    51               |   ARB     |
>> | 877023 |    37    |    37               |   ARB     |
>> | 877029 |    37    |    37               |   ARB     |
>> | 877047 |    37    |    27               | submitter |
>> | 877057 |    37    |    37               |   ARB     |
>> | 881489 |    30    |    30               |   ARB     |
>> | 881515 |    30    |    26               | submitter |
>> | 881515 |    30    |    30               |   ARB     |
>> +-----------------------------------------------------+
>> |   18   |   ∅ 113  |  ∅ 87               |  89% ARB  |
>> +-----------------------------------------------------+
>>
>> This is quite unscientific, but I just counted how long the bug had been
>> sitting there already and what the longest time between a request for
>> comment and a sufficient reply was. Also does this not take into account
>> how many bugs have been closed already.
>>
> 
> Being a new recruit on the ARB, I'm getting a hold on the terms now
> and I'll definitely kick on very soon in this regard.
> 
>> Still it would be good to get the numbers down. I'll start a separate
>> conversation with the myapps team to find out if we can build in some of
>> these statistics.
>>
>> [1] https://bugs.launchpad.net/ubuntu-app-review-board/+bugs
>>
>>
>> Most of the requests above require the initial review, others require
>> some action to get the package in. What do you feel would help you? Are
>> the bug statuses / bug tags you use sufficient? Would it help for ARB
>> members to have a dedicated ARB day in the week where they spend some
>> time on the queue? Would you like to invite others to help review the
>> packages?
> 
> Bug tagging does help us a lot in identifying bugs and we have already
> a program similar to the patch pilot program in place (which was one
> of the action items decided in the ARB sessions at UDS)
> 
> https://wiki.ubuntu.com/AppReviewBoard/ReviewShifts
> 
> Secondly, One of the suggestions made at the ARB session in UDS was to
> create a team such as ARB-contributors and make it a open team for
> other developers who wish to help us in reviewing (and packaging).
> 
> Any other ideas welcome.
> 
> Regards