The New Charm Store Review Process

[Jorge O. Castro]

Hi everyone,

If you weren't at UDS we're streamlining the review process for juju
charms. I'm going to propose the summary of all the changes here for
discussion on the list to see if there are any burning issues that
people would like to see resolved. Unless we have major questions I'd
like to move to this process as soon as we can (end of the week at the
very latest) so we can start working on this backlog!

## Quick Summary (read this): The adhoc review process was light and
loose, but we're getting busy now so we're going to do patch review
and piloting how Ubuntu does it, using their same policies and tools.
If you're in the https://launchpad.net/~charmers/+members#active group
and aren't going to be available to do reviews, you should leave the
team!

## The Proposed Review Process:

- https://juju.ubuntu.com/CharmsProposedProcess

The big change here is instead of using the "new-charm" tag for
reviews we're asking people to just subscribe ~charmers. When
~charmers is subscribed to something it gets put in our handy dandy
queue (thanks Juan Negron, Kapil T, and Daniel Holbach):

- http://jujucharms.com/review-queue

The queue is sorted by age, and we will be measuring 2 things. Average
review time, and first response time. We will then use those numbers
to adjust review scheduling so our review process is fast, correct,
and friendly towards our contributors. As you can see we're at a
whopping 34 items. (The contest really smoked us!) Some of those items
are old and just need to be finished off. This is why this queue is
awesome, no lost items lost in the cracks. We've also purposely
included not only new charms but old charms that are getting updates,
so that we can be responsive to both contributions.

If a charm is old and broken and you're not getting a response you can
just unsubscribe ~charmers and remove the tags (We support the
new-charm still but we'll stop using it from now on) and if the charm
is broken the maintainer will get pinged and if the charm still stays
broken it gets flagged for removal or whatever by the charm testing
(Mark/Clint, perhaps a follow up mail detailing how we're doing charm
testing?)

## Reviewers

This section covers how to review:
https://juju.ubuntu.com/CharmsProposedProcess#Reviewers

You should be in the ~charmers team. From now on if you are in
~charmers I am assuming you are available to review and I will
schedule you to do reviews. If you do NOT want to do reviews, remove
yourself from the team! Core devs, you own the ~charmers team so I
will assume you're not available for reviews unless you explicitly add
yourself to ~charmers. (We could use the help!)

Ubuntu uses 4 hours a month for it's reviews. That is, you're expected
to review for at least 4 hours during your assigned shift. Canonical
employees in ~charmers, please double check with your manager if 4
hours a month of dedicated review time is acceptable, I know many of
you are already doing more than that. Daniel Holbach has provided a
handy tool that lets me schedule you automatically, so I can handle
the scheduling that will schedule your work calendar. Community
members, I of course won't be assigning you review days, so feel free
to work on the review queue at your own leisure on whatever interests
you.

### Catching up the queue

I think 4 hours a month with the people in ~charmers is fine, Clint
mentions that it's better to have one concentrated shift per person
instead of smaller split up shifts, but we can adjust accordingly.
Right now my main concern is the queue is up to 34 and we now have a
tool to keep track of this, so I would like to propose I just schedule
all of you for reviews one after the other for the next week or so
until the queue is in way better shape, then we go to a more
predictable 4 hour/month cadence.

Thoughts?

-- 
Jorge Castro
Canonical Ltd.
http://cloud.ubuntu.com