The New Charm Store Review Process

[Clint Byrum]

Excerpts from Jorge O. Castro's message of Tue May 22 07:11:53 -0700 2012:
> Hi everyone,
> 
> If you weren't at UDS we're streamlining the review process for juju
> charms. I'm going to propose the summary of all the changes here for
> discussion on the list to see if there are any burning issues that
> people would like to see resolved. Unless we have major questions I'd
> like to move to this process as soon as we can (end of the week at the
> very latest) so we can start working on this backlog!
> 
> ## Quick Summary (read this): The adhoc review process was light and
> loose, but we're getting busy now so we're going to do patch review
> and piloting how Ubuntu does it, using their same policies and tools.
> If you're in the https://launchpad.net/~charmers/+members#active group
> and aren't going to be available to do reviews, you should leave the
> team!
> 

Thanks for getting this out Jorge, this is going to be great!

> ## The Proposed Review Process:
> 
> - https://juju.ubuntu.com/CharmsProposedProcess
> 
> The big change here is instead of using the "new-charm" tag for
> reviews we're asking people to just subscribe ~charmers. When
> ~charmers is subscribed to something it gets put in our handy dandy
> queue (thanks Juan Negron, Kapil T, and Daniel Holbach):
> 
> - http://jujucharms.com/review-queue
> 

Note that this does not have everything as we transition between new-charm
and subscribing charmers. The bugs will return soon.

> The queue is sorted by age, and we will be measuring 2 things. Average
> review time, and first response time. We will then use those numbers
> to adjust review scheduling so our review process is fast, correct,
> and friendly towards our contributors. As you can see we're at a
> whopping 34 items. (The contest really smoked us!) Some of those items
> are old and just need to be finished off. This is why this queue is
> awesome, no lost items lost in the cracks. We've also purposely
> included not only new charms but old charms that are getting updates,
> so that we can be responsive to both contributions.
> 
> If a charm is old and broken and you're not getting a response you can
> just unsubscribe ~charmers and remove the tags (We support the
> new-charm still but we'll stop using it from now on) and if the charm
> is broken the maintainer will get pinged and if the charm still stays
> broken it gets flagged for removal or whatever by the charm testing
> (Mark/Clint, perhaps a follow up mail detailing how we're doing charm
> testing?)
> 

We're still getting that in shape, not much to report just yet.

> ## Reviewers
> 
> This section covers how to review:
> https://juju.ubuntu.com/CharmsProposedProcess#Reviewers
> 
> You should be in the ~charmers team. From now on if you are in
> ~charmers I am assuming you are available to review and I will
> schedule you to do reviews. If you do NOT want to do reviews, remove
> yourself from the team! Core devs, you own the ~charmers team so I
> will assume you're not available for reviews unless you explicitly add
> yourself to ~charmers. (We could use the help!)
> 
> Ubuntu uses 4 hours a month for it's reviews. That is, you're expected
> to review for at least 4 hours during your assigned shift. Canonical
> employees in ~charmers, please double check with your manager if 4
> hours a month of dedicated review time is acceptable, I know many of
> you are already doing more than that. Daniel Holbach has provided a
> handy tool that lets me schedule you automatically, so I can handle
> the scheduling that will schedule your work calendar. Community
> members, I of course won't be assigning you review days, so feel free
> to work on the review queue at your own leisure on whatever interests
> you.
> 
> ### Catching up the queue
> 
> I think 4 hours a month with the people in ~charmers is fine, Clint
> mentions that it's better to have one concentrated shift per person
> instead of smaller split up shifts, but we can adjust accordingly.
> Right now my main concern is the queue is up to 34 and we now have a
> tool to keep track of this, so I would like to propose I just schedule
> all of you for reviews one after the other for the next week or so
> until the queue is in way better shape, then we go to a more
> predictable 4 hour/month cadence.
> 
> Thoughts?

+1. Perhaps check with people on what the best days of the month or week
would be best so we don't butt up against meetings and such.