[Ecosystem-engineering] Proposal: New review queue and review process

Marco Ceppi marco.ceppi at canonical.com
Wed Sep 3 19:43:29 UTC 2014


I wanted to update everyone to let them know the first iteration of this
review queue is out. While it's a very bare MVP it's the direction I'm
taking the review queue in. I've written a blog post,
http://marcoceppi.com/2014/09/a-new-charm-review-queue/ which outlines the
features and the short term roadmap for the queue. There are still quite a
few things which need to be implemented, mainly the top navigation does
nothing and a few queues are empty but quite a few new items have showed up
in the queue!

Feedback, suggestions, comments, and pull requests welcome!

Thanks,
Marco Ceppi


On Tue, Aug 26, 2014 at 4:34 PM, Cory Johns <cory.johns at canonical.com>
wrote:

> Marco,
>
> I just wanted to follow up and say that this proposal sounds great to
> me.  The only addition I would make is clarifying what the logic is
> when there are conflicting reviews; e.g., if you mark something as
> TRIAGED (I don't suppose there's any possibility of getting a FIX
> ACCEPTED status added?) but I mark it as INCOMPLETE, does it get
> hidden / sorted down on the main review queue or not?
>
> In addition to having it formally defined, I also think it should be
> clearly and easily visible from the review queue itself; that is,
> there should be an obvious "How Does This Work?" link that displays
> all of this information for reviewers and reviewees alike to see.
>
>
> Thanks for working on this.  I think this will be a definite
> improvement to the review process.
>
> - Cory
>
> On Mon, Aug 18, 2014 at 1:59 PM, Marco Ceppi <marco.ceppi at canonical.com>
> wrote:
> > Hi everyone, I'm going to jump right in with this email be briefly
> > describing a short summary of the issues people have brought up around
> the
> > review process and the review queue, then outline what my proposal to fix
> > this is, outline the new process for feedback prior to implementation,
> > finally listing actions left to implement this. The current review queue
> has
> > been an indispensable tool for helping maintain the ecosystem as it is
> today
> > and the code that the new review queue is built on shares a lot of the
> same
> > logic/concept. However, since we've grown a lot in the last few years
> this
> > is an attempt to help bullet proof the process as we move forward.
> >
> > # Issues
> >
> > One of the biggest feedback we've gotten from users lays along the lines
> of
> > "where is my charm in the review process and what are next actions".
> > Currently, the review queue doesn't illuminate any next actions for a
> user.
> > As queue times increase this can leave a poor user experience. Another
> > feedback point we have seen recently are reviews being lost in the
> process.
> > This is arguably the primary reason behind a revamp of the review queue
> and
> > review process to ensure a submitted item for review does not get lost in
> > the process. These are two primary examples, while a more exhaustive list
> > exists these items have the most impact for users contributing to the
> > ecosystem.
> >
> > # Solution in Development
> >
> > For the past two weeks I've been working on a replacement review queue
> which
> > uses a lot of the initial code from the current review queue. A few major
> > points:
> >
> > * Ingests all bugs and merges regardless of state, owner, or series
> > * Tracks reviews once ingested
> > * Decoupled from charmworld
> > * API access
> > * Detailed statistic gathering
> > * Track reviews per user
> > * Track merges from LaunchPad and Github
> > * Priority sorting of reviews
> > * Track multiple queues of work at once
> > * Modular code base for extensiblity
> >
> > This is what has been done for the first version of the review queue
> which I
> > plan to release this week for review. However, as the review queue is
> > currently tracking ALL items (and not items for which only a charmer can
> > take action against) we will need to update the review process for
> charmers
> > and charm-contributors (Jr. Charmers) alike. Those proposed changes are
> > outlined below:
> >
> > # Proposed review policy changes
> >
> > ## New Charms
> >
> > With the new review queue, charmers being assigned to the bug is no
> longer a
> > requirement for it to appear in the queue. Any bug against the charms
> > distribution which isn't targeted at a source package (charm) will be
> > ingested. Currently, "NEW" and "FIX COMMITTED" charms appear in the
> queue.
> > Going forward, LP statuses will have more effect on the queue. "NEW"
> status
> > will appear in a separate queue. These are charms that are just being
> > submitted. The new queue is designed to get first contact to that user
> > faster. "TRIAGED" should be what a reviewer (who isn't a charmer) moves
> the
> > bug status to if it passes an initial review "FIX COMMITTED" will
> indicate
> > that the charm author has received review items and is ready for another
> > review, "INCOMPLETE" or "IN PROGRESS" indicates that someone has reviewed
> > the charm and it needs work to complete. "FIX RELEASED" when the charm
> has
> > been promulgated. All other statuses indicate an abandoned/closed review.
> >
> > ### Proposed workflow
> >
> > 1. User submits a new charm by opening a bug on Launchpad and linking a
> > branch to that bug
> > 2. It appears in the incoming review queue, any person (charmer,
> > charm-contributor) provides initial review (Welcome, thanks for
> submitting,
> > proof, etc) and outlines the process for the user by either linking to
> the
> > documentation or use of boilerplate text. If the charm passes initial
> > review, reviewer moves bug to TRIAGED, if it fails, moves bug to
> INCOMPLETE.
> > Author will need to move bug to "FIX COMMITTED" once issues are resolved.
> > 3. Once bug is in TRIAGED, a ~charmer will review the submission. If this
> > passes review charmer promulgates and moves bug to "FIX RELEASED",
> otherwise
> > places the bug in "INCOMPLETED".
> > 4. If further action is needed, once author resolves, moves bug to "FIX
> > COMMITTED" to have it re-reviewed.
> >
> > ## Updates to charms
> >
> > When reviewing merges to a charm, the merge proposal no longer needs to
> be
> > assigned to charmers to show in the queue, any merge proposal against a
> > promulgated charm branch will appear in review. Likewise, voting on a
> merge
> > proposal will not directly remove it from the queue! So feel free to
> vote in
> > the affirmative/negative often when reviewing merges. If a merge needs
> work
> > you will need to move the merge status from "NEEDS REVIEW" to "WORK IN
> > PROGRESS" which will remove it from the queue. The author then needs to
> move
> > the review back to "NEEDS REVIEW" for it to re-enter the queue.
> >
> > # Actions
> >
> >  * Seek feedback and approval of updated process
> >  * Release first version of review queue
> >  * Clean up outstanding (no longer valid) items
> >  * Update documentation to reflect new review process
> >
> > I'm interested in concerns, feedback, and suggestions as I continue to
> work
> > on an improved charm experience.
> >
> > Thanks,
> > Marco Ceppi
> >
> > --
> > Mailing list: https://launchpad.net/~ecosystem-engineering
> > Post to     : ecosystem-engineering at lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~ecosystem-engineering
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> --
> Mailing list: https://launchpad.net/~ecosystem-engineering
> Post to     : ecosystem-engineering at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~ecosystem-engineering
> More help   : https://help.launchpad.net/ListHelp
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20140903/dd0fd073/attachment.html>


More information about the Juju mailing list