Improvements to the developer approval process

[Iain Lane]

Hi,

[ Not sure why you set Reply-To to yourself only, but I'm going to keep the
mailing lists in the loop so that we can actually have a discussion. I
suggest others do the same too. ]

On Fri, Jul 08, 2011 at 03:22:44PM -0700, Jono Bacon wrote:
> Hi All,
> 
> Firstly, apologies to the DMB for not including you in the original
> discussion. I reached out to the TB because this is a general Ubuntu
> technology policy discussion, but I should have included the DMB. It was
> a rookie mistake, and I apologize.
 
No worries, fixed now.

> [...] 
> I thought it could be useful to restart the thread with both the TB and
> the DMB included. I have also been doing some more thinking on the
> topic, and consulting with some of you further on the phone, and some of
> the thoughts here have evolved from my mail to the TB.

Who did you speak to on the phone? Can you provide some notes from these
calls?

> A few people have asked for examples where these cases have occurred.
> Some community members have indeed shared concerns with me privately
> about areas in which they felt the developer approval process was a
> little too heavyweight or intimidating, and I am conscious to respect
> that privacy, but I think the issues I raise here are still of value
> without the specific context.
 
Fair enough. I'd appreciate it in future if you'd encourage people to
additionally communicate with the DMB though. After all, being on the other
side of the process to applicants means it's hard to see the other
perspective.

> I will begin by re-stating some of these challenges I see we face, and
> some thoughts on a solution. None of these are big policy changes, and
> none should result in increased bureaucracy.
> 
> > Challenges <
> 
> I see two primary areas that we could improve:
> 
>      1. There are two components in an assessment: the summary of work
>         and the testimonials. Sometimes it appears that missing pieces
>         from the body of work will overrule strong +1s from core-dev or
>         MOTU developers in the testimonials. I trust our developers more
>         than anyone, and I believe two strong recommendations from two
>         respected developers should get someone most of the way to an
>         approval.

I don't know about the words 'strong' and 'respected', but speaking
personally I do consider the endorsements quite strongly when considering
applications for upload rights. These wiki pages list what we expect of new
developers. DMB members assess against these criteria, and they are oft
quoted

  https://wiki.ubuntu.com/UbuntuDevelopers
  https://wiki.ubuntu.com/DeveloperMembershipBoard/ApplicationProcess

I'm sure we can reword the above to be more clear to both applicants and
assessors alike.

Fundamentally we want two things: enough evidence to assess that an applicant
knows what they are doing as pointers to previous good work and testimonials
from existing developers that this person should be given the upload access
they seek right now.

> [...]
> I would like to propose a few adjustments to how the DMB assesses
> people:
> 
>       * Before The Application - today everyone is welcome to apply to
>         be a developer. We should still provide this door in to people,
>         but I recommend we strongly encourage that people only apply
>         when another developer recommends them to. This would present
>         two options for how they end up at an approval meeting:
>               * Developer Recommendation - if an approved developer
>                 recommends they apply (which would infer a testimonial
>                 from that developer), the candidate should also strive
>                 to find another developer to +1 their application. The
>                 DMB would consider these developer-recommended
>                 applications (complete with two developer testimonials)
>                 with a high-probability of success.
>               * Applicant Applies Out Of Free Will - if the applicant
>                 just decides to apply (without recommendation from an
>                 approved developer), the DMB should dig a little deeper
>                 into their application.
>       * The Preparation - we should ask every applicant to provide their
>         summary of work as we do today, but to strongly require
>         testimonials from either core-dev or MOTU developers. An
>         application with zero or one testimonial from a developer should
>         not be reviewed.
>       * The Assessment - the assessment will assume that the candidate
>         is suitable to be an Ubuntu developer and work to find
>         weaknesses that the applicant can focus on and assess of those
>         weakness are strong enough to result in a -1. There are two ways
>         of assessing the candidate:
>               * Developer Recommendation - if a dev asks the applicant
>                 to apply and there are at least two strong testimonials
>                 from core-dev or MOTU applicants, the DMB should assume
>                 approval unless a valid reason for rejection can be
>                 found.
>               * Applicant Applies Out Of Free Will - if the applicant
>                 applies on his/her own free will, more investigation
>                 should be made into their application (if there is a
>                 lack of developer testimonials). 

I'm not really sure what practical changes this entails. Currently it is
highly unlikely that someone would apply without existing developers
assenting, as our process strongly recommends this. So it seems like this is
quite similar to what is done already.

I guess what would change is that somewhere this presumption of acceptance
would be written down somewhere. If that were to be the case, then it would
make the DMB's work even more open to question and sniping than it already
is. I don't think that would be a positive move.

*** Let me make myself totally clear again. Currently, without any changes to
anything, endorsements from an applicant's peers should be afforded
significant weight by the DMB when considering an application. I can't speak
for anyone else, but I definitely consider endorsements to be the most
important part of an application. ***

>       * If a candidate is rejected, the DMB should provide a list of
>         reasons and recommendations for methods in which these
>         weaknesses can be addressed and a recommendation for when a
>         re-evaluation could occur. This will provide a sense to the
>         candidate that we want them to succeed and we will advise in
>         areas in which they can resolve issues.

Actually, this is something we could approve on, and I think it would be very
beneficial if we were to do this. If an application is not approved, those
members who did not vote in the affirmative should briefly outline why they
felt the application was deficient, and what steps the applicant can take to
rectify this.

> So, in a nutshell, if the DMB has an application and the applicant was
> asked by a developer to apply for approval, and there are two
> testimonials from devs, we can reasonably assume they are likely to get
> approved. Any areas of weakness would be noted as action items for
> improvement with a recommended plan for meeting those needs.
> 
> I think if we encourage the above approach and that people apply when we
> recommend them to, it will increase the success rate of applications and
> we will see fewer DMB rejections, as we will see better and more
> equipped candidates applying.

Here's the premise for this discussion. I read it as saying that the DMB is
currently too strict on applications, and that there's a presumption around
rejection. I must admit that recently I don't feel like this is at all the
case. In my experience the board as a whole very rarely defers applications.

This is probably because applicants are self selecting — people are not
likely to apply until they are either pushed by other developers or are
certain themselves of being accepted — but I'm not sure this is so terrible:
it leads to this presumption of acceptance. And is in any event not mitigated
by the proposal.

The interview process may seem onerous to applicants, but people should be
aware that the DMB is interested in their success, and that our function is
to /grow/ the developer community. We're not in the business of turning away
good people unnecessarily.

So, to summarise, I'm not entirely clear what the problem is. It's my
impression that we do give enough weight to endorsements and that we are
fulfilling our goal of growing the developer community pretty well. I guess
it might be helpful to this discussion if you could review some recent
meeting logs and point out specific examples of where you feel we could have
done better*. We can then discuss reasonable changes in response.

Cheers,
Iain

* Yes, one of our current major failures is that we often do not reach a
  quorum. We should think about how we can fix this. One option is to /not/
  require a quorum, but just require any outstanding votes to be cast on the
  mailing list. Or we can get more strict on board members.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/technical-board/attachments/20110711/f6e8a0e7/attachment.pgp>