Developer Application Criteria - Was Re: New Application processes

[Jordan Mantha]

On Thu, Jan 8, 2009 at 10:25 AM, Dustin Kirkland <kirkland at ubuntu.com> wrote:
> On Thu, Jan 8, 2009 at 11:11 AM, Jordan Mantha <jordan.mantha at gmail.com> wrote:
>> I don't think that's necessarily a logical conclusion. You're saying
>> that if the +1/-1 of a MOTU Council member is based on a subjective
>> decision that they can't use objective data in making that decision.
>
> That is not at all what I'm saying.
>
> I'm saying that if a given Council Member (or even an existing MOTU
> who feels compelled to give feedback on an application) makes a
> criticism of a given candidate, and that criticism is based on
> measurable, quantitative, objective data, then such an objective
> threshold should be established.  If it's not going to happen at on a
> MOTU-wide basis, then, I'd hope the voting individual would take the
> initiative to define the threshold by which *they* are assessing
> applications.

My issue is that threshold may be different on a case-by-case basis
and from MC member to MC member. For instance a lack of experience in
packaging from scratch could be compensated by a wealth of merge/sync
experience and vice versa. However, I do think being more verbose as
to why you think a person is not ready would be a good thing to do.

> Perhaps something more like:
>  "I'm giving this individual a -1 because they simply have not been a
> MOTU long enough to be considered for Core Dev.  I believe that such
> an individual should be a MOTU for X months before applying for Core
> Dev.  At that time, I will reconsider my vote."

I don't think we can tell people how long they have to be a MOTU
before applying to Core dev, it just depends on too many variables.

> On the other hand, a more subjective criticism would be:
>  "I believe this individual lacks maturity."  or "So-and-so is
> difficult to work with."
>
>>> Criticisms of the form:
>>>  * You haven't done enough merges
>>>  * You haven't touched enough Universe packages
>>>  * You haven't been a MOTU or Developer long enough
>>>  * You haven't ...  enough ...
>>> should be invalid.
>>
>> Why? There's no a priori reason to reject such reasons just because
>> they are objective data. What you're really arguing against is that
>> objective data is being used to make subjective decisions and I don't
>> know why that would be a problem pre se.
>
> It's a problem because different people are held to different
> standards (see ScottK's poignant comments).

That's not a problem as I see it. That's why we elect the MOTU Council
in the first place, we don't have nor should have an across the board,
objective standard. If we see wildly different views of what it takes
to be a MOTU lets hash that out, but it's still a personal decision on
the part of each MOTU Council member.

>>> These things are actually measurable, and we could very well set the
>>> value of "enough".  If we are consciously choosing *not* to set these
>>> values, I think it's totally unfair to criticize someone for not
>>> achieving these arbitrary, dynamic, mystery thresholds.
>>
>> I very much doubt that "enough" could be easy to set. The fact of the
>> matter is that each MOTU will see that differently, and the point of
>> having a MOTU Council is to take in all the data and determine if
>> "enough" has been met. If it was easy to do we'd just have a secretary
>> go down a checklist to make MOTUs.
>
> I have never suggested that MOTU privileges be reduced to completing a
> checklist.  That's ridiculous.

Great, we agree. :-)

> I have suggested that "a minority component" of MOTU/CoreDev
> applications be based on some objective criteria.  In place of such a
> process, I also believe that Bryce's suggestion of a "workbook" would
> mostly serve the same purpose.

My problem is that this "minority component" will become the majority
component because it will be the only objective criteria. Bryce's
suggestion is probably helpful but we need to be careful about how we
word/suggest it.

>> My overall feeling with this thread is that perhaps we're attacking
>> the wrong problem. People are wanting to reject the criteria and the
>> subjectivity of a mostly subjective decision. I think rather the
>> problem is that the MOTU team has lost its collective understanding of
>> what being a MOTU is. The main argument has been "my sponsors told me
>> I was ready and I got rejected". I think there are some reasons why
>> this could be happening:
> ...
>>  4) people seem to take the MOTU applications much more personal
>> these days. "rejected" "criticized" seem to come up a lot when it's
>> always been "not yet, but keep trying". This perhaps is due to losing
>> some of that Ubuntu Ethos Jono has been talking about.
>
> These application process are inherently "personal", when you're
> judged by your peers on entirely arbitrary and changing criteria.

It should be neither arbitrary nor constantly changing. If what the MC
is looking for is unclear please ask them for clarification. That is
an orthogonal discussion to creating purely objective criteria for
MOTU membership.

>> I think perhaps effort would be better spent trying to figure out how
>> to make MOTU a *real* team again rather than trying to figure out how
>> to objectify an essential subjective decision. People should be
>> tapping into collective knowledge and understanding rather than
>> reading page after page of docs and running through checklists. We
>> should be a community rather than a MOTU mill. The self-assessment
>> checklist might help people figure out what we're looking for but I'm
>> afraid it would quickly turn into "but I can do the checklist, you
>> have to let me in".
>
> I disagree.  Here's an analogy...
>
> You want to attend XYZ private university.  They're a private
> institution, so they can accept whoever they want.
>
> In fact, most of the decision will be subjective.  However, before you
> get to that point, you have to meet some objective goals.
>  * You have to have a high school diploma.
>  * You have to have some minimum SAT/ACT test score
>  * You must have earned some minimum GPA.

Note that those imply fairly extensive and objective testing which is
something we have culturally believed we can not nor want to do. We've
purposefully said that the MOTUship process would not turn into Debian
NM. We've purposefully shunned purely objective criteria as we believe
it does not foster a team mentality and does not properly emphasize
the subjective qualities we want.

> Meeting the minimum objective criteria is clearly not enough.  That
> expectation is well established by your high school guidance
> counselors (ie, mentors).  But, you know what you have to do before
> you even apply.
>
> And once you've met these minimum expectations, the university's
> admissions board will evaluate the application in toto, taking into
> account all sorts of qualitative considerations (extracurricular
> activities, sports, essays, family history with the school, community
> volunteerism, leadership skills, languages spoken, etc.).

I think your analogy is somewhat inaccurate/incomplete for what we're
doing here. MOTU is not a university. A university uses these things
to accept/reject people who they have no prior knowledge nor personal
connection to. MOTU would be more like a university where existing
students recommend people they know to the university admissions
office. The current students should have a decent understanding of
what it takes to be a student at the university, if not, that's a
problem.

> Such processes allow us to "calibrate our expectations" (to use
> Bryce's term), as to whether we should apply to Harvard, our state
> university, or the local community college.  Or, more optimistically,
> help establish some guidelines and goals such that we can work our
> butts off to achieve certain criteria and one day apply to Harvard.

I think we *do* need to "calibrate our expectations" but that, IMO at
least, is a cultural and collective experience thing. We can still
hash out what our understanding of MOTU is (and this is what I
suggested in my email) without having to create these objective
criteria. If there is a disconnect between what MOTU and the MOTU
Council view as what it takes to become a MOTU then *that* needs to be
addressed.

A final point that I'm wondering is how often are people rejected
because of purely objective criteria? It seems to me that most people
are rejected based on things more like:
  * immature understanding of Ubuntu
  * doesn't play well with others
  * lacks overall packaging experience

Only the last one would have any chance of being objective but I don't
think that's even possible. You have things like # of packages
uploaded, difficulty of packaging tasks, # of mistakes, etc. What I
worry about it people picking the easiest things they can find in
order to meet some "standard" and then using that to leverage
themselves into MOTUship.

-Jordan