Patch Pilot report

[Andrew Bennetts]

Thanks for the feedback everyone.  Obviously the way I did the patch
piloting isn't set in stone, so it's good to have some criticism and
questions to help us figure out how to make it better.

Stephen J. Turnbull wrote:
[...]
>  > I took a "just do it" approach to piloting patches.
> 
> That's not really what I expected of a patch pilot ...
> 
>  > If the changes required were fairly small, or even moderate, it
>  > usually seemed to me to be easier to just make the tweak and write
>  > the missing tests myself, rather than teach and cajole the
>  > contributors to do themselves.
> 
> ... agreed, "teach and cajole" is expensive.  "Suggest and teach if
> interested" is less so, and a real investment in the future.  But
> that's just theory: is it worth it in your opinion, accounting for the
> fact that it surely means fewer patches are landed per pilot?

Well, one way to think about it is that I'm matching effort: if it took
someone say 20 minutes to put together a simple patch to fix some
trivial UI issue that's bugging them, it probably won't take me much
longer than that to write a missing test and apply minor tweaks.  So I
see it as saying “we value your contribution, and we'll show that by
matching it” (as well as showing it by making sure the contribution gets
used).

I'm not sure this means less patches landed per pilot, though.  I can
easily enough spend 20 minutes on a simple patch writing review
comments, replying to questions that arise from those, teaching people
about our testing APIs, etc, and probably more.  And the inevitable
context switches and round trips add to that cost.

I'm more uncertain about the long term effects... will this mean we
handhold contributors indefinitely when they might have graduated to
self-sufficient sooner?  My guess is no; a sufficiently motivated
contributor will learn from the example of the tests I write for them,
and try to do that themselves on the next patch, etc.  I think in fact
“here's what the tests for your patch should look like” is probably
almost as good as walking the person through writing them themself, if
done soon enough that their contribution is still fairly fresh in their
mind... I know if I were contributing to a new project I would read and
learn from that sort of relevant example.  I think for this to work it
has to be easy for them to find that diff, though, hence why I took to
pasting it in the review comments before landing it.

So, my speculation is thta it's actually not far off in terms of
teaching the motivated, and saves a lot of time on contributions where
the motivation was fleeting.  Hence I speculate that it's a net win.  I
emphasise the word “speculate” though!

>  > Also, I deliberately chose patches from new contributors over
>  > regulars.
> 
> I'm not sure I like the sound of that.  My understanding was that the
> patch pilot was supposed to help less experienced and/or one-off
> contributors to deal with the relatively technical aspects (proper
> docs and tests, for example).  That may be a misunderstanding, but you
> see where I'm going: if one is needed to get regulars past the
> Charybdis of PQM and the Scylla of committers too busy to review,
> there's a constricted bottleneck that needs shattering.

I think you misunderstood me?  I piloted patches from the less
experienced and one-off contributors.  I basically ignored contributions
from regulars all week, because I knew they would be taken care of with
no pilot intervention.

-Andrew.