Vivid QA incentives report

Simon Steinbeiß simon at
Wed May 20 19:04:38 UTC 2015

Hey Lyz,

first of all: sorry it took me so long to reply and thank you for this great and comprehensive write-up!

The short version: Not all points in the argument convince me and I'm not sure we should jump straight to the conclusion of discontinuing the programme. (Maybe we should still do it, though. Thanks Lyz and Elfy for your hard work on this!)

The long version:

It's true, the handing out of stickers is sort of a "carrot and stick" situation. But then again it isn't. First of all, that is one tiny carrot. In my opinion, it's about as much as the "email from the project lead". So I'd say it depends on your point of view, whether this is truly "stuff" or just a sign of recognition like an email. (I agree that an email is less work than sending stickers by snail mail, so if the workload is the main argument for discontinuing, I won't argue.) Sending stickers automatically to contributors might be a bad idea indeed, so if someone doesn't want them, well, I see no reason to convince them to take them. But yeah, I agree that the size of the carrot is not the problem here in the sense that we would have needed a bigger one or that that would have made things better. I think it's the smallest possible carrot and in that I'm wondering whether it's truly doing the damage of devaluing the work. I for one am happy that you once sent me a sticker! :)

Another counter-argument is that we obviously didn't advertise this enough (the winners being surprised and not knowing about it), so it couldn't even work as a carrot in these cases. So us not gaining contributors could also be due to a lack of advertising the stickers programme or lack of an audience.

Looking at similar incentives, the bug bounty programme a la bountysource is an obvious candidate for evaluation under these psychological/behavioristic theories. It's true that bountysource doesn't work for everyone, at least with Xfce it didn't seem to bring in new contributors (at least so far). However, it also seems to work just fine for others like elementary [1]. So yeah, might really depend on the community/audience one has.

From where I stand it's really hard to motivate people who are otherwise not into QA for testing, or maybe FOSS in general. I wouldn't say that the stickers programme has worked, especially after your compelling argument (or the fact that we didn't gain new contributors) that it didn't. However, I guess the stickers programme could only work if people really took note of it and that's my general take-away from this: we probably need to do a better job at communicating with users or the community (as far as we have one, or if not, try harder to build one).
In the end that might be a more time-consuming task than sending out stickers and yeah, this is obviously nothing new.

As indicated in the short version, it might still be for the best to discontinue the stickers programme. It didn't help us get new contributors, it actually meant more work for the people in the team. Maybe we can also alter the programme so it doesn't cause that much work (like sending the stickers of one cycle out at once at release time) or try to make the stickers more of an offer than an automatic reward (a la: "Thanks for your contributions, we'd like to thank you with a sticker. If you want one, send us your physical address and we'll mail it to you at the end of the cycle, if not live long and prosper!"). Step forward if you have creative ideas or if any of my suggestions make sense to you.

Either way, I felt like chiming in here since I was one of the folks pushing this initiative at the outset, but obviously others did the heavy lifting. So first of all, a big thank you to Lyz and Elfy for taking care of this – even if we find that it hasn't resulted in what we had hoped for, I think it was worth a shot and if anything, this email is intended as a sign of appreciation and recognition of your work!



More information about the xubuntu-devel mailing list