An idea on the structure of QA

Thu Mar 29 21:49:55 UTC 2012

Jiří, sorry for the delay in response.. We released beta2 this week --
all of QA was busy :-)

On the team growth questions -- there's no set standard for it. However,
all memberships can be set to expire, and many teams grant open
membership but have it expire after a few months. This keeps the head
count closer to the number of people actively contributing. If there
active when there membership expires, they simply renew it. If not, they
will drop off. Doesn't seem like a bad idea :-)

Nicholas

On 03/22/2012 10:15 AM, Jiří Kovalský wrote:
> Hi Nicholas,
>
> please see my comments in your text.
>
> On 21.3.2012 18:52, Nicholas Skaggs wrote:
>
>> Hi Jiří!
>>
>> Thank you for your feedback. Let me try and respond to some of your
>> questions.
>>
>> The document has been thru a few iterations, so likely the goals
>> aligning with the solution so tightly is an aspect of that. Those goals
>> were the goals I had in mind when I started down this road. The goals
>> are probably the most important piece of the document -- it's important
>> everyone in the community is unified around them. If not, it's hard to
>> discuss how to implement them; and harder still to achieve them if the
>> community is divided.
>
>    yes, I agree the goals are important. Everyone involved must pull
> the rope in the same direction. :)
>
>> The use cases is something I added to the document as a means of
>> thinking about the problem. Use cases are part of the template for
>> specifications -- and I believe they are included to remind you to think
>> about the problem from different perspectives :-) I want to keep all
>> these types of contributors in mind -- I outlined some current potential
>> scenarios as well as users in those scenarios. The proposal doesn't
>> directly address all of those use cases, but I wanted our plans going
>> forward to keep them in mind.
>
>    I see, we do it similarly.
>
>> On the team participation front, it's certainly possible to participate
>> in multiple teams under the proposal. This is the same as the current
>> structure; it is possible now to participate in multiple teams. How
>> effective someone can be at it is up to them and the requirements they
>> take on for both teams. I certainly don't see it as a harmful thing, but
>> I wouldn't expect it to be commonplace.
>
>    It's true that participation in multiple teams is rather an
> exception than a common case and from that point of view my question
> can be perceived as a useless concern. However, from our experience a
> typical NetBeans community tester can offer ~4.92 hours weekly which
> we prefer to be spent within just one team focused on one goal. We
> believe it brings more solid contribution. Views can differ of course
> and geeks who manage everything exist. :)
>
>> The question on membership renewal is a great one. Generally the
>> approach taken by ubuntu is that you are a member until you feel you no
>> longer can or wish to meet the requirements of being a member (afiak!).
>> Additionally, being a member or not, if your dodging your
>> responsibilities on the team for whatever reason, the team is likely to
>> re-assign them so as to not be hampered. I am intrigued by your new
>> testers comment -- do you feel you have done things to allow new folks
>> to be so productive in comparison to seasoned testers? Why do you think
>> they excel?
>
>    I would lie if I said I knew why :) but my personal interpretation
> is no matter how cruel it may sound that the experienced members are
> probably "burnt out" to some extent if you understand what I mean.
> It's like in "A new broom sweeps clean." proverb. Everything is new
> and exciting for the newcomers and they don't know how much is already
> enough. On the other hand the seasoned participants know exactly what
> they are expected to do and usually don't exceed that threshold. For
> example, Mark Wilmoth won in the last NetCAT 7.1 program [2] - a
> person whom we have not heard about before.
>
> [2] http://qa.netbeans.org/processes/cat/71/activity.html
>
>    However, my point was something else. There are always many people
> who only sign up and do nothing. While I understand that staying in
> such a watch-only mode can be interesting for some individuals, we
> obviously prefer active contributors. That's why with each new release
> we unsubscribe everyone and form a brand new team. Those who liked the
> program will immediately join again and those who didn't care will not
> bother getting back anyway. And of course this approach helps release
> those who became busy at work and hesitated whether to stay or go.
> When their job allows it, they will surely return.
>
>    From what you wrote I got that you only let teams grow. Is this
> correct? Do you measure and evaluate productivity of the teams
> somehow? If so, what are the trends? Does the gain more or less copy
> the head count?
>
> Thanks for your time too Nicholas!
>
> -Jirka
>
>> Great questions/comments-- I appreciate the dialog! Keep'em coming ;-)
>>
>> Nicholas
>>
>> On 03/19/2012 10:09 AM, Jiří Kovalský wrote:
>>> Hello Nicholas,
>>>
>>>     I am quite new to this mailing list so I apologize if my post will
>>> sound ignorant. :) Actually, I admit that my intention was to learn
>>> how community QA is organized at Ubuntu to get some inspiration and
>>> improve our own processes [1] at NetBeans.
>>>
>>> [1] http://wiki.netbeans.org/NetCAT
>>>
>>>     The new structure proposal is well written and clear to me. The
>>> Goals section though seems like you created it after the solution was
>>> found and not vice-versa as it should normally be in my opinion. Also
>>> I didn't understand the purpose of Use Cases section. Did you want to
>>> assign Mark, Jim, Kathy and Michelle to some team later in the
>>> document or these were only mentioned to keep the four basic types of
>>> contributors in mind?
>>>
>>>     Finally, I might have overlooked it in the text, but would it be
>>> possible to participate in some Infrastructure team and in another
>>> Testing team at the same time? If so, is this what you really want?
>>> And out of curiosity, would there be a membership renewal process? Our
>>> 8-years experience from cooperation with the NetBeans community is
>>> that although well known and seasoned testers are very useful, its
>>> typically brand new participants who excel.
>>>
>>> I hope this feedback is at least somehow helpful.
>>>
>>> Best regards,
>>> -- 
>>> Jiří Kovalský
>>> NetBeans Community Manager
>>> http://www.netbeans.org
>>>
>>> On 14.3.2012 21:03, Nicholas Skaggs wrote:
>>>
>>>> Hello everyone,
>>>> Today during the weekly QA community meeting, I shared my idea for
>>>> organizing the QA community to be more effective at communication and
>>>> working efficiently with each other, in addition to helping recruit
>>>> and
>>>> retain new members and grow. I'd like to also share this idea with the
>>>> mailing list and the community at large. I'll just repeat a little bit
>>>> of what was spoken about on IRC for reference. The full log is
>>>> available
>>>> here:
>>>>
>>>> https://wiki.ubuntu.com/QATeam/Meetings/QA/20120314
>>>>
>>>> The background on this proposal stems from my own attempts at learning
>>>> about QA in ubuntu. I went on a misson to list and catalog everyone
>>>> doing QA work in ubuntu (although I'm sure I missed some people,
>>>> and if
>>>> so, I apologize!). I posted the results of this on my blog the other
>>>> day.
>>>>
>>>> http://www.theorangenotebook.com/2012/03/whos-who-on-quality-in-ubuntu.html
>>>>
>>>>
>>>>
>>>> Once I had the list of teams, it became apparent that communicating
>>>> and
>>>> understanding everything that was going on was going to be hard. In
>>>> the
>>>> weeks following me creating my list, I learned about more teams, more
>>>> interesting work being done, etc. It seemed like when I would hear
>>>> about
>>>> a new tool I would find out someone else in ubuntu had used/was using
>>>> that tool and here was there work, etc. Given these experiences, I
>>>> started writing some thoughts about a proposal to organize the QA
>>>> community to meet 3 specific goals that I thought would be hard to
>>>> meet
>>>> under the current structure:
>>>>
>>>> Ease of Communication
>>>> Ability to recruit and retain community members
>>>> Ability to scale with growth potential
>>>>
>>>> These are also in the proposal, which you can read here:
>>>> https://wiki.ubuntu.com/QATeam/ProposedTeamStructure
>>>>
>>>> I'd like everyone to remember that of course this is just an idea.
>>>> I am
>>>> hoping to spark some discussion about solving the problems that I have
>>>> brought up. Namely, how can we better communicate as a diverse
>>>> group of
>>>> teams?; how can we work more effectively?; how can we grow our
>>>> community? Ideas and input on the proposal, as well as the
>>>> problems/solutions are very welcome. I want us to rally around solving
>>>> these issues, and come to the best solution as a community for us to
>>>> pursue.
>>>>
>>>> Lastly I wanted to bring up an important piece about the proposal.
>>>> It is
>>>> purposefully sparse on implementation details. I gave a proposed
>>>> structure, but I did not directly assign teams into that structure.
>>>> This
>>>> was intentional. I want us as a community to talk about specific teams
>>>> and the changes would happen to them as part of drafting a
>>>> blueprint to
>>>> implement this plan. To this end, the plan is focused more upon the
>>>> work
>>>> items we value and hold as part of the QA community and the people and
>>>> roles they can fill to accomplish that work. The specifics on the
>>>> teams
>>>> those people belong to, I see as a part of the next steps in
>>>> writing and
>>>> executing an implementation plan.
>>>>
>>>> The timeline of next steps is to gather feedback and discussion on
>>>> this
>>>> proposal, decide to move forward with a proposal (this proposal, a
>>>> modified version of it, or perhaps a different proposal entirely),
>>>> create a workplan and finally execute the plan.
>>>>
>>>> Thanks,
>>>>
>>>> Nicholas
>>