Summary of results of the Ubuntu IRC Survey.

[Jussi Schultink]

Hello all,

Here is the summary compiled by Juha (topyli) of the survey answers
from the Ubuntu IRC survey. Please note, this is a summary only - we
are planning an in depth analysis in the weeks to come, but we wanted
to get this out so people can follow progress. If you would like a
copy of this in a PDF format, you can find it at the following link:
http://jussi01.com/upload/uploads/20111125-083521-UbuntuIRCSurveypreliminaryoverview.pdf

Jussi


------------------------------------------------------

Ubuntu IRC Survey: preliminary overview of data

The ubuntu IRC Council conducted a survey for the IRC team during
November 2011. The results are in, and this is an attempt to pull
together a short summary of the data gathered. The survey consisted of
two kinds of questions. Multiple-choice questions were presented on
variables, in the style of “On a scale of 1 to 5, how happy are you?”.
Free form answers were requested for problem descriptions,
suggestions, and other suitable interest points.

On their own, the questions for determining satisfaction in various
areas were were not very useful, possibly due to being too general.
The free form questions were less frequently answered but offer
greater insight to the state of our community and organisation, as is
usual in any survey. Further work on this data should concentrate on
qualitative analysis, guided by the multiple-choice data as
appropriate.

This overview quickly narrates through the multiple choice questions,
and picks up a few recurring issues from the free form ones. The
purpose of this paper is to present the data acquired. No analysis has
been conducted, and no conclusions are to be drawn from this report.
There are likely be several omissions on the one hand, and subjective
focusing on other issues in this report. It is compiled very quickly
by one pair of eyeballs, and it makes no claim about its merit, other
than that it is as accurate and honest as I could produce within such
a time frame as I could afford for myself. Suggestions for further
work are provided as the conclusion of the paper.
Demography of survey participants

84 people took the survey. Of those, the majority were seasoned Ubuntu
IRC participants, with 81% reporting that they have been involved for
more than three years. They were also active IRC users: half of the
participants reported spending more than 20 hours on Ubuntu channels
per average week. For simplicity and for research-economical reasons,
the numbers and percentages in this report represent all participants,
including those with incomplete answers. 63 people completed the
survey fully. Channel operators, technical help providers, LoCo
leaders, and generally interested users are relatively well
represented among the participants.

Satisfaction with the Ubuntu IRC community

The most general question of “How happy are you with the Ubuntu IRC
community” yielded rather even levels of happiness throughout the
range, with 36% of completely or moderately happy people and 40%
unhappy ones. Respectively, “41% enjoyed being a part” of the
community, and 35% not so. The quality of service provided by the
community again divided opinions rather evenly (37% vs. 34%). When
questioned about a particular aspect of this sevice, keeping channels
free of spam, trolls and other problems, 45% of participants thought
the team was successful, while 35% did not.

34 participants answered the free form question requesting a list of
motivating factors in the community. A few common themes emerged
there.
Real-time support was quoted from both sides of the equation: on one
hand, the IRC community provides solutions and advice quickly, and on
the other, helping others offers instant gratification from the people
being helped.
The friendliness of the Ubuntu community and people.
Teamwork and collaboration provided by the synchronous medium.

Demotivating factors were also listed by 34 participants. Among these
where the occasional failures to deal with trolls and spam, but three
factors stand out that are related to our team:
Overworked and uptight operators, “collective burnout” as expressed by
one participant. Mistrust by ops toward users, forming a clique that
defends itself.
Poor communication. Council decisions appear at times to arrive out of
the blue, due to not being communicated well and early enough. Council
is disconnected from everyday work of the team.
Bureaucracy, heavy processes that make taking action slow. Problems
getting solved slowly and thus having the time to cause harm to the
team and users.

Organisation

Generally, respondents’ satisfaction with the organisation of the
community varied broadly, with 32% agreeing that it is well organised,
and 25% disagreeing. As many as one in five respondents had no opinion
on the matter. The high level of indecision continued with questions
regarding the IRC Council - a fifth of respondents did not have an
opinion on the quality of the council’s service, but with 30% being
unhappy however (vs. 26%). Only 53 participants answered this
question, pushing the failure rate to 17%. Nearly one third thought
the Council’s decisions are not representative of the community (30%
vs. 23%).

The structure of our documentation was good according to 25% of
respondents, while 25% disagreed. Another 25% had no opinion, while
the final fourth did not answer at all. More specifically, the pathway
to becoming an operator was clear according to 29%, and membership
through IRC contributions was easy according to 20%.

The usefulness of the team mailing list, the team coordination IRC
channels, as well as participation in IRC council meetings all yielded
similarly polarized results, with the IRC channels achieving the
highest approval rate of 30% and the mailing list lowest, at 21%.

Additional free form comments on team organisation was provided by 16
participants. Some notable ones include:
The contradictionary role of the Council: acting on feedback from the
team and serving it on the one hand, an having the responsibility of
ultimately making the decisions.
Disconnect between the Council and the Team.
LoCo teams can feel disconnected from decision making that happens in
the “center” of the Ubuntu project.
Suggestion about having each IRCC member responsible, offering
leadership, and acting as a contact person in different areas of the
team’s work.

General freeform suggestions

The questionnaire concluded with free form questions which also
deserve their own, thorough analysis. A few popular themes:
QA: ensuring that the help given is correct.
Better conversion of the helped to helpers.
Make documentation easier to find. Document the bots better.
Translated documentation.
Step down when you are no longer useful, as adviced by the CoC. Ways
to encourage people to take a break or retire.
Clearer understanding of everyone’s role. “Who do you call?”
Openness. The “IRC team” should more clearly encompass more than those
in official positions. IRC Team meetings. Reaching out to community
more.

Suggestions for analysis of this data

The free form questions received delightfully good answer rates, and
are the real outcome of this survey. If properly resourced to do so,
we should turn this into a series of interviews around the IRC
community, in order to further enrich the material.

Already as they are, the free form answers provide a moderately rich
data set for qualitative analysis, which should be conducted as soon
as possible. It is likely to be useful in developing the community,
the team, and their organization in the future.

The quantitative data provided by this survey is not likely to be
useful on its own, but can provide help in structuring the qualitative
analysis, which will produce the key findings.