Ensuring Quality in Ubuntu Translations

[Matthew East]

Sorry for the length of this email, I kinda wrote it as a blog post so
it's a bit more wordy that I'd like.

An interesting discussion on the #launchpad irc channel yesterday has
been making me think about the question of quality assurance for Ubuntu
translation teams. To put the question into context, this is how the
discussion arose:

An upstream GNOME translator for the Dutch language mentioned that often
there are complaints on their mailing list about the quality of
translations of the GNOME desktop environment for Ubuntu. He says that
these flame wars are particularly irritating given that the translations
complained of are not made by GNOME translators at all, but are
introduced by the Ubuntu translation team, overwriting the upstream
translations. One of the Ubuntu translation team joined the
conversation, and it became clear that there was very little QA going on
to ensure that the members of the Dutch team are (a) good translators,
and (b) familiar with the GNOME and other upstream translation
guidelines.

Thanks to the two Dutch guys who made a lot of good points, some of
which I've stolen in this post.

This is a topic which has particularly interested me recently because
the Italian translation group has been debating this question over the
last few months, and has evolved a quality assurance technique to try
and prevent this problem from happening. Also, I think this problem is
not just with the Dutch team, but is likely to be common. I'd be
interested to hear whether anyone else has experienced problems like
this.

Basically, my view is that the blame for this problem lies partly with
Rosetta, and partly with the translation teams themselves. In reverse
order:

= Translation Teams =

The basic starting point is that a central part of the Ubuntu
philosophy[1] is that software should be available to all in their local
language.

[1] http://www.ubuntu.com/ubuntu/philosophy

In order to achieve this, Ubuntu has given a lot of authority (and
responsibility) to the various translation teams that exist in
Launchpad[2]: these teams are responsible for what the operating system
looks like, because the translations which they enter in Rosetta will
eventually go into the operating system.

[2] https://launchpad.net/people/?name=ubuntu-l10n&searchfor=teamsonly

This is a lot of responsibility for the translation teams. It is clear
that randomly accepting any new member to a team can result in bad
translations. It seems that in the case of the Dutch team it has had
really bad consequences. I refuse to believe that this problem doesn't
exist elsewhere. For example, the Ubuntu French translation team has 250
members (and 1 administrator to approve/disprove new candidates!!), the
German team 86, the Brazilian team 78, etc. It's difficult to imagine
that these members have all been through some kind of quality assurance.

Upstream translators on the other hand _do_ go through rigorous quality
assurance. Translations are uploaded to (e.g. GNOME) CVS if the
translator is already well known for good quality translation, or
alternatively if the individual translation is checked first.

So how can Ubuntu translation teams do similar quality assurance? This
is where teams should share their experiences, in my view. So here is
what the Italian team does:
 * When a new member applies to join, he's asked to join the mailing
list and write a mail of presentation
 * In order to be admitted to the group, the member goes through the
following (relatively informal) process: 1. sign the code of conduct, 2.
create a wiki page on the italian wiki with contact details, 3. read the
upstream translation project guidelines, and the GNOME translation
guidelines, 4. begin translating in rosetta by submitting "suggestions".
 * When the proposed member has done #3 above, an existing member of the
group checks the suggestions the proposed member has submitted. If they
are ok, the proposed member becomes a real member.

Now, this process may not work for every team. Some teams have lots of
people, others not so many. Equally, this process is by no means perfect
(I'd be very interested to hear what other teams do). It's up to each
team to figure out what quality assurance system works for them.
However, a quality assurance system IS necessary, if the problems like
those experienced in the Dutch team are to be avoided.

What concrete proposals could assist here? I'd suggest that some common
"translator group guidelines" would do some good. But it's a very big
job to "reform" existing groups, both in terms of the amount of work,
and the delicacy of the social problems (it's important to get the
balance right between encouraging inclusive participation, and quality
assurance). However, if groups set up well thought out mechanisms for
quality assurance, I feel convinced that it is a job which can be
successfully carried out.

= Rosetta =

There are lots of ways in which Rosetta can and should help this QA
process, in my opinion. They are all fairly well known bugs, I think.
But they are important ones.

The first is technical. It is not nearly as easy to check a proposed
member's translations as it should be. This is a oft-cited bug in
Rosetta. It should be possible to go to a person's profile, and view
each suggestion that person has made for a translation. At the moment,
it is only possible to view which template the person has contributed
to, and then you have to go through all the untranslated strings for
that template, and look for the person's name. Not very convenient.

The second is technical too. You can't search a package for a particular
string, which means that if you see a bad translation, it's harder to
fix. Worse than that, once a translation is committed, there is no
obvious way of seeing who committed the translation, so people who are
not following guidelines cannot be approached to discuss the problem.

The third is technical and social. It would be a bad thing to make
upstream translations take precedence over Ubuntu translations (because
then it would be impossible to correct mistakes upstream, or alter a
translation where the context is slightly different in Ubuntu), but most
of the problems are caused by the fact that upstream translations are
not currently imported quickly enough into Rosetta. This results in
Ubuntu translators translating strings which they otherwise would not
touch, if the upstream translations were already there. Earlier
importing of upstream translations would save much of the pain that
Dutch Ubuntu users have experienced, I'll guess. (The social aspect of
this is that Ubuntu translation teams should AVOID translating until
they know that all upstream translations have been imported).

The fourth is purely social. The main reason that translation groups
don't do QA is that they are not aware of this need. Given that Ubuntu
has given the translator groups this immense responsibility, it is their
duty (and by implication, that of Rosetta/Launchpad) to make them aware
of it. New teams and team owners/administrators should be made aware of
the importance of assuring quality translations in the distribution. The
other reason that Rosetta needs to take on this social task is that
Rosetta really does make translation very very easy indeed, which rocks.
However, it's vital to ensure that "easy" doesn't equate to "sloppy".

= Conclusion =

My conclusion is that Rosetta helps to go half way towards fulfilling
the promise in Ubuntu's philosophy of making the operating system
available to users in their local language. However, now for the hard
bit: making the operating system available to users in their local
language and _professional_ at the same time. In order to do this,
translation teams need to put quality assurance in place and Rosetta
needs to help them to do this.

Matt
-- 
mdke at ubuntu.com
gnupg pub 1024D/0E6B06FF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/rosetta-users/attachments/20060410/8118dd3d/attachment.pgp