Should we use launchpad to track manual|automatic tests for Ubuntu flavours (kubuntu, lubuntu, xubuntu, etc)?

[Elfy]

On 04/01/14 20:01, Javier Lopez wrote:
> Hi,
>
> Happy new year to all QA ubuntu members =). Since some time ago, launchpad has
> been used to track the progress on manual|automatic tests for Ubuntu and derived
> flavours (kubuntu, lubuntu, xubuntu, etc).
>
> https://bugs.launchpad.net/ubuntu-manual-tests
> https://launchpad.net/ubuntu-autopilot-tests/
>
> Recently however, Paul White has pointed out that some of the flavours use
> other mediums to track progress on this aspect, eg. Kubuntu uses
> https://trello.com/kubuntu.
>
> https://bugs.launchpad.net/ubuntu-manual-tests/+bug/1183934
>
> Should keep using launchpad to track manual|automatic test for ubuntu derived
> flavours?
>
> Greetings
>
Happy new year to you too :)

My position is simple enough.

*Manual Testing.*

If a flavour team wants to use the trackers then they should.

If I'd seen this discussion before the bugs for manual testcases were 
created, I would at that point have been -1 to a mass creation of 
testcases for flavours without any discussion with them.

It should be up to a team how to deal with their QA infrastructure.

Different teams have differing resources - I'm positive that the only 
way that we (Xubuntu) managed to get the testcases we did written, onto 
the package tracker and actually tested was down to a lot of work by a 
very few people at the beginning.

It was of some concern to find ~50 bugs all tagged Xubuntu - I didn't 
see any discussion on any mailing list about this.

Now that we have the testcases - the tracker itself mostly looks after 
itself. Though actually getting tests done and reported is a different 
story.

As far as Kubuntu goes - if they aren't using, and seem to have no 
inclination to do so - why shouldn't the Kubuntu testcase bugs be marked 
as invalid - as they surely are that ;) [1]

I see that ubuntu-studio has 53 outstanding testcase bugs - they have 
less people than we do.

Edubuntu has 65 of them - though I've no idea how many people there are 
available to do their testcases.

*Autopilot testing*

I can see the value in there being testcase bugs raised here - once the 
testcase is done then the need for people to get involved reduces 
remarkably - I assume.

I'm surprised to be honest that there isn't a massive list of these. 
Perhaps because it  would seem that autopilot was set up purely for 
Ubuntu - which is understandable - we are having massive problems even 
getting anything to run for us in our environment, it would certainly 
have been of enormous help for us to have automatic testing - it is 
looking increasingly likely that we'll not ever be in a position to 
really take advantage of this.


*Summary*

In summary - it seems to me that we're looking at discussing something 
now that should have been discussed earlier.

But the main thing we should look at is whether a flavour has any 
intention of using the package tracker. If QA as a team hasn't touched 
base with them - then perhaps we should be.



Apologies if I did miss mailings.

regards

Elfy


[1] https://lists.ubuntu.com/archives/kubuntu-devel/2014-January/007653.html




-- 
Ubuntu Forum Council Member
Xubuntu QA Lead

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20140104/ae562960/attachment.html>