Outdated Laptop Test plan makes testing a less pleasant experience

Sun Oct 24 15:29:35 UTC 2010

2010/10/15 Daniel Kulesz <daniel.kulesz at informatik.uni-stuttgart.de>

Hello Daniel!

exc-001: There is no example folder / symlink anymore, the content was moved
> to /usr/share/example-content already some time ago. It is not clear which
> files / applications are targetted, and what the expected output is. (not
> only
> the applications should start correctly, but they should also handle the
> files
> correctly)
>

In fact I don't know how to handle this, because I don't know actual and
future plans about this folder.
Suggestions and comments are welcome.

> snr-001: Entering the password is not needed when testing via LiveCD, this
> should be mentioned.
>

Done.

> snr-002: Executing more than 30 cycles of sleep/wakeup could lead to
> hardware
> damage, limiting the cycles to 5 should be really enough!
>

This kind of test is present even in checkbox, I don't have an answer about
this.
Maybe a member of kernel team could clarify this.

> him-001: It is not clear, if this applies only to internal card readers
> only.
>

It should be clear because it's in the "system" section. External card
reader tests are in "USB" section (hum-*).

> khk-001: The expected behaviour is, that Ubuntu also shows the volume meter
> on
> screen; but on many older machines the volume is adjusted, but the user is
> not
> given appropiate (visual) feedback. This should be included in this test
> case
> as well, to spot the affected machines.
>

This test is about checking if multimedia hotkeys do the appropriate action
after pressing them. (e.g. lowering volume)
Some laptop hotkeys are not mapped yet and this need to be reported (for
example).
The issue you are reporting is a "visual" one and even if it doesn't seem to
be an hardware problem it should be reported as bug as well, specifying it
on the bug description.

> hsi-001: the media file "fables_01_01_aesop.spx" seems to be outdated,
> there
> is other media included in 10.10; It should be also verified, that the
> sound
> plays correctly and does not produce any stottering
> hhi-001: see hsi-001
>

Now it should be ok.

> his-001: Changing refresh rate not covered
> his-001: (video in general) there is no test case to test for fluent video
> playback (maybe except exc-001)
>

We are testing laptops, that should be important using crt monitors.
About fluent video playback,exc-001 test could be sufficient to cover it.
In fact, an application test (mediaplayer) should be considered, not an
hardware one.

> bfu-001: Not clear, what is meant by app - is this the whole operating
> system?
>

It should be ok now.

> hpu-002: Ubuntu has various bugs regarding wrong paper size (especially A4
> vs.
> letter) in some drivers, this should be verified here as well.
>

As mentioned before, this could be considered in an application test, not an
hardware one.
Anyway this shouldn't prevent you or someone else to report and link bugs
you find doing a test, even if not directly related to it.

> hpu-002, hds-001, hds-002, hds-003, hds-004 : missing here - at least some
> of
> them should be included into LaptopTesting as well, this is pretty
> important
> for many mobile users
>

Good point.

>
> Other, general aspects:
> * the template for the Laptop entries
> (https://wiki.ubuntu.com/Testing/Laptop/Reports/Template) is *very*
> incomplete. Most people seem to include info like lshw output, dmidecode
> details etc. - why is this all missing in the template? Therefore, instead
> of
> re-using the template, testers have to copy/paste the structure around from
> existing reports.
>

This is because the original template included just a table, then some
people start adding other info on their own without mentioning in ML or
changing template.
For Natty I'm going to enrich it with other data to be filled in by users.

> * the main wiki page (https://wiki.ubuntu.com/Testing/Laptop/Reports) has
> no
> consistency regarding the naming of the tested "editions"
>

What do you mean?
Putting the edition name in that page should be sufficient.

>
> It would make testing Ubuntu (Pre-)Releases a much more pleasant
> experience,
> if you could update the test plan and look into the mentioned issues. I
> strongly believe there are much more people out there who would be willing
> to
> contribute to systematic Laptop Testing, but finding the wiki pages and
> test
> plan in the current (not very useable) state could be pretty frustrating.
>

Thank you very much Daniel for your suggestions.
The laptop testing is a community project so everyone can help and
contribute updating testcases and suggesting improvements etc.
In the future feel free to keep on sharing your thoughts and proposing your
ideas here in ML so we can discuss together.

Regards,
Sergio.

P.S.: Apart from issues you mentioned, I can say that the main problem of
this project is the use of wiki pages to report results.
I proposed a blueprint to integrate the laptop testing in the ISO tracker,
and it'll be discussed at the Ubuntu Developer Summit for Natty [¹]

P.S.2: Thanks to people that helped updating testcases.

[¹]
https://blueprints.edge.launchpad.net/ubuntu/+spec/ubuntutheproject-qa-n-laptop-testing-tracker

-- 
Sergio Zanchetta
https://launchpad.net/~primes2h
<https://blueprints.edge.launchpad.net/ubuntu/+spec/ubuntutheproject-qa-n-laptop-testing-tracker>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20101024/e1b9f87a/attachment.html>