Updating Lubuntu's testcases

[Chris Guiver]

Lubuntu's QA is mostly handled on
https://phab.lubuntu.me/w/release-team/testing-checklist/ with a note
at the top that doc is supposed to be unnecessary after
https://phab.lubuntu.me/T56 which pushes us (Lubuntu) back to
iso.qa.ubuntu.com...

fyi: most install tests are done by Leó (Leokolb) & myself; and
personally I like our checklist as I can choose the oldest test
performed & redo/update it.. instead of a fresh page for each daily...

We have some 'updated' checklists, created long ago which never
completed review.

1.   Is there a guide to the formatting, ie. I wanted more than one
line at the top so created two <em> lines... (emphasis?) but is there
somewhere where what I-can-do | I-cannot-do is defined?  I gather
labels are DL = descriptive.list, DT = description term within DL, DD=
description & EM = emphasis/strong

2.   Tests end up MANDATORY or OPTIONAL, where is that set?

3.   Is there a tool where I can view the created testcase in somewhat
PREVIEW state (without codes) so I can re-read & hopefully detect
errors?

4.   Is there a guide for reviewers I can read?

Walter (wxl) originally created the list in our QA checklist; the
issue is we now have lots (a guide to understanding them I created
here - https://discourse.lubuntu.me/t/testing-checklist-understanding-the-testcases/2743),
and I sure don't want them all mandatory. I also consider the FOUR
BIOS installs as roughly equivalent (variations of encryption,
internet & swap), our FOUR EFI installs the ~same, as well as FOUR
Secure-uEFI.. To counter this I've a testcase [script] that attempts
to get a tester to select one & run it (so one bios can be mandatory,
one uefi mandatory, one secure-uefi..) but that's also more complex
than the four testcases each script replaced..

Any advice or direction would be appreciated.

Chris g.

(my current focus is focal.5 QA, so this is intended for after that release)


---  Example testcase
<em>Lubuntu QA-test: Full disk install, on BIOS hardware</em>
<em>Proceed in your native language if you wish. Instructions will
remain in English</em>

<dl>
    <dt>Boot up the image and select your language</dt>
        <dd>The system boots properly and loads the language selection menu</dd>
    <dt>Press enter over the "Try or Install FAMILY" text</dt>
        <dd>The system should boot to the live desktop</dd>
    <dt>On the desktop, click the "Install FAMILY XX.YY" icon to open
the installer</dt>
    <dt>Select your language/localization in the dropdown menu and
click "Next"</dt>
        <dd>The installer should be localized in your language</dd>
    <dt>Select your timezone and click "Next"</dt>
        <dd>A map should appear allowing you to select your timezone,
and if the internet is connected, should already be accurate</dd>
    <dt>Select your keyboard layout and click "Next"</dt>
        <dd>Test the keyboard layout in the given box, which should be
set correctly</dd>
    <dt>Select the "Erase disk" and options that match one of the
following install types"
        <dd> "testcase: full disk, no-encryption, BIOS, internet, swap" </dd>
        <dd> "testcase: full disk, encryption, BIOS, internet, no-swap" </dd>
        <dd> "testcase: full disk, no-encryption, BIOS, no-internet, swap" </dd>
        <dd> "testcase: full disk, encryption, BIOS, no-internet, no-swap" </dd>
        <dd>Please include a brief description of your hardware in the
first comment line on iso.qa.ubuntu.com</dd>
        <dd>Please copy and paste your chosen testcase into the second
comment line on iso.qa.ubuntu.com</dd>
        <dd>Please ensure you see BIOS listed besides your drive. If
you see EFI it's covered in another testcase, meaning there is a
problem with the installer (please file a bug report) OR your hardware
doesn't match this install type</dd>
        <dd>A partition layout should be shown which uses all disk
space, and the bootloader should be installed to the correct disk</dd>
        <dd>Please select drop-down to match your chosen swap/no-swap
option</dd>
        <dd>Please also select encryption if chosen, and enter
passphrase as requiredtwice</ddt>
        <dd>Confirm Boot Loader location is correct</dd>
        <dd>If "Erase disk" does not show, your target device has a
mounted partition. Please close installer, umount your partitions
(possibly a swap partition that was mounted on boot), and return to
the 'click the "Install FAMILY XX.YY" icon step'</dd>
    <dt>Enter details about the main system user and click "Next"</dt>
        <dd>All details (name, username box-name etc) should be
correctly filled in</dd>
    <dt>Confirm the details you entered throughout the install are
accurate and click "Install", then "Install now"</dt>
        <dd>All details (location, keyboard, partitions etc) should
match what you entered</dd>
        <dd>The slideshow should correctly display, and when the
installation is done, a menu should display asking you to confirm your
settings</dd>
    <dt>Allow the machine to reboot</dt>
        <dd>You should be required to enter the encryption passphrase
before login</dd>
        <dd>The system boots properly and loads into FAMILY showing
the username you entered</dd>
</dl>
<strong>If all actions produce the expected results listed, please <a
href="results#add_result">submit</a> a 'passed' result.
    If an action fails, or produces an unexpected result, please <a
href="results#add_result">submit</a> a 'failed' result and <a
href="../../buginstructions">file a bug</a>. Please be sure to include
the bug number when you <a href="results#add_result">submit</a> your
result.</strong>