Modification to test plan layout

[Maciej Kisielewski]

Hey!

I'm totally with you on making definitions more readable.

But...

We're using RFC822 for all our definitions and it's painfully limited in
what it can handle.
Currently a test plan is a one, flat record (flat meaning no subrecords),
and for instance the whole `include` field is just one field, and rfc
doesn't differentiate if it's a list or not.
If we were to have a tree structure, just like one you proposed, we'd have
to do a bit more complicated parser for checkbox and write a lot of
docs/tests to make it good. And this would just be Yet Another Checkbox
Syntax. And you probably know where I'm going with this :) Your example is
strikingly similar to YAML, so if we are to do a radical change in how we
define jobs and other units, my vote would go for something standard, like
YAML (with all its tools, metas, tests, all that jazz.

Other benefits:
- The pxu -> yaml translation could be very much automated (and would help
validate the effort).
- I think we all agree that rfc822 sucks (think multiline entries)
- SPEEEEED - after my optimisation efforts from last year, at current HEAD,
most of the time spent while waiting on *box to spool up, is the pxu
parser. I'm expecting yaml parser to be 'a bit' faster.
- From a snappy perspective: kind of unification of description languages.

On Wed, Mar 22, 2017 at 7:42 PM, Jeffrey Lane <jeffrey.lane at canonical.com>
wrote:

> Forgive me if this is already planned out, but it occurred to me that
> adding more and more "features" to tests means we should make sure
> test plans can follow the same format as job descriptions.  This
> already happens, really, but I'd propose ensuring that the add-ons are
> also treated the same way, rather than simply tabbed out from the job
> name.
>
> This would allow the addition of any number of items to a test without
> being terribly wide or unweildy.
>
> An example might look something like this:
>
> testcases:
>     mandatory_include:
>         some tests in a list
>     include:
>         name: testA
>             options:
>                 retry=true
>         name: testB
>             options:
>                 certification-status=blocker
>                 retry: true
>     bootstrap_include:
>         name: testF
>         name: testH
>
> Additionally, it seems to me that the status option is a bit overly
> complex.  Rather than:
>
> certification-status=blocker
> certification-status=non-blocker
>
> I'd propose simply looking for the keyword blocker. like this:
>
> name: testA
>     options:
>         blocker
>         retry
> name: testB
>     options:
>         retry-once
>
> especially if there no real use for the "non-blocker" status.  Since
> lines without any certification-status option and lines with
> certification-status=non-blocker seem to be handled the same way, the
> only one that really has any effect on results display is "blocker".
>
> So I thought I'd share the idea...
>
> Thoughts?  Can we do something like this?
>
> Jeff
>
> --
> "Entropy isn't what it used to be."
>
> Jeff Lane -
> Server Certification Lead, Warrior Poet, Biker, Lover of Pie
> Phone: 919-442-8649
> Ubuntu Ham: W4KDH                          Freenode IRC: bladernr or
> bladernr_
> gpg: 1024D/3A14B2DD 8C88 B076 0DD7 B404 1417  C466 4ABD 3635 3A14 B2DD
>
> --
> Checkbox-devel mailing list
> Checkbox-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/checkbox-devel
>



-- 
Have a good one,
Maciek