Modification to test plan layout
jeff at canonical.com
Fri Mar 24 17:42:19 UTC 2017
Interestingly, The first draft of that message was suggesting YAML,
but then I changed it to RFC822 since we already use that for the job
descriptions, and it seems a bit odd to have two different definition
templates for the same tool.
The YAML looked something like this: (maybe not the best design, but
it's kinda what I was imagining).
Anyway, it was an idea I had, maybe in some future modifications, such
a thing could be put on the board.
On Fri, Mar 24, 2017 at 6:36 AM, Maciej Kisielewski
<maciej.kisielewski at canonical.com> wrote:
> I'm totally with you on making definitions more readable.
> We're using RFC822 for all our definitions and it's painfully limited in
> what it can handle.
> Currently a test plan is a one, flat record (flat meaning no subrecords),
> and for instance the whole `include` field is just one field, and rfc
> doesn't differentiate if it's a list or not.
> If we were to have a tree structure, just like one you proposed, we'd have
> to do a bit more complicated parser for checkbox and write a lot of
> docs/tests to make it good. And this would just be Yet Another Checkbox
> Syntax. And you probably know where I'm going with this :) Your example is
> strikingly similar to YAML, so if we are to do a radical change in how we
> define jobs and other units, my vote would go for something standard, like
> YAML (with all its tools, metas, tests, all that jazz.
> Other benefits:
> - The pxu -> yaml translation could be very much automated (and would help
> validate the effort).
> - I think we all agree that rfc822 sucks (think multiline entries)
> - SPEEEEED - after my optimisation efforts from last year, at current HEAD,
> most of the time spent while waiting on *box to spool up, is the pxu parser.
> I'm expecting yaml parser to be 'a bit' faster.
> - From a snappy perspective: kind of unification of description languages.
> On Wed, Mar 22, 2017 at 7:42 PM, Jeffrey Lane <jeffrey.lane at canonical.com>
>> Forgive me if this is already planned out, but it occurred to me that
>> adding more and more "features" to tests means we should make sure
>> test plans can follow the same format as job descriptions. This
>> already happens, really, but I'd propose ensuring that the add-ons are
>> also treated the same way, rather than simply tabbed out from the job
>> This would allow the addition of any number of items to a test without
>> being terribly wide or unweildy.
>> An example might look something like this:
>> some tests in a list
>> name: testA
>> name: testB
>> retry: true
>> name: testF
>> name: testH
>> Additionally, it seems to me that the status option is a bit overly
>> complex. Rather than:
>> I'd propose simply looking for the keyword blocker. like this:
>> name: testA
>> name: testB
>> especially if there no real use for the "non-blocker" status. Since
>> lines without any certification-status option and lines with
>> certification-status=non-blocker seem to be handled the same way, the
>> only one that really has any effect on results display is "blocker".
>> So I thought I'd share the idea...
>> Thoughts? Can we do something like this?
>> "Entropy isn't what it used to be."
>> Jeff Lane -
>> Server Certification Lead, Warrior Poet, Biker, Lover of Pie
>> Phone: 919-442-8649
>> Ubuntu Ham: W4KDH Freenode IRC: bladernr or
>> gpg: 1024D/3A14B2DD 8C88 B076 0DD7 B404 1417 C466 4ABD 3635 3A14 B2DD
>> Checkbox-devel mailing list
>> Checkbox-devel at lists.ubuntu.com
>> Modify settings or unsubscribe at:
> Have a good one,
Jeff Lane - Server Certification Lead, Tools Developer, Warrior Poet,
Lover of Pie
Ubuntu Ham: W4KDH
Freenode IRC: bladernr or bladernr_
gpg: 1024D/3A14B2DD 8C88 B076 0DD7 B404 1417 C466 4ABD 3635 3A14 B2DD
More information about the Checkbox-devel