Gherkin DSL for testcase description and automation

[Руаньяк]

Hi all,

After some hot debate in the mailing list about re-writing and
categorizing testcases, I've decided to propose an adoption of another
testcase DSL: Gherkin.
Current ISTQB definition doesn't seem to be elegant enough to quickly
describe the expected behavior of the system. As for me, these tests
with actions separated from expected results are not very readable, as
the tester has to do too many action while performing the test: a)
perform the action, b) look down for expected result, c) verify the
result, d) look up for another action and so on.
Also, current testcases can be used for manual testing only and it is
not directly suitable for Mago or any other automation framework.

There is also another way to describe the behavior of the system,
developed to be readable and directly used for automation: Gherkin
DSL. See more info on the links sections

This is a sample test case written using ISTQD definition for Gnome Calculator:

---
Test Case ID: TC-CAL-002
Test Case Description: Check that sums are calculated
Assumptions: Gnome Calculator is started, mode is switched to the Basic mode

Actions:
1. I input "1+1"
2. I click '=' button
Expected Results:
1. '1+1' appears in the calculator textbox
2. Result is 2
3. No error appears
---
As you may see, there several flaws in this testcase description:
1. As we are using lists in testcase description, it won't be easy to
add more steps in the middle of the scenario, especially if testcase
is long
2. We have to use 2 expected results (result is 2 and no error
appears) for one action (I click '=' button), as there two things to
be checked after this action.

And here is a sample in Gherkin DSL:
---
Feature: Basic Mode

 Scenario: TC_CAL-002: Calculate sums
   Given Calculator is in Basic mode
   When I input "1+1"
    And I calculate result
   Then Result is "2"
    And No error occurs
---

This description is much more readable and can be easily copy-pasted
in bugreport, as it doesn't have lost of unnecessary info, lists and
can be broken anywhere with actual error description.

Note, that automation suites can easily use the same scenarios. The
only thing implemented is just actions for steps, which are basically
procedures (using i.e ldtp), which use regexp to parse steps. If you
guys need more info on this, I will write a separate post on
automation with Gherkin DSL.

There also some more interesting stuff in Gherkin - such as tagging,
step argument transform and step reusing - I guess, I can describe
them in other posts, if this whole propasl will be interesting.

Hope to hear for more ideas and thoughts.
Vadim Rutkovsky

-----
Links:
Gherkin DSL examples and structure:
 https://github.com/cucumber/cucumber/wiki/Gherkin
 http://code.google.com/p/spectacular/wiki/WritingBDDTests
 http://en.wikipedia.org/wiki/Behavior_Driven_Development#Application_examples_in_the_Gherkin_language

Sample automation code for GCalc:
 https://code.launchpad.net/~roignac/+junk/freshen_mago