Jacques and all,
Here are the latest spec and errata
changes, based upon email comments and "scripting syntax" thread.
I modified the scripting syntax again ( this
schema is different from the one I emailed to the list on
Friday).
It is based upon Jacques suggestion for
threaded/asynchronous test steps.
Sorry it took so long to get this out. I
had the flu last week and could not get caught up until today.
I believe that the XML syntax illustrated in the
attached TestStep.jpg file would provide all of the suggested
features
(Monica's either/or/concurrency/parallel and
Jacques threads and exception handling) with a minimum of (2) new XML
tags.
A summary of features is:
A <TestStep> can contain either a
"PutMessage" operation, "GetMessage" operation or a group of one or more
<TestStep> children..
[Jacques Durand]
<JD> For better readability of the Test Case, and also reusability of
parts of it, can we give a name to such subgroups of testSteps? i.e. TestStep
may contain just a reference to a sequence of TestSteps (something like a
subroutine call), instead[Jacques Durand]
of embedding the group of teststeps.
A <TestStep>, based upon success (boolean
"true") or failure (boolean "false"), can branch to an
optional <Continue> or <Exception> tag respectively
The <Continue> and <Exception> tags
are optional, and only necessary if one wants to provide a branching option
for a <TestCase>
A <Continue> and <Exception> branch
leads to another <TestStep> ( this is your if/then/else case
)
[Jacques Durand]
<JD> If <Exception>
does not convey really an other semantics than branching to another teststep, I would
rather use a lower-level, general branching statement instead. Maybe if/then/else
can do, or some "branch to".We also must keep in mind that Exception
situations may be triggered by reception of messages "in addition" to
expected messages, not just by failure conditions on expected messages.
(or, also by the lack of expected message)
For example: a (put + get) test
case may go through its "get" step successfully,yet if an error message is
generated in addition to the business response (before or even after), the test case should
fail. In order to catch the additional Error message, that could arrive at any time, an
"Exception" branch or thread should be started, even though no exception situation
occurred yet. The Exception branch will just wait for a possible error
message, and trigger failure only when getting it.So even if the "main branch"
of the test case completed successfully, the test case completion may have to wait
for the Exception branch to timeout without receiving an error, in order to pass. (but
again, an "exception" branch would not be different from any other branch started
asynchronously)).
A <TestStep> can be asynchronous ( parallel
) or synchronous ( serial ). By default, it is synchronous unless the
test writer sets the asynchronous attribute to "true".
A <TestStep> can contain a group of
<TestStep> children, connected by "and" or "or", based on the value of
the "connectivePredicate" attribute of the <TestStep>
<Exception> and <Continuation>
branching is based on the boolean result of the <TestSteps> grouped by
the "and" or "or" predicate for that <TestStep>
[Jacques
Durand]
<JD> Not sure if we need
to give a boolean semantics for the grouping of test steps: the branching is
the same: regardless of "or" or "and",start all these test steps
concurrently. Only, the
execution of the next step after the group, would be conditioned by an or-join or and-join.
Should we use such workflow connectors instead?
[Jacques Durand] I guess my
discomfort with a very declarative boolean logic, is that we loose sight of the actual
workflow, which is driven by external and timing events (messages received,
timeouts...) Using a test case execution
model that is closer to a state machine / workflow, seems more
intuitive.
<TestSteps> can be nested to any depth,
with boolean evaluation beginning at the lowest level in the nesting
and logically proceeding up to the top level.
<TestSteps> are executed serially or
in parallel, each based upon its asynchronous attribute value of
"true/false".
Boolean <TestStep> forks are evaluated
after completion of all asynchronous <TestSteps> for that
fork.
Groups of <TestSteps> are executed in
a single asynchronous thread if they are the children of an asynchronous
<TestStep>.
[Jacques Durand] means they are
serialized, although this sequence is itself concurrent to the spawning
thread, right?
You can have synchronous and asynchronous child
<TestSteps> within an asynchronous parent <TestStep>
thread.
This schema only adds 2 new ( and optional ) tags
to the current schema <Exception> and <Continue> ( plus an
"asynchronous" and "connectivePrediate" attribute to <TestStep>
)
This would add a lot of power and flexibility to
the current scripting.
I still need to test this model, and define BPSS
use cases to generate some POC test cases.
and I welcome any comments.
Cheers,
Mike