----- Original Message -----
Sent: Tuesday, August 24, 2004 3:06
PM
Subject: RE: [ebxml-iic] comments on
use case #2, and on termination of test case
Mike:
That
would be satisfying: as long as we have a way to exit from a thread
(abort) without
exiting
from the test case.
The
only argument I'd have against your position, is that it is not always on
the "intuiton" side
(e.g.
you spawn an error-detection thread that will explicitly exit on failure
as soon as an error message is received
(<WhenTrue><Exit>Fail...) , but if you forget the
<WhenFalse>Continue,
- which
additionally may sound odd if at the end of a thread - then this thread
wil still Fail the entire test case...)
[MIKE3] - Understood.. this wasn't the best example I could
think of..
So we
have now 5 possible actions to follow a Test Assertion result (either True
or False):
-
continue (or Continue_Thread?) (default for
Assertion=True)
- abort
(or Abort_Thread?)
- exit
Fail (default for Assertion=False)
- exit
Pass
- exit
Undetermined
Am I
right?
[MIKE3]
- Yes. We are implmenting now.
Additionally:
- if
the test case terminates its execution (all threads?, or just main
thread?) without exiting either on Fail or Undetermined, then it
Passes.
[MIKE3]
- Add "exception" in there as well and yes
[MIKE4] - A test case terminates
ONLY when (anywhere in the workflow) it encounters an explicit exit (insde
of <TestAssertion>, or an exception condition (system error).
All other operators (continue, abort)
simply continue or re-route execution. When a Test Case
terminates, all Threads are stopped, and the main Thread (TestCase) sets
the
final
state (pass/fail/undetermined/exception) of the Test
Case.
- If a
thread aborts, its container thread (or caller thread)
aborts.
[MIKE3]
- No.. why do we need to do this??? Why kill the parent/caller
Thread in which you may wish to do your join???? Also, why not let
logic
contiue
to a final resolution?
- if
main thread aborts: test case Fails? (abort is a logical meaning) or just
Undetermined?
[MIKE3]-We'll go with undetermined, and this will only occur
at the <TestCase> level ( treating TestCase as the "main"
Thread). It simplifies scripting greatly,
achieves the same goal. Having a "main" Thread is
only meaningful from a semantic view.. TestCase is in f act the
main Thread.
Comments? ( We need to move quickly on resolution of
this)
Jacques
Jacques,
Below are some of my thoughts
regarding "abort". I think that it is a useful and
necessary function for concurrent Threads
that will later be "orJoined".
I think however, that <Abort> should
be explicitly set by the Test writer, since, in 99% of
<TestAssertion> operations in a typical Test Suite,
a boolean result of "false" for a
<TestAssertion> will signal a defaul exit condition for the
Test Case, with a final result of "fail". This is the case
for
ebMS testing, and will likely be the case
for Registry testing as well. BPSS testing will of course have more
complex, concurrent Thread opterations
that would benefit this
functionality. However, making <Abort> the default logical
branch on a failed <TestAssertion> does not represent the typical
use case
in our test scripting.
Also, requiring the test writer to
"micro manage" every <TestAssertion> in the test suite, and
explicitly set a
<WhenFalse><Exit>fail</Exit></WhenFalse>
and
<WhenTrue><Continue/></WhenTrue>
is (in my opinion) against the intuitive
meaning of a <TestAsssertion> operation, where "true" = "pass" and
"false" = "fail". Such
default behavior could be "overriden"
in the case of a (rare) "abort" situation. Forcing explicit
declarations of branching will require unnecessary
and labor-intensive micro-scripting for
each Test Case, where more intuitive default behavior rules for <TestAssertion> could handle the
majority
of test cases.
I would like to propose
keeping the existing implied <TestAssertion> logic, but adding an
explicit <Abort> option for <TestAssertion> that would let
the test writer abort a
(concurrent) Thread if a particular <TestAssertion>
fails.
Comments?
Mike
----- Original Message -----
Sent: Monday, August 23, 2004
4:01 PM
Subject: [ebxml-iic] comments on
use case #2, and on termination of test case
Mike:
I think we need to tighten the semantics of a Test
Assertion failing without explicit exit statement:
I am concerned that these implicit rules we have will be
confusing on complex test cases that have concurrent
threads...
(see my comment in 1.1.4, attached)
I would suggest we consider the following
proposal:
- no "fail" outcome is produced
unless an explicit fail exit statement is met during exec.
- no "pass" outcome unless an explicit pass exit
statement is met during exec.
- a failed
assertion without explicit exit statement, by default will "abort" the
thread, but just the thread.
- a passed
assertion without explicit exit statement, by default will "continue"
the thread.
- when threads are joined, an
aborted thread will automatically cause failure of an and-join (which
aborts the container thread). In case it is an or-join, the aborted
thread will just be ignored by the or-join (the or-join will fail if
all joined threads abort). If a thread that was split but never
joined, aborts, then it just stops and is simply ignored for the rest
of the test case exec and outcome.
- if the Main thread of a test case aborts, the
outcome is "Undetermined" by default (this is the only case of
implicit outcome, in addition to other explicit "Undetermined"
outcomes)
Some additional Comments starting p.5 of the
attached doc (mostly, use case #2).
Also, I was searching for the section where we
specify the test step timeout (MaxDuration?) , in the draft
spec,
and did not find it (same for the
"sleep" statement).
Cheers,
jacques
<<section7.1-JD1_MK_JD2.doc>>
To unsubscribe from this mailing list (and be removed from the
roster of the OASIS TC), go to
http://www.oasis-open.org/apps/org/workgroup/ebxml-iic/members/leave_workgroup.php.