----- Original Message -----
Sent: Tuesday, August 24, 2004 1:08
PM
Subject: Re: [ebxml-iic] comments on
use case #2, and on termination of test case - more comments
Jacques,
Following up on
yesterday's discussion, I would propose a modification to the logic
rules:
----- Original Message -----
Sent: Monday, August 23, 2004 4:01
PM
Subject: [ebxml-iic] comments on use
case #2, and on termination of test case
Mike:
I think we need to tighten the semantics of a Test
Assertion failing without explicit exit statement:
I am concerned that these implicit rules we have will be
confusing on complex test cases that have concurrent threads...
(see my comment in 1.1.4, attached)
I would suggest we consider the following proposal:
- no "fail" outcome is produced unless an explicit
fail exit statement is met during exec.
[MIKE] - I suggest that "exit/fail" should remain the
"default" behavior for a <TestAssertion> with a "false" result, even
in a concurrent thread
(for example, what if a concurrent Thread just
"listens" for Error messages if any are received?) The
default
behavior under the current logic scheme is for the
Test Driver to "exit/fail" if a <TestAssertion> finds an
Error message.
Under the proposed new logic, the default result would
be that the the Error Listening Thread would "abort", and have no
effect on the outcome of the Test Case.
I suggest that if we wish to <AbortThread>, that
we explicitly say so in the script, otherwise keep the logic for
<TestAssertion> simple and consistent throughout the Test Suite so
as to
minimize confusion:
<Thread>
<TestAssertion>.... (some XPath assertion)
</TestAssertion>
<WhenFalse>
<AbortThread>
</WhenFalse>
</Thread>
[MIKE2] - Additionally, I would suggest
(just to keep the rules as simple as possible) that <AbortThread> be
permitted for "serial" Threads as well. I do not see any reason
why/how this would have a negative effect on execution flow.
Basically, this would say that an "aborted" serial Thread simply exits (
for whatever <TestAssertion> result in that Thread) and workflow
continues wiht the next instruction after that Thread. This can
be thought of as the serial version of the "dangling concurrent
thread", whose result is not consequential to the ultimate result of the
Test Case. This keeps the rules simple for all Threads.
Comments?
[Jacques
Durand] Everything above and before is OK to me, with some
reserve on the notion of "serial" thread. I assume "serialization of
threads" has the following semantics (since I assume that
splitting/joining are the only way to combine threads
here):
Thread
Main
split
T1
join
T1
Split
T2
join
T2
Split
T3
join
T3
end
Main
Thread
T1
...
end
T1
Thread
T2
...
end
T2
...So I
assume your "serial thread" notation is just a shortcut for
the above.
I
see two ways to go from there:
(1)
a "ThreadAbort" of a thread T1 involved in "and-join", will cause in turn
abortion of the containing thread. So the above is still an abort case for
the Main thread. (even if the above is
"or-join", there ahs to be at least one argument thread - here only T1 -
that completes for the join to succeed.) So if we keep this logic for
"join", then we abort Main.
(2)
if for some reason you want the thread to jump directly to "successful
completion" if TestAssertion = True/False, then we need a 4th action
besides "continue", "abort", "exit[testcase]": we could call it
"endthread". In that case that is just one more way to terminate
positively a thread (so join semantics is same as for normal
completion)
- no
"pass" outcome unless an explicit pass exit statement is met during
exec.
[MIKE] - Agreed. That is the
current assumtion. "exit/pass" MUST be explicitly set in the
<TestAssertion>
[Jacques
Durand] OK, I subscribe to that (my latest mail might have
contradicted myself...)
- a
failed assertion without explicit exit statement, by default will "abort"
the thread, but just the thread.
[MIKE] - I suggest that <AbortThread> MUST be
explicitly set for a <TestAssertion> that returns a value of
"false"
- a passed assertion without explicit exit
statement, by default will "continue" the thread.
[MIKE] - Agreed. This is the current logic
assumption
- when threads are joined, an aborted thread will
automatically cause failure of an and-join (which aborts the container
thread) [MIKE2] - My question is, why abort the
container Thread as well? If a Join simply allows/disallows
(gates) workflow continuation based upon the result of the Join,
then it appears that in either case (whether Join condition is satisfied
or not), there is no reason to "abort" the parent Thread. There is
no "error" (that I can see) either way with the Join that would
require an "abort" of the parent Thread. If the and/or
Join result allows continuation of workflow , then the
parent Thread would simply finish its workflow execution, and return
control to IT's parent Thread (ther parent's parent). If the
and/or Join result precludes workflow continuation... fine... The logic
rules have been followed, and again, control is returned to the "parent's
parent Thread". No abort is necessary. The only scenario that I can
see that would require the parent Thread to "abort" is if a
<TestAssertion> operation run IN THAT Parent Thread
implicilty or (as I would prefer) explicitly sets an
<AbortThread> instruction based upon the result of that
<TestAssertion>.
Comments?
[Jacques Durand]
previous comment applies.
. In case it is an or-join, the aborted thread will
just be ignored by the or-join (the or-join will fail if all joined
threads abort). If a thread that was split but never joined, aborts, then
it just stops and is simply ignored for the rest of the test case exec and
outcome.
[MIKE] - Agreed
- if the Main thread of a test case aborts, the
outcome is "Undetermined" by default (this is the only case of implicit
outcome, in addition to other explicit "Undetermined" outcomes)
[MIKE] - There are a couple of issues here:
The only way a "Main" Thread could abort (based on
your suggested logic for concurrent Threads) is if it is concurrently
run (i.e. the Main Thread is <Split>). But if it is the
Main thread, why would you <Split> it? Disregarding
that, if an "abort" means "terminate this Thread but continue
execution" then the Test Case would still execute successfully to
termination, and should "pass" (i.e. an "abort" is saying that
a concurrent Main thread's result is "inconsequential").
[Jacques Durand] an "abort" should also
abort the container thread if the completion of the used thread was
expected (now we can introduce an "exitThread" statement in
addition?)
[ [MIKE] -
Also, use of a "Main" Thread is a RECOMMENDED PRACTICE. There is no
need to use a Main Thread in the Test scripting if one is not
necessary. For example, with the ebMS Test Suite, a Main Thread is
not required for any of the 200+ Test Cases, since all <PutMessage>
,<GetMessage> and <TestAssertion> operations can be
completed as child elements of the <TestCase> itself (
which I believe can be considered a Main Thread). This greatly
simplifies scripting as well.
[Jacques Durand] the Main thread is just implicit here
- but still there logically. Right?
[MIKE2] - I would like to suggest
setting the final state of the Test Case to "undetermined" if a child (of
the <TestCase>) <TestAssertion> encounters an
<AbortThread> instruction during its workflow. Otherwise, all other <Thread>s simply "abort" and pass
control back to their parent Thread. If execution proceeds to
logical conclusion without an exception conditions or explicit exits, then
the Test Case final result is "pass".
Comments?
Some additional Comments starting p.5 of the attached
doc (mostly, use case #2).
Also, I was searching for the section where we specify
the test step timeout (MaxDuration?) , in the draft spec,
and did not find it (same for the "sleep" statement).
Cheers,
jacques
<<section7.1-JD1_MK_JD2.doc>>
To unsubscribe from this mailing list (and be removed from the
roster of the OASIS TC), go to
http://www.oasis-open.org/apps/org/workgroup/ebxml-iic/members/leave_workgroup.php.