[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Section 7.1 of Test Framework revision
I sent this yesterday under a different "subject" heading (IIC Conf Call Tomorrow)..so some may have missed it. The bottom line with this modified section 7.1(see attached) is that the
default behavior of <TestAssertion> has a dramatic effect upon how we must implement our test execution model. If the default behavior of
the Test Driver (when it encounters a <TestAssertion>="false") result is to "exit/fail" the Test Case, this makes it very suitable for simple
types of testing (ebMS, ebRegistry) where <Thread>s are not used and are not necessary.
I call this OPTION A:
It may not be as suitable for BPSS testing.
OPTION B: (which you are suggesting)
Changing the default behavior of the Test Driver to "abort" the current Thread when one of its <TestAssertion>s fails means that the subsequent <Join> operation reference that <Thread>
MUST also "abort" ITS parent <Thread> if it is not satisfied ( since we are in essence saying that a failed <TestAssertion> "failed/aborted" its container <Thread>.. and that the subsequent <Join> may not be possible
..depending upon whether it is an <AndJoin> or <OrJoin>). "Aborting" based upon the <Join> result makes sense, and would allow for the logic of OPTION A to "bubble up" through the
<Thread> hierarchy, ultimately achieving the same result.
For example, a <Thread> that listens for any received Error messages and finds one "aborts" if its <TestAssertion> (which verifies any received Error messages) fails.
Therefore the subsequent <AndJoin> fails. Expanding upon that, the <AndJoin> could "abort" its container <Thread> .. and continue (aborting) up the execution tree until either no
<Join> is possible (exit/fail) or some possible continuation of execution flow is possible (based upon a satisfied <OrJoin>).
This is very similar to my initial proposal for a "cascading boolean chain" of <Thread> results, where the boolean result of a lower <Thread> would "bubble" up the <Thread> hierarchy
and execution would either continue or fail based upon logic flow. Here we are using <Join> as the determining operation for continued execution.
If we choose this approach, then section 7.1 MUST be re-written to show this change in logic. My latest revision does not reflect this approach.
Additional rules would be:
If a <TestAssertion> fails in the "main" <Thread> ( which is <TestCase> ), then final <TestCase> result is "fail".. because the implicit <Join> of the main <Thread> is not possible.. since
that <Thread> is "aborted".
Failure of a <TestAssertion> in a serial <Thread> (i.e. a <Thread> declaration with an implied <Split><Join>) follows the same logic. The "serial" <Thread> is aborted, and therefore its parent
<Thread> is also aborted ( since the implicit <Join> will fail).
This means that <Abort> MUST exist as an implicit (and therefore, I would guess "explicit" as well) operation for the Test Framework. You had suggested removing it and only keeping <Return>.. which is the
symmetric opposite of <Abort>. We can't do any of this without <Abort>.
My only concern with option B is based upon traceability of conformance/interoperability failure. We MUST be able to trace exactly
where the testing failure occured. By eliminating "exit/fail" as the default behavior for a failed <TestAssertion> , we MUST be able to trace back to that failed <TestAssertion> which triggered the <Abort>.
In the case of an <OrJoin>.. an <Abort> can actually happen in multiple <Thread>s. So there are multiple points of test failure.