OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [ebxml-iic] call tomorrow Wed 8th, 3pm PT

Title: call tomorrow Wed 8th, 3pm PT



  I modified section 7.1 based on comments from last conference call (changed structure/design of logic flow HTML table.  Plus I modified the “pass”, “fail”, “undetermined” description at the end.  Please see attached document (turn on "tracking" to see changes).


It seems that the last sticking point is the “default” behavior of a  Test Driver when encountering a “failed” TestAssertion.  Please see my comments below:



It is annoying that the interpretation of an assertion failure (what to do if fails) , is context-dependent (presence of a step behind or not…) Shouldn’t we say that an assertion failure just aborts the thread, in the absence of any other explicit exit statement?


[MIKE] – Why can’t there be a “default” behavior for a failed TestAssertion.  The fundamental meaning of TestAssertion is just that.. testing an assertion.  Generally, if such an assertion is not met, this means the implementation failed.  However, we have created a more flexible meaning..saying that one can now “continue” from a failed TestAssertion..or conceivably even “pass” after failing a TestAsssertion operation.  I believe that the meaning has been “watered down” to the point where it is now nothing more than a logic flow operator.   I believe that the default behavior should be “fail” for a “failed” TestAssertion.  One can still “override” it using the <Return> instruction ( or <Abort>… although I think that you suggested that we not use <Abort>).  I am in favor of trying to keep close to the meaning of <TestAssertion> as we originally defined it.



 This thread may be or-joined later and the entire test case may still succeed thanks to other concurrent threads that have successfully completed.


[MIKE] – I’m OK with that. I just do not think that <Abort> should be the default Test Driver behavior when a TestAssertion fails.  In our 216 ebMS Test Cases, that would result in 216 “undetermined” Test Case results.


 Can’t we say that an aborted test case wil be by default interpreted as “Undetermined”?

[MIKE] – An “aborted” Test Case should  result in an “undetermined” result.  But I do not believe that  a “false” TestAssertion result should not (by default) <Abort> a Test Case.



Below are the rules added to the end of section 7.1 for your review


A “default” Test Case state of "continue" exists after a TestAssertion operation returns a boolean result of "true" , or a “Return” instruction is encountered by the Test Driver, indicating that the current Thread MUST be terminated, but that logic flow will continue within the “calling” Thread.



   A final Test Case state of "pass" occurs when the Test Driver encounters an explicit "exit/pass" instruction from within a TestAssertion, OR logical testing execution proceeds from beginning to end without:


a) An explicit "exit/fail" or "exit/undetermined"

b) A system "exception" condition

c) A failed Join that precludes further testing execution beyond the Join


   A final Test Case state of "fail" is given to a TestCase where :

a) A TestAssertion boolean operation  returns a result of "false"  (default behavior) (i.e. it is assumed that a TestAssertion is a meaningful measurement of conformance/interoperability that MUST pass.. unless explcitly overriden by the test writer in such cases where a TestAssertion verifies a "precondition" to further testing, in which case the test writer may wish to "exit" with a state of "undetermined".  Additionally, the test writer may with to "continue" if the failed result of the TestAssertion is used to alter the flow of Test Case execution.

b) The Test Driver encounters an   explicit "exit/fail" instruction within a TestAssertion operation.


   A final Test Case state of "undetermined" occurs if:


1) An explicit "exit/undetermined" instruction occurs from a TestAssertion operation

2) Testing logic flow cannot proceed beyond a Join due to failed andJoin or orJoin condition





----- Original Message -----
Sent: Tuesday, September 07, 2004 5:27 PM
Subject: [ebxml-iic] call tomorrow Wed 8th, 3pm PT

We'll have exceptionally a meeting tomorrow Wednesday at 3pm PT.

Primary objective is to close on Test Framework spec remaining points being discussed.
I'll mostly give a quick update on #2, #3 in agenda below.
We'll likely postpone #4 to Monday, unless Patrick is on the call.


Time: Wed 8th, 3pm-4pm PT
Host: Fujitsu
Toll only :  1-512-225-3050
Participant code: 385218#

tentative Agenda :

1. Test Framework 1.1 technical:
- use cases , BPSS relevance.
- Threads: abort semantics, return feature.
- Monica comments.
- levels of conformance, if any?
- any "binding" to consider? (optional, but normative)

2. ETSI follow-up:
- CD status for the test plan
- feedback on test suites: few bugs detected, updates needed.

3. Request from ebXML Asia for a n Interop test suite without:
- sync messaging (bus responses only?)
- empty messages.

4. Others:
- follow-up/update on Patrick Hogan / Linux plans.


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]