Latest IIC MS Conformance Test Suite doc, and feedback on comments

["Michael Kass" <michael.kass@nist.gov>]

Title: next meeting Monday 25, and some MS conf review feedback

Jacques,

 

  Below are my comments re: suggestions and fixing errors. Also, attached is the latest ebMS Conformance Test Suite specification,

version 0.91, with all changes based upon your, Pete's and Monica's comments.  Actually, for Monica's comments ( regarding abstract representation

of "untestable" requirements) , we will need to discuss if all tests deemed untestable because of test framework constraints should abstractly be

described anyway. I think that we can do that, I just have not gotten around to doing them yet. 

 

I've also attached the latest ebXML MS 2.0 Conformance Test Suite Specification, version 0.91 The executable Test Suite XML file is also updated

to reflect all of the changes.  I have not included that one yet, as there are still a lot of XPath fixes to be corrected, but it follows the Abstract Test Suite

defined in the spec.

 

 

  Here is my response to Jacques suggestions:

 

 

Regards,

Mike

 

 

General Notes:

---------------------

- For requirement level column, we have sometimes "REQ", or "REC".

-As an editorial note: all message actions ("received" "generated"...) are relative to teh candidate MSH ("...by MSH")

 

Detailed:

----------

funreq_id_70:the assertion should be more explicit: "a message generated in response

is sent on the same connection, e.g. an HTTP response, depending on the SyncReplyMode value".

the message received was

[MIKE] - Fixed

 

funreq_id_71 and _72: it is a REQ (not OPT)

[MIKE] - Fixed

 

funreq_id_77 to _79:

no Ack was received ---> and no Ack was generated (by the MSH)

[MIKE ] - Fixed and reduced to 2 requirements (send) and (receive)

funreq_id_77 and _80: I guess one of them is redundant: "sent" means "received"?

If it means "generated" (by the MSH), then we need to be more precised about

"processed as if the interruption had not occurred."

That measn for example, that after Recovery the MSH will resend the message the correct number of times if it does not receive an Ack.

funreq_id_78 and _81: same as for _77 _80.

funreq_id_80 and _82: same as for _77 _80.

[MIKE] - I agreee there is a lot of redundancy in #78 - #84. I believe that all can be

specified in 2 requirements.

If no Ack is generated and recovery is within TimeToLive window and there is a system interrupt, the message is resent by candidate MSH ( if it is the sender).

If recovery is within TimeToLive window and there is a system interrupt, the message is PROCESSED by candidate MSH ( if it is the receiver).

[MIKE] - AGreed and done. I reduced the 7 tests to 2 (send) and (receive). All the other requirements were really redundant.

funreq_id_97: "For each *generated* Ack...":

[MIKE] - Done

funreq_id_106: assertion: replace "The From Party MSH " by "the MSH" (this is the candidate, there is no other)

[MIKE] - Done

funreq_id_114 _115: replace sent by "received"

[MIKE] - Done

funreq_id_130: seems teatable to me ! we just analyze the Ack generated.

[MIKE] - This was an error in the spec. I fixed it.

funreq_id_131: I think it is "for each generated Ack", in the precond.

[MIKE] - Done

funreq_id_137: seems testable: (replace "Sending MSH" by [candidate] MSH): test driver will observe the resending from MSH.

[MIKE] - This was an error in the spec. I fixed it.

funreq_id_138: spec ref is 6.5.4 (not 6.5.3) should be testable if the ErrorAppNotify is properly interfaced with candidate MSH, in Test Service.

So we could say this in the test coverage.

[MIKE] - That is the problem. What is the "proper interface"? It is not defined in the

Test Framework, in particular, what errors should "ErrorAppNotify" catch? I'll write

this up as an "Abstract Test" for now.

funreq_id_139: replace "sent" by "generated". (Check for replacing "sent" appropriately for all test reqs...)

[MIKE] - Done

funreq_id_141: (and _142) I believe at least, we can check that the *same* Acks are sent back, independently

on storage situation:

((For each message received, which is a duplicate of a reliable message previously received and

the syncReplyMode is not set to none and

The CPA indicates that an application response is included) then

The generated Ack is returned on same connection, and is the same as the initially generated Ack.

[MIKE] - Does this get into the "signalsAndResponse" or "signalsOnly" case of syncReplyMode? Is an Acknowledgment a signal? If so, then the above test requirement would only hold for "signalsAndResponse" correct? I'll be happy to

reword 141 and 142 if we can resolve this issue. Perhpas testing the above case

with these two possibilities is the answer.

--- stopped review at _141 -----------

 

 

 

IIC_MS_Conformance_TestSuite_V0.91.doc