[ebxml-iic-msg] MS Test req comments

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: MS Test req comments

Mike, Matt:

Here is my review of the last sending from Mike, on test requirements for Level 1.
Looks like we are close to get the first batch out.

We will need someone doing a coverage review: making sure we cover all
MS Core features in the spec. A simple approach, is to scan for all "MUST" or "SHALL"
keyword, and see if we address them...

Cheers,

Jacques
<<Req_L1_comments.txt>>

NOTE1: it is useful to specify, for each test item, the context of its verification:
Should it be verified: 
(1) for messages received by the candidate MSH? 
(2) for messages generated  by the candidate MSH?
(3) on the callback from application interface (e.g. some errors)?
So I'll add this info for each test item.

NOTE2: in the test req description (Assertion), we should remind the context of
the description. Often, this is the title of the subsection of the spec that 
is referred  to. That avoids ambiguities. For example, in r1.2.10, Assertion
 should state that these conditions concern the Service element 
(even though the test name clarifies that). 
(similar conditions may be needed in a different message 
element, requiring a different test.)

NOTE3: a few of these were not always reported as such in the level
of req, from the spec: SHOULD --> RECOMMENDED, MAY --> OPTIONAL.

r1.1.2:(generated M)
can we detail a little more what that means?
r1.1.5: (received M)
r1.1.7:  (received M)
The "charset = UTF-x" MIME parameter can still - and should - be verified?
r.1.1.11:(received M)
Partial check can be done? (by falsification of a message: we put all possible
other MIME headers in, and just see that the message is well received.)
r1.1.25: (received M)
The rejection must be in "accordance with SOAP", I believe there should be a SOAP Error,
that should then translate into an MSH error. So by falsification, we should be
able to partially check this (i.e. just for one instance of a non-recognizable header.)
r1.2.1: (generated M)
Actual test is that the "From" and the "To" are present. (identifying the 
party can't be verified)
r1.2.3: (generated M)
As worded, I believe we have no way to verify that... however other validation can be done:
(uniqueness of type attr value, position of "role" element if any)
r1.2.5:  (received M)
Note: this is the kind of test that can be "doubled", i.e. verified for both received
and generated messages. In that case, we should make it two test req items:
For rceived M: stated as you said (we SHOULD observe an error message)
For generated M: 
(generated M) either the PartyId type attr is present, or if it is not, its value MUST be URI.
r1.2.7:  (received M)
Is the spec ref wrong (3.2.1)? it seems to refer to XLinks in my version.
Anyway the scope of that test seems too broad - too high level - normally should be split in several
test items. Can we ignore it and rely on more specific discrepancy checks as they 
show up later in the spec? 
r1.2.9: (generated M)
Only partial verification possible: this requirement actually involves the application 
behavior: this is a usage guideline for the app... if the app does not set the conv id right, 
no way can check that: could be another conversation. We may just verify that the conv id
set by our test service is well reported in the msg?
We should also add a falsification test (generated M, here) for uniqueness of convID 
within a CPAid, where the app (our test service) on top of candidate MSH try to send two messages 
with different convID but same CPAid. Normally, the candidate MSH should complain - report
error to the app.
r1.2.10: (generated M)
Remind in test description that this is about the Service element.
And also, this is a "double" test item:
(generated M): "condition C must be satisfied."
(received M): "if condition C is not satisfied, an error should/must be generated."
r1.2.11:  (received M)
probably partial verification: we can only verify for some particular case of bad message
we generate (falsification).
r1.2.14: (generated M)
Only partial verification possible: it depends on the application to set the RefToMsgId properly.
So we can simulate some unauthorized case, (but that is r1.2.15) by initiating a new conversation 
with our test service, that sets RefToMsgId (should not), and see an error. 
(yet, nothing is said about Error generated.
Is there an implicit assumption throughout the spec that failed reqs generate errors?)
r1.2.18:  (received M)
The error msg should be returned to the "From" party.
Other test items should be derived from (3.1.6.4) , like TimeToLive if present should have 
valid UTC value (and not local time...)
r1.2.19:  (received M)
Even though this req is in "MS Core features", it really relates to Reliable Messaging
(MS additional features). We should handle this in Level 2.
r1.2.20:  (generated M)
(An example of the way r1.2.7 should be split into sub-items.)
Yet another "double" test, if in case we send a "bad" message: should there be an error?
r1.2.24:  (received M)
Yet another double test: we should also add the (generated M) version of this test req item:
For any generated messages from the MSH: "either the xlink:href does NOT contain a cid URI, or 
if it does, a MIME part with contetn-id must be present in payload container."
r1.4.5:  (generated M)
I believe we can test that if no error are reported in a message sent by MSH, then 
the ErrorList elt must not be present.
r1.4.6:    (generated M)
Only partial check, as it may be hard to observe this from candidate MSH: we would need to 
make it generate such a message with several error elements.
r1.4.9:  (generated M)
reword as: the Error code must be valid. (we can't check more...)
r1.4.12:  (generated M)
I believe we can at least check the XPointer well-formedness, and in case of payload-related
error, this is not app-specific (all these errors are MSH level): 
we can send a message with a bad payload *container*.
so that we can check if the error generated is OK.
r1.4.18:  (generated M)
I believe its very hard to test that, as it is the case where an error message is bundled
with a business message response. Some MSH may simply not support this feature.
We may not have any control on that from our test service. So I would say no coverage.
r1.5.1:  (generated M)
We can test if we got the business response (generated by our test service)
as an HTTP response to the previous sending... so I would say coverage is OK 
(req level: RECOMMENDED, as SHOULD is the same).
Note: spec 4.3.1 also report yet another case of CPA discrepancy that is not allowed:
SyncReplyMode. Should be in a separate test req item.