[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Subject: [ebxml-iic-msg] RE: [ebxml-iic] "Final" resend of level 1 requirements
Mike:
Thanks for quick iteration.
Here is another review on it, and comments on your comments...
I believe we make progress in getting to a Test Req format that is
more specific than the spec in terms of what the test content will actually be,
and yet is abstract enough so its does not depend on a particular test
procedure/framework.
Cheers,
jacques
-----Original Message-----
From: Michael Kass [mailto:michael.kass@nist.gov]
Sent: Wednesday, June 19, 2002 11:37 AM
To: ebxml-iic@lists.oasis-open.org
Cc: michael.kass@nist.gov
Subject: [ebxml-iic] "Final" resend of level 1 requirements
Finally figured out mail problems. I apologize for
the bad attachments. Here is the level 1 requirements.
Mike
My comments tagged: [Jacques]
---------------------------
NOTE1: it is useful to specify, for each test item, the context of its verification:
Should it be verified:
(1) for messages received by the candidate MSH?
(2) for messages generated by the candidate MSH?
(3) on the callback from application interface (e.g. some errors)?
So I'll add this info for each test item.
[Jacques]: one way to do that, is to use the "Condition" part of the tset req,
which states what are the pre-conditions required for applying this test
(more on that below)
So for example, we could now have in a COndition part that was empty before:
"for each generated message" , in case the test applies to generated messages.
Or, we could have "for each received message", in case it applies to received
messages.
And if Condition used to contain a cond "C1", then well have:
"for each received message that satisfies C1", etc.
What do you think?
....
r1.1.2:(generated M)
can we detail a little more what that means?
[MIKE] - I believe that the following excerpt from the SOAP With Attachmentts
specification provides the requirements for validating SOAP MIME headers:
The primary SOAP 1.1 message must be carried in the root body part of the
Multipart/Related structure. The type parameter of the Multipart/Related
media header will always equal text/xml.
Referenced MIME parts must contain either a Content-ID MIME header structured
in accordance with RFC 2045, or a Content-Location MIME header structured
in accordance with RFC 2557.
It is strongly recommended that the root part contain a Content-ID MIME
header structured in accordance with RFC 2045, and that in addition to
the required parameters for the Multipart/Related media type, the start
parameter (optional in RFC 2387) always be present.
I added 4 conditions against the above.
[Jacques]: That is definitely more precise than before.
a bit of editorial job still needed I think: i am still uncomfortable with the
way we use condition/assertion, although we improved a lot:
even though the distinction Condition / Assertion is not much important
from a test execution viewpoint, I believe we could/should use it with the
following meaning:
o it should help readers understand *in which case* the core of the
test requirement applies, i.e. if the COndition is not satisfied,
then we cannot say that the test req has been actually verified:
just that no matching situation occurred.
o In case we cannot drive the test harness so that it generates such a situation
that satisfies the "Condition" part of a test req, that means we cannot guarantee
coverage of this test req (coverage = null). In case we can only generate
a subset of the situations, coverage is "partial".
[Jacques]: so to get back to our test req item:
- in r1.1.2, I think the interpretation should be:
[Condition]:"IF the message is multipart MIME (or in any case is NOT a pure SOAP msge)
[assertion]: THEN it should satisfy SwA requirements"
- So the COndition part (which maybe should be renamed "Pre-condition" to avoid
ambiguity) tells that this test verification only applies to multipart messages
(because straight SOAP messages are allowed when no payload...).
- So the test req item could be:
[precondition]: for each generated message, if is multipart MIME, OR is not
enveloped like a pure SOAP ("text/xml")
[assertion]: the message must then be SwA compliant, which means:
(...all you listed previously in "Condition")
- if all you listed in Condition part is enough to capture SwA, we don't need
to restate what you had before in previous Assertion (as it is just a recap of SwA).
- We might have to revisit the content of [Pre]condition/Assertion
throughout the other test reqs (I'll try for some).
r1,1,5: (generated M)
[Jacques]:
A way to state that in a "testable" way is:
[precondition]: In case the mesage is NOT a multipart MIME
[assertion]: it must have no payload.
(Note 1: only "partial" testing can be done here, as we can't force an MSH to
satisfy the pre-condition: we have no control on it.)
(Note 2: this is also a "double test": according to spec 2.1.2, we should check
for Received M as well:
[precondition]: For each received message, if it is NOT a multipart and is
a well-formed SOAP message without payload,
[assertion]: The MSH should accept it.)
[Jacques]:
r1.1.9:
- The way this test req is stated is not wrong, but too restrictive:
it does not cover cases where there are SOME payload elts that are identified
in Manifest, but SOME are not... (in fact, that is stated in r1.1.10,
but I believe r1.1.9 is actually completely subsumed by r1.1.10 as stated,
i.e. if 10 is satisfied, then 9 always is...)
It needs be rephrased:
[precondition]: for each generated Message M, if M has some payload containers
[assertion]: each container content must be identified in Manifest.
- we should state "for generated Message" (generated M)
I really believe we should always add this "context", either as a new column,
or as part of the "precondition" ("generated Message M has no payload
identified in Manifest.)
- now, that sums-up previous r1.1.9 and r1.1.10. But another point in 2.1.4 is:
"If there is no application payload within the Message Package then a Payload Container MUST NOT be present."
So here, I think we need to check that if there are payload containers, then they are not empty.
r1.1.7: (received M)
The "charset = UTF-x" MIME parameter can still - and should - be verified?
[MIKE] - Designated test coverage as "partial".... will only veritfy that
UTF-X is the value supplied for the MIME content-type
r.1.1.11:(received M)
Partial check can be done? (by falsification of a message: we put all possible
other MIME headers in, and just see that the message is well received.)
[MIKE] - Fair enough. Designated test coverage as "partial", based on the assumption
that we will be putting in every possible MIME header.
[Jacques]:
We should interpret a little more what "ignore" means (spec 2.1.5) from a tset req viewpoint:
I would say it means no error with code "Error" is generated, and behavior is same
as if feature wasn't there.
r1.1.25: (received M)
The rejection must be in "accordance with SOAP", I believe there should be a SOAP Error,
that should then translate into an MSH error. So by falsification, we should be
able to partially check this (i.e. just for one instance of a non-recognizable header.)
[MIKE] - Agreed.
r1.2.1: (generated M)
Actual test is that the "From" and the "To" are present. (identifying the
party can't be verified)
[MIKE] - Agreed. Since we are now only checking if "From" and "To" are present,
I removed this requirement because schema validation ( done for every message
generated by the candidate )
will always verify whether both "From" and "To" are present.
r1.2.3: (generated M)
As worded, I believe we have no way to verify that... however other validation can be done:
(uniqueness of type attr value, position of "role" element if any)
[MIKE] - Agreed. Substituted partyId attribute "uniqueness" requirement
for this requirement.
r1.2.5: (received M)
Note: this is the kind of test that can be "doubled", i.e. verified for both received
and generated messages. In that case, we should make it two test req items:
For rceived M: stated as you said (we SHOULD observe an error message)
For generated M:
(generated M) either the PartyId type attr is present, or if it is not, its value MUST be URI.
[MIKE] - "Doubled" this requirement into a "send" and "receive" scenario.
r1.2.7: (received M)
Is the spec ref wrong (3.2.1)? it seems to refer to XLinks in my version.
Anyway the scope of that test seems too broad - too high level - normally should be split in several
test items. Can we ignore it and rely on more specific discrepancy checks as they
show up later in the spec?
[MIKE] - Agreed. Removed this requirement. CPPA/message discrepency tests will have to
be done on a lower-level, individual basis.
r1.2.9: (generated M)
Only partial verification possible: this requirement actually involves the application
behavior: this is a usage guideline for the app... if the app does not set the conv id right,
no way can check that: could be another conversation. We may just verify that the conv id
set by our test service is well reported in the msg?
We should also add a falsification test (generated M, here) for uniqueness of convID
within a CPAid, where the app (our test service) on top of candidate MSH try to send two messages
with different convID but same CPAid. Normally, the candidate MSH should complain - report
error to the app.
[MIKE] - If tests are performed serially, then it would seem that this
type of a test could be done. You assume that other messages will be
sent simultaneously while testing is taking place. Why not require that
this test suite be run standalone against candidate ebXML MS implementations?
For now, I am following your recommendation and changing the test coverage to "partial"
[Jacques]: no, even for a "standalone" conversation: I mean that convID may change under
the control of an application, from one message to the other,
and we are not authorized to see anything wrong in this... This is an application
behavior. Unless we refer to the CPAId: the uniqueness of convID within
the same CPAId:
[precondition]: for generated messages , if several of them relate to same CPAId
[assertion]: these messages should all have same conversation ID.
So again this depends on tha app side: can't cover that fully in a testbed.
But if we assume an Error must be generated
if that is not the case, then we can cover the following test req:
[precondition]: for generated messages , if two of them relate to same CPAId
but have different conv ID:
[assertion]: the MSH should generate an error.
(and the test could be doubled with a "Received" version.)
r1.2.10: (generated M)
Remind in test description that this is about the Service element.
And also, this is a "double" test item:
(generated M): "condition C must be satisfied."
(received M): "if condition C is not satisfied, an error should/must be generated."
[MIKE] - Done.
r1.2.11: (received M)
probably partial verification: we can only verify for some particular case of bad message
we generate (falsification).
[MIKE] - Done.
r1.2.14: (generated M)
Only partial verification possible: it depends on the application to set the RefToMsgId properly.
So we can simulate some unauthorized case, (but that is r1.2.15) by initiating a new conversation
with our test service, that sets RefToMsgId (should not), and see an error.
(yet, nothing is said about Error generated.
Is there an implicit assumption throughout the spec that failed reqs generate errors?)
[MIKE] - The spec is unclear as to what type of Error message is generated if the
condition is not met ( because the previous message never existed, for example ).
I do not find any "implied" assumption that all failed requests generate an error.
This should be brought to the attention of the MS spec team as an item to be clarified.
For now, I would mark this test coverage as "none", since we cannot write a test for
this requirement.
[Jacques]: I propose that, when the spec states MUST or SHALL, and does not specify
what happens when that is not satisfied, that we interpret this as a RECOMMENDATION
for an Error generation.
This should be the case for r1.2.9.
r1.2.18: (received M)
The error msg should be returned to the "From" party.
Other test items should be derived from (3.1.6.4) , like TimeToLive if present should have
valid UTC value (and not local time...)
[MIKE] - Correction made.. and an additional derived requirement for UTC created
r1.2.19: (received M)
Even though this req is in "MS Core features", it really relates to Reliable Messaging
(MS additional features). We should handle this in Level 2.
[MIKE] - This requirement, along with r1.2.20 were both moved to level 2 requirements
r1.2.20: (generated M)
(An example of the way r1.2.7 should be split into sub-items.)
Yet another "double" test, if in case we send a "bad" message: should there be an error?
[MIKE] - Moved to level 2 ( reliable messaging ), also "doubled" the requirement
r1.2.24: (received M)
Yet another double test: we should also add the (generated M) version of this test req item:
For any generated messages from the MSH: "either the xlink:href does NOT contain a cid URI, or
if it does, a MIME part with content-id must be present in payload container."
[MIKE] - Done
r1.4.5: (generated M)
I believe we can test that if no error are reported in a message sent by MSH, then
the ErrorList elt must not be present.
[MIKE] - Done
r1.4.6: (generated M)
Only partial check, as it may be hard to observe this from candidate MSH: we would need to
make it generate such a message with several error elements.
[MIKE] - Done
r1.4.9: (generated M)
reword as: the Error code must be valid. (we can't check more...)
[MIKE] - This is done in requirement r1.4.13... so this requirement will be removed.
r1.4.12: (generated M)
I believe we can at least check the XPointer well-formedness, and in case of payload-related
error, this is not app-specific (all these errors are MSH level):
we can send a message with a bad payload *container*.
so that we can check if the error generated is OK.
[MIKE] - Done.
r1.4.18: (generated M)
I believe its very hard to test that, as it is the case where an error message is bundled
with a business message response. Some MSH may simply not support this feature.
We may not have any control on that from our test service. So I would say no coverage.
[MIKE] - Done.
r1.5.1: (generated M)
We can test if we got the business response (generated by our test service)
as an HTTP response to the previous sending... so I would say coverage is OK
(req level: RECOMMENDED, as SHOULD is the same).
[MIKE] - Done.
Note: spec 4.3.1 also report yet another case of CPA discrepancy that is not allowed:
SyncReplyMode. Should be in a separate test req item.
[MIKE] - Added this requirement, actually "doubled" it for generated and received
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Powered by eList eXpress LLC