[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Subject: Re: [ebxml-iic-conform] comments on latest issues
To all, Attached are my comments to your suggestions/questions/comments regarding the ebXML MS Conformance Test Requirements and Annotated MS Specification documents. I've incorporated those suggestions into changes in both of those documents, and have now posted them for a vote this week. Attached are "my comments to your comments", reflecting those changes. Thanks for all of the great input, Mike PS- Changes can continue to be made this week as we move toward resolution in voting by Friday.
Monica, I went through Eric's comments, and changed/modified accordingly. I came across some more potential "spec/requiremnet gap" items that can be added to the list to submit to the MS TC. They are: urn:semreq:id:10 - Spec states: If provided, the MIME charset attribute MUST NOT contain a value conflicting with the encoding used when creating the SOAP Message.. yet says both must have same value. So UTF-8 MIME charset and ASCII SOAP encoding is illegal? urn:semreq:id:145 - Very vague specification section... "The ultimate destination of the error message is informed of the failure by some undefined means." urn:semreq:id:154 - Spec says SequenceNumber status attribute SHALL be there, but XML Schema says that it does not have to be present. Is this a schema design error? At 02:35 PM 8/1/2002 -0700, Monica Martin wrote: ----------------------------------------------- urn:semreq:id:37 What determines if a conversationID is out of context ? The CPPA 2.0 lists concurrentConversations and invocationLimit is this what the test requirement aludes to ? Also what is the error to generate ? (The spec doesn't say the error code to use ?) [MIKE] - See comments below. urn:semreq:id:155 It implies there should be an error if this is not the case, but the spec doesn't say it. It would seem the receiving MSH would throw back an error - like illegal reset or hold the communication open until it's clear to reset. I think the test requirement covers the specification as written- but I think the specification missed something here. --------------------------------------------- [MIKE] - See my comments below. Michael, Need to be added to ebMS TC gap questions...do you want me to do so and resend? Thank you. [MIKE] - Outside of the two requirements above, what I see below however are fixable typos, naming, grammar, sequencing problems. I will fix them this weekend, and re-submit to the OASIS webmaster on Monday. -----Original Message----- From: Eric VanLydegraf Sent: Mon 7/29/2002 10:49 PM To: 'Michael Kass'; ebxml-iic@lists.oasis-open.org; ebxml-iic-interop@lists.oasis-open.org; ebxml-iic-conform@lists.oasis-open.org Cc: Subject: [ebxml-iic-conform] TestReqs Comments Well as many times as I have read the ebMS document, it seems you truly never know it. I have to commend the authors on the annotated ebMS spec for the thorough highlighting and the test requirements generated from it. Translating specification speak to actual test requirements is not always obvious. Upon review here's some comments, questions and corrections I've found. Page 3 (Test Requirements Doc) Table Fields ... "Name: A unique name for the Test Requirement." these ids have identical names 73,74 158,159,160 163,164,165 [MIKE] - Done ================================== Refering to urn:semreq:id:10 Assertion They shall have the same value annotated_ebms_v2_0.doc 576-578 If provided, the MIME charset attribute MUST NOT contain a value conflicting with the encoding used when creating the SOAP Message My understanding of the specification is that you can list different value encodings, as long as the encodings are compatible (I don't think they're are many such possibilities, but the spec authors didn't disallow it) If an implementation were to have different encoding values but one is a compatible subset of the other (e.g. UTF-8 and ASCII). How would this test requirement play out in the testing driver. According to the assertion they would fail, but they would be in compliance with the spec ? [MIKE] - This statement in the spec seems to contradict requirement #10, which states that the MIME charset value shall be equivalent to the SOAP encoding value. So it would seem that a UTF-8/ASCII combination would be illegal.... even though this spec assertion seems to indicate that such a combination would be OK, as long as they don't "conflict". This requirement should be added to the "spec/requirement gap list". Refering to urn:semreq:id:18 Has the same situation Would there be any value in just making one encoding test requirement, covering MIME Header and XML Prolog ? It seems these 2 test requirements are covering the same situation. Just spelled out differently. [MIKE] - I read #10 and #18 the same way. I think that we only need one here. I am going to delete #18. ----------------------------------------------- urn:semreq:id:37 What determines if a conversationID is out of context ? The only way to call a conversationID "out of context" would be if it is has no context relevant to any CPAId. The CPPA 2.0 lists concurrentConversations and invocationLimit is this what the test requirement aludes to ? Also what is the error to generate ? (The spec doesn't say the error code to use ?) [MIKE] - This is another spec ambiguity that needs to be better defined. Also, there is no specified error to be generated if a ConversationID is determined to be out of context. I think that this requirement should be removed from the list, since it has no real reference in the specification. ------------------------------------------------ Would it provide value to list test requirement dependencies ? e.g. urn:semreq:id:79 ^--- urn:semreq:id:77 ^--- urn:semreq:id:78 They struck me as building up from each other. So you could say urn:semreq:id:78 is Ack was not received plus (urn:semreq:id:77 testrequirement) ... It would also give you an escalting test scenario in the test driver itself or reuse of test artifacts. [MIKE] - Monica brought up this question in another post. I believe that this is something we can discuss at the F2F... enhancing the test harness to incorporate dependencies between test cases. ----------------------------------------------- urn:semreq:id:104 May want to spell it that Reference is "XMLDsig Reference" element and not "ebXML Manifest Reference" element. urn:semreq:id:105 is more clear concerning the reference - another possibility is to switch 104 & 105 so the XML Signature is listed first. [MIKE] - I followed your first suggestion, and added the "XMLDsig" clarification. --------------------------------------------- urn:semreq:id:110 Suggestion: The DuplicateElimination element [must|is] not [be|] present. [MIKE] - Done --------------------------------------------- urn:semreq:id:130 PreCondition: [Suggestion for clarity - the phrasing was little hard to follow] (For each reliably received message, having a SOAP actor URI targeting the receiving MSH and the AckRequested element is present) Assertion: [Correction] fvalue => value MessageIde => MessageId [Suggestion] An Acknowledgement Message is generated with RefToMessageId having the MessageId value of the message being acknowledged. [MIKE] - Done -------------------------------------------------------- urn:semreq:id:137 & 138 (dangling quote) I think you meant to enclose "original message" 138: "Acknowledgement Message" or just remove the end quote [MIKE] - Done -------------------------------------------------------- urn:semreq:id:141 PreCondition: [... AND the syncReplyMode is [not] set to none ...] [MIKE] - Done ----------------------------------------------------- urn:semreq:id:142 PreCondition: {this must have been a tough one to decipher - back to boolean algebra :&) [... AND [if] the syncReplyMode is [not] set to none and the CPA indicates that an application response is NOT to be included] [MIKE] - Done ----------------------------------------------------- urn:semreq:id:145 remove this one ? I wonder if we should include this one - while the spec says it - it has no meaning for testing purposes. I mean hell I could fedex you a error log and that would satisfy the specification requirement. Doesn't really offer any value - this is more like a documentation item for say implementation guidelines. [MIKE] - We will leave it in, but annotate the specification with a coverage of "none", due to spec ambiguity. This will be added to the list of spec/requirement gaps. ------------------------------------------------------ urn:semreq:id:150 dangling quote ------------------------ urn:semreq:id:150-151 should be combined (they are both required together) [MIKE] - I agree that the spec indicates that the "status" attribute is required for requirements #150 and #151. They are only "split" for now because it is not clear whether the test harness will support aggregate assertions. If so, then 150 and 151 will be merged. urn:semreq:id:152 (is there a need to differentiate first message versus reset as far as the assertion (status="reset" and sequenceNumber="0" ) they are the same test. seems like all 3 could be just one testrequirement - I don't see any reason to make the semantic difference highlighted when the test is identical. [MIKE] - #151 tests the case of a "first message" in a conversation, while #151 tests the scenario where sequence number is reset in a conversation. The spec sees these as semantically different cases, and so we wrote them that way. ---------------------------------------------------------------------- urn:semreq:id:153 Remove this requirement I understand ebMS2#9.1.1 that status="reset" is with the sequenceNumber="0" only in cases 1 & 2 and not the numbers following it. And thus is the same as 152. [MIKE] - You are right, that the status is not "Reset" in this case, but should have a value of "Continue", along with a sequenceNumber of "0" according to the spec. So I will modify this requirement to reflect that. ------------------------------------------ urn:semreq:id:154 Assertion: [... AND attribute status="Continue" (if attribute status is present?)] [MIKE] - This is a case where the ebXML schema says one thing, while the semantics of the specification says another. As you mentioned in the case of #150 and #151, the spec requires both the sequenceNumber and status attribute to be present... and uses the word SHALL when referring to the status value, but the schema says that the status attribute is optional. When I see it explicitly spelled out in the semantic text of the spec, I give the specification wording precedence over a schema document, and bring it to the attention of the MS TC that their schema does not enforce the wording of the specification. This is another item for the "spec/requirement gap" list. ----------------------------------------- urn:semreq:id:155 It implies there should be an error if this is not the case, but the spec doesn't say it. It would seem the receiving MSH would throw back an error - like illegal reset or hold the communication open until it's clear to reset. I think the test requirement covers the specification as written- but I think the specification missed something here. [MIKE] - This requirement examines the state of the MSH prior to sending a Message with a SequenceNumber attribute of "Reset". I do not see where there is an implied "Error" message to be sent if all previously sent messages have not been accounted for. The specification does not indicate any such behavior. --------------------------------------------- urn:semreq:id:186,187,188,189,190,192,193,194,195 [PreCondition] an party => a party 188,189,190, 192,193,194,195,198 (redundant "the the") 191 messaged => message [MIKE] - Done --------------------------- Couldn't urn:semreq:id:196 be a generic test requirement that all optional features that are unsupported must prove this assertion ? [MIKE] - We can write one if the specification states that all optional features that are unsupported will return an Error element with an errorCode of "NotSupported" and a highestSeverity attribute set to "Error". I could not find that in the specification. ----------------- urn:semreq:id:206 GenerateAckBasedBasedOnActor Should Based be repeated ? [MIKE] - Thanks. Fixed. ------------------------------ That's all I found on this pass - phew - lots of stuff here ;) ---------------------------------------------------------------- To subscribe or unsubscribe from this elist use the subscription manager: < http://lists.oasis-open.org/ob/adm.pl>
Attachment:
CT_issues_Monica.doc
Description: MS-Word document
Reqts Level 2 urn:semreq:id:2 - Separate two conditions (1) MIME/multipart and text/xml [MIKE] - Agreed. Done urn:semreq:id:15 - Separate assertion - may need to be two requirements tested individually to determine the unrecognized MIME headers are ignored and that no error message is sent. [MIKE] - Agreed. Will split this into 2 if no assertion aggregation is permitted. [MIKE] - Agreed. Done General: With the sequential numbering of conditions you are not able to see - other than in the XML which conditions go with which semantic requirement. [MIKE] - Agreed, will ask Matt if he feels this is significant issue for test harness. I agree it is textually confusing not being able to associate conditions with a parent requirement id. Then again, conditions can be "re-used" via reference in other requirements ( some are, although not currently done by reference to id, they are just "repeated"). urn:semreq:id:41 - Couldn't this be tested by a profile where you actually link a series of discrete test cases? [MIKE] - Yes, I think that you could do it this way. I hope we can explore these options when we start building the test harness. This could add more power and flexibility to testing. urn:semreq:id:47 - misspelled word in condition 54 - ff not if. [MIKE] - Done urn:semreq:id:43 - misspelled word in condition 49 - and not an. [MIKE] - Done urn:semreq:id:64 - erroneous character in condition 71 and 72 - · [MIKE] - Done urn:semreq:id:13 - erroneous character in condition 15 - manifest᾿ [MIKE] - Done urn:semreq:id:65, 69 - Can't we at least partially test by logging the error? Regardless of the other actions - resolve by other means, etc? [MIKE] - I think that this will be determined by how much logging capability will be defined in the testing service. To be determined by Jacques. urn:semreq:id:75 - misspelled word in assertion - behaveas not behaves. [MIKE] - Done urn:semreq:id:77, 79, 82 - Shouldn't this assertion be broken into two parts? (1) The message is kept in persistent storage and (2) processsed as if the interruption had not occured. [MIKE] - Done Occurred is misspelled. [MIKE] - Done urn:semreq:id:91 - How are we verifying it is consistent with the CPA in the conditions? This would apply on any like requirements regardless of 1,2 or 3, and conditions. [MIKE] - As I understand it, the CPA ( or CPA-like document ) will be available for comparison ( XPath evaluation ) to check consistency of a message with CPA configuration urn:semreq:id:94 - Shouldn't the assertion be broken into discrete test cases? (1) The MSH generates an Error of type Inconsistent and (2) severity = Warning [MIKE] - Yes, I think that it should. I Will split this. Will split if no assertion aggregation is permitted [MIKE] - Agreed. Done urn:semreq:id:105 - Same (1) Any Reference elements included in a generated Acknowledgment element are namespace qualified to the XML Signature namespace and (2) conform to the XML Signature specification. [MIKE] - Agreed. Will split it into 2 requirements if no assertion aggregation is permitted. urn:semreq:id:123 - condition 154 - How are we verifying what happens in application? Reference "the message is presented once and only once to the application" [MIKE] - I will have to defer to Jacques and Matt regarding what the test harness will and will not be able to do regarding logging such events. urn:semreq:id:130 - misspelled word in assertion "fvalue as the MessageIde" [MIKE] - Done urn:semreq:id:152 - Shouldn't the assertion be broken into discrete test cases? (1) The SequenceNumber element has value of 0 and (2) the status attribute of the message is set to Reset. [MIKE] - Agreed. Will split into 2 requirements if no assertion aggregation is permitted. urn:semreq:id:170 - Shouldn't the assertion be broken into discrete test cases? (1) The generated XML Signature Reference element includes a child Transform element and (2) which in turn includes a first Transform element with an Algorithm attribute of value "http://www.w3.org/2000/09/xmldsig#enveloped-signature". [MIKE] - Agreed. Will split into 2 requirements if no assertion aggregation is permitted. urn:semreq:id:187 - Shouldn't the assertion be broken into discrete test cases? (1) In the returned StatuResponse element, the RefToMessageId element child of the MessageData element specifies the MessageId of the message containing the associated StatusRequest element. and (2) In addition the RefToMessageId element child of the StatusResponse elements always contains the MessageId of the message whose status is being queried. [MIKE] - Agreed. Will split into 2 requirements if no assertion aggregation is permitted. urn:semreq:id:190 - Did we resolve if we understand what unauthorised means? Reference in condition 258 - "the message is received from an party deemed to be unauthorised" I ask because we had a query back to TC and we designate this requirement as full. [MIKE] - This does sound very hard to test "fully", or even "partially", since it seems very application dependent. I would recommend changing this requirement to "none", unless Jacques can explain how authorization will be determined through the test service.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Powered by eList eXpress LLC