[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [ebxml-iic] Conformance Test Case Coverage - Issues to be Resolved
Here are my comments on the remaining issues. In some cases, I've posed deeper questions that may need to be discussed, and things that need to be brought to the attention of the Messaging TC. --Pete Pete Wenzel <pete@seebeyond.com> SeeBeyond Standards & Product Strategy +1-626-471-6311 (US-Pacific) Michael Kass wrote: > To all, > > Attached is the Test Case Coverage document for those conformance > test requirements where > the specification, the nature of the test ( application vs. MSH testing > ) or the Test Framework itself > is an issue regarding testability. > I integrated comments from the last conference call, but I believe > that we still need a further iteration > on these, as we did not get to cover all of the requirements at issue. > > Those covered in the last call include: > > 86 > 86 > 87 > 88 > 89 > 148 > 155 > 156 > 161 > 175 > 159 > 193 > 194
funreq_id_9: Future work: extend test interface, giving it access to full MIME message, including headers. funreq_id_27: Add to assertion: Any @id values that are present must be unique. (Can such an assertion be expressed in XPath/XSLT?) The full list of extension elements is: Acknowledgment, AckRequested, Error, ErrorList, Manifest, MessageHeader, MessageOrder, StatusRequest, StatusResponse, SyncReply. (Pong is not an element, it is a Service.) Obviously not all of these elements can be exercised in a single test case, so do we revisit the assertion in future test cases to get full coverage? funreq_id_29: Presence and correct namespace qualification of this attribute is asserted by the echema and tested by virtue of schema validation. Required for these elements: Acknowledgment, AckRequested, ErrorList, MessageHeader, MessageOrder, SyncReply. (Side note, perhaps for ebMS TC: value of "1" is required in each case; why is it not a fixed value in the schema?) funreq_id_31: Not only is this application-level testing, but whether or not uniqueness of the type value is required depends on the nature of the party identification system chosen. funreq_id_47: Remove "or Manifest" from Name and Assertion (it is not possible to place application data inside Manifest anyway). Manifest, StatusRequest, StatusResponse are the allowed SOAP Body extension elements. funreq_id_48: (not on this list, but I noticed it when looking into the next one). The applies only if href URI scheme is "cid". funreq_id_50: Not application-level. Test by verifying that each Manifest/Reference CID matches the Content-ID: of an attached MIME part. funreq_id_56: Will test driver report an error if unexpected ErrorList element is present in any received message? funreq_id_61: Value of XPointer is predictable; assert that its value matches the path of the generated "bad" element of the original, referenced message. Should precondition be "for each received message"? funreq_id_62: If test driver is capable of generating a "bad" MIME part, the cid value is predictable; assert that its value matches the Content-ID of the generated MIME part in error in the original, referenced message. Should precondition be "for each received message"? funreq_id_63: Raise this issue with ebMS TC? I think there's no problem here, just that the spec contains a rather odd statement, since Error element has no content anyway (it is a complex type). The spec does say Description content is vendor-defined. So we just don't know why the prohibition against using "Short Description" is there. funreq_id_77-84: I suspect that many customers will demand assurance of data reliability in the face of certain types of system failure. Glad to see that the abstract test cases will be described, so they can at least be performed manually if desired. funreq_id_85: Is message persistence proven implicitly by reqs 77-84 above? funreq_id_89: Can abstract test case be described? Something like, send request to MSH, ignore response, interrupt/restart MSH, expect MSH to resend response. funreq_id_115: In addition, this is app-level testing. (We can't tell whether a duplicate is ever delivered to the application.) Does this mean that we could/should provide abstract test cases for all other reqs we determine are app-level? funreq_id_146: Issue for ebMS TC: Error for invalid presence of SyncReply is covered in section 9.2 and tested by case funreq_id_157, but what is to be done if DuplicateElimination is absent? funreq_id_154: Not application-level, but impractical to test (we would have to instruct MSH to send 100,000,001 messages to observe the wraparound condition). funreq_id_177, 178, 180, 181: Future work: Define a way for framework to query transport protocol security parameters that were used. funreq_id_197: Not application-level, but may be framework limitation. Future: extend framework to provide a way of instructing MSH to generate a Ping.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]