RE: [ebxml-iic-msg] MS Conformance checklist: comments

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: RE: [ebxml-iic-msg] MS Conformance checklist: comments

Matt / Michael:

Quite good material here from XMLG - I assume you guys will use this to complete
the current (XML) "test reqs" document?
Some comments:

1. About some test items:
- the DuplicateElimination tests (section 3.1.7, testing actual duplicates), really
belong to "Additional features" (Reliability module), however the "wellformedness" of
the header element (if present) can still be considered as core Level (schema compliance)...
- a few items in this list are CPA-dependent. We may want to put them aside for now until we
introduce CPA-content modeling in our tests(3.1.2, 3.1.5, 4.1.2, 4.2.4.2...).

2. As we have noticed before, many test items are in the form of IF-THEN rules.
Either the "IF" is controlled :
(a)- by the MSH :
(i.e. is an implementation choice, e.g. 2.2.1 &2.2.2 "XML prolog in SOAP")
In that case, this test item may never apply, as the MSH will never generate messages
that satisfy the "IF".
(b)- or the "IF" is controlled by the application (e.g.
or 3.1.8 "if the Description elt is present...", 2.1.4 "if there is no payload...")
In that case, a well-rounded test suite is expected to produce messages that will
satisfy the "IF" of these test items.
(c)- or the "IF" is controlled by the CPA (e.g. all test items conditional to the presence
of Security elements)
In order to test (b), relevant messages need be generated. That means we need also to define
a sampling of "test messages" that we want to see sent by the tested MSH.
We need then to define a format (XML) for message data to be fed to the MSH.
To be more precise, to be fed to the Test Driver that controls the tested MSH.
These test messages should be expressed in an "envelope-neutral" XML format (i.e. NOT ebXML!)

3. The XML format for message sampling used by test driver in (2) above, could adopt
a JMS style: Part 1 will specify header elements as a list of "property / value" pairs,
Part 2 will specify the payload(s). Roughly, that could be of the form:

<messagedata name="testABC">
<header_properties>
<fromPartyID>123</fromPartyID>
<toPartyID>456</toPartyID>
<Service>PurchaseOrdering</Service>
<Action>Accept</Action>
...
</header_properties>
<payloads>
<payload> .... an XML payload, or a ref to a binary payload...</payload>
</payloads>
</messagedata>

The test driver would have to "translate" this in API calls to the MSH (using
a small adapter specific to the candidate MSH's API).
Defining such samples of messages will also help down the road to check the
"service-conformance", that is, the MSH really sends the message content that
 the application (here the test driver) expects it to send out.

4. We may have a similar approach for defining the CPA under which a test is to be
performed (i.e. we define an XML format that encodes the subset of CPA needed to
"configure" the MSH).

5. As our "test requirements" definition document will also somehow "bind" the
test items, with the test procedures that check them, it may be good to distinguish
the test procedures these test items require. So far I see the test items you
described fall into two test procedures:
(P1)- "drive-and-receive" procedures, where the test driver is fed some message data,
and controls the candidate MSH so that it generates a messages that we capture on
the HTTP destination. Most of the test items you described are tested that way.
A Test case definition for such procedures is made of:
(1) test item requirement def (and associated procedure),
(2) message data to be fed to driver.
(P2)- "send-and-receive" procedures, where we send a message to the candidate MSH,
and observe its response-message. This usually involves the test driver on candidate
MSH side, to provide response data (e.g. 3.1.6.3a), but sometime the response is
generated directly by the candidate MSH (in case of errors, e.g. 3.1.6.3bc, 3.1.6.4)
 A Test case definition for such procedures is made of:
(1) test item requirement def (and associated procedure),
(2) message data to be sent to candidate MSH (described using same XML neutral format).
(3) message data to be fed to driver for response to test driver of candidate, if applicable.
 
We probably should start thinking of how to more completely describe such test cases.
I can give a first cut at a  "message data" XML format, and give it for review.

Regards,

jacques Durand

-----Original Message-----
From: Matthew MacKenzie [mailto:matt@xmlglobal.com]
Sent: Wednesday, March 27, 2002 11:02 AM
To: ebxml-iic-msg@lists.oasis-open.org
Subject: [ebxml-iic-msg] MS Conformance checklist

Group,

Our (XMLG's) ebxml messaging team has been working on a checklist to
guide our own product testing efforts, I've attached it for your
information.  There are quite a bit of items here, and it is my hope
that I will be able to use these as a basis for our notion of "level 1"
conformance.  I expect that Mike and Myself will release the next
iteration of our conformance requirements schema and the level 1
instance of that schema relatively soon.

Regards,

Matthew MacKenzie
XML Global R&D
PGP Key available upon request.