OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic-msg message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]

Subject: [ebxml-iic-msg] RE: other Schema docs - usage?

Title: Welcome
Michael, Matt:
I believe the "Test requirements" document and schema are on good tracks.
Let me try to summarize where we are, and raise some remaining issues:
1- The Requirements schema/doc, so far captures Level 1 ("core", or most of it).
When the corresponding tests are run (controlled by ANT + JXUnit), one can check that
an ebXML message is well-formed and also semantically sound, according to Level 1.
That actually captures "wire-conformance" Level 1.
2- We expect to drive the candidate MSH so that it generates a diverse-enough sampling of messages,
each of which will undergo the Level 1 "wire-conformance" tests above.
3- Issue: as we will drive the candidate MSH to generate messages to be tested, it is not enough
to test each of these through the "Level 1 tests" above. Indeed, we want to make sure in addition that
the set of generated messages (test set), uses all features expected in Level 1.
Example: if for some reason the candidate MSH fails to generate some optional message elements - e.g.
digital signature, etc. - then its message format may still be correct, as tested on the wire, but
we cannot claim Level 1 passes, as the test coverage would not have checked all expected message features.
Solutions: we need to have a way to "require" and to check, in the generated message test set,
that all expected message features have ben produced at least once. That could be done by (a) either
keeping a checklist of all message features that are concerned by Level 1, and match it with what has
actually been teted, (b) or by making sure that the test associated with each Level 1 Requirement has been
used at least once (<condition> part successful?), (c) or by doing a more through comparison between
what the test driver has been fed (input test cases) and what is observed on the wire. note
that would have  some "service-conformance" aspect.
Just following up on Michael schemas (ebXMLCompare.xsd, ebXMLSet.xsd, ebXMLTestSuite.xsd):
4- Michael: could you give a quick overview / use case of how you see instances of these schemas being used?
(I assume that instances of ebXMLTestSuite.xsd will be input to a "comparison engine"?)
5- I assume these schemas will then be used to compare actual output / expected output, right?
(that could then help on issue #3-c above.)
6- Issue: We'll need to define how to compare schema instances, e.g. in some cases, some distance is allowed
with the reference document (e.g. optional element, order, different values...)
7- COnformance reports - should it be instance of an extended Requirements schema with test results?
-----Original Message-----
From: Jacques Durand [mailto:JDurand@fs.fujitsu.com]
Sent: Monday, March 18, 2002 3:42 PM
To: 'ebxml-iic-msg@lists.oasis-open.org'
Subject: [ebxml-iic-msg] Welcome


Welcome to the ebxml-iic-msg sublist.

This is where you will read or post mails about MS conformance testing design,
and more generally about the "Conformance Testing Task Force" activities and material.


Jacques Durand
ebXML IIC chair

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]

Powered by eList eXpress LLC