RE: Request of information about ebXMLTestFramework

["Michael Kass" <michael.kass@nist.gov>]

Jacques and all,

   Here is a copy of my reply to Laura Bianchini re: Test Framework
issues/discrepancies.

Regards,
Mike


-----Original Message-----
From: Laura Bianchini [mailto:bianchin@cs.unibo.it]
Sent: Wednesday, March 10, 2004 11:47 AM
To: michael.kass@nist.gov
Subject: Request of information about ebXMLTestFramework


Dear mr Michael Kass,

I am concerned with the B2B standardisation for the Textile/Clothing sector.
(http://www.moda-ml.it)

We have developed a framework based on XML and ebXML transport
specification and we are trying to build a conformance test to check the
adherence of the implementations to the requirements of the framework.

We have analysed the document (downloaded from
"http://www.oasis-open.org/committees/documents.php"; ):

   - ebXML Test Framework specification, version 1.0 (downloaded from

"http://www.oasis-open.org/committees/download.php/1990/ebXML-TestFramework-
10.zip")

   - ebXML Messaging Service Specification, version 2.0 (downloaded from
        "http://www.ebxml.org/specs/index.htm";)

   - ebXML MS V2.0  Conformance Test Suite
        ("IIC_MS_20_Conformance_TestSuite_V1.0.doc" downloaded from

"http://www.oasis-open.org/committees/download.php/4086/ebMS2.0ExeConfTestSu
ite.zip")

   - ebXML MS V2.0 Executable Conformance Test Suite V1.0
        ("IIC_MS_20_Executable_Conformance_TestSuite.doc" downloaded from

"http://www.oasis-open.org/committees/download.php/4086/ebMS2.0ExeConfTestSu
ite.zip")

   - an XML version of the TestSuite
        ("ebxml_ms_20_executable_testsuite.xml"  downloaded from

"http://www.oasis-open.org/committees/download.php/4086/ebMS2.0ExeConfTestSu
ite.zip")


We want to realize a structure for "Remote Conformance Testing of a Message
Handler", but we have meet a lot of problems because in each document "the
test cases" are sometimes explained in different ways and we found that
the examples of "ebXML Test Framework specification" are not exactly
adherent to the specification.
Example:
        -  In the "ebXML Test Framework specification, version 1.0", 9.1.1
Conformance Test Requirement,there aren't the attributes ConditionRef and
AssertionRef, defined as "required" in the 5.2 of the same document.
[MIKE] - Added to Test Framework errata and fixed. ConditionRef and
AssertionRef are now "optional" in documentation, to match schema
definition.


        - In the "ebXML Test Framework specification, version 1.0", 9.3.1
Conformance Test Suite, the element "<ebTest:TestStep
ebTest:mode="connection">" in the specification hasn't the attribute mode.
[MIKE] - Removed the "mode" attribute from <TestStep> because it is highly
unlikely that we will switch "modes" in the middle of a <TestCase>


        - "IIC_MS_20_Conformance_TestSuite_V1.0.doc" VS
            "ebxml_ms_20_executable_testsuite.xml"
          Testcase 12 in the first document is different from Testcase 12
in the second. In the first,there is only an essertion while in the second
there are Precondition and Assertion.What is the right solution?
[MIKE] - Both should be <TestAssertion> operations.  I also found a number
of cases where <TestPreCondition> is improperly used in both the conformance
test suite and executable test suite. <TestPreCondition> must only be used
in cases where an "optional" feature may or may not exist as a prerequisite
to testing an assertion.  I have modified the executable test suite to fix
this (removing <TestPreCondition> elements, or changing them to
<TestAssertion> elements where appropriate.  Changes are reflected in the
Executable Test Suite errata list, to be posted on the OASIS IIC web site.


Please could you tell us if we have used the right documentation or give us
any link or material, as much recent as possible, that could allow us to
find more up-to-date information on this area.
(If you are interested we can tell you the more details about the problems
that we found in our work)

[MIKE] - We will shortly be posting V1.1 of the Test Framework specification
, along with 1.1 of the Executable Test Suite, Conformance Test Suite,  and
Conformance Test Requirements.  They will be available on the oasis IIC web
site at:
http://www.oasis-open.org/committees/documents.php?wg_abbrev=ebxml-iic


Thanks for you attention.

Laura Bianchini
Undegraduated student of computer science (Bologna-Italy)