RE: [ebxml-iic] Latest MS Conformance Documents

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: RE: [ebxml-iic] Latest MS Conformance Documents

Mike, Monica:

I am not sure if we should have all the current status on
test centers, and interop initiatives (KorBIT/POSTECH, ITG, OAGI, NIST...)
in this document, at least I am questioning this in the introduction.
I see the following issues in having this material in intro of MS conformance:
- these are mostly related to interoperability, for now. But this is a conformance test suite.
- we do not have yet assurance of these entities committing yet to do conformance testing,
at least the way we recommend it. It would be embarrassing if any of these choose later to
do conformance in another way than IIC...
- as a spec document, it is delicate to use names of companies / orgs that are supposed to
either implement the spec, or provide requirements. It was OK for the MS interop suite,
as our objective was openly to design a common test base (subset of each.), and there was
already material out there that we could not ignore.
- unlike interop suites, which are admittedly more dependent on user requirements, conformance
is driven mostly by the spec.
- we may forget to mention some of the players...

We may want to add more explicitly an "application" section, where we abstractly refer to
the context of test centers, and how we fit in, or how we expect such a test suite to be used.
(though we did that in the Test Framework I think). We may also expand on how complementary
conformance and interop suites are. The notion of "profiles", and our choices here,
may be explained here. If we still believe concrete initiatives need be mentioned, that could be in
an appendix.
But that is my opinion... we can discuss this further.

Regards,

Jacques

-----Original Message-----
From: Michael Kass [mailto:michael.kass@nist.gov]
Sent: Saturday, July 19, 2003 1:37 PM
To: Monica J. Martin; ebXML IIC - main list (E-mail)
Cc: Monica J. Martin; Jacques Durand
Subject: Re: [ebxml-iic] Latest MS Conformance Documents

Monica,

  Thanks for your comments and addition to section 1.6.2
I'll be sending out a revised version of the spec tomorrow ( Sunday ).

  My comments are included below:

Mike

7.2, line 248, recommend SHOULD rather than MUST as we want to strongly
encourage/recommend.

[MIKE] - Agreed.  Wording changed to SHOULD

We may also want to soften line 252, to show value of
conformance testing as an input to interoperability tests.

[MIKE]  - I changed the wording to emphasise that conformance testing
SHOULD be done prior to interop testing, and that certain types of tests
belong
in conformance vs. interop test suites.

1.6, line 198 and 205, suggest we combine these two paragraphs to compare
the goals and results of conformance and interoperability testing, which
will increase understanding of the importance of the former.

[MIKE] - I combined the two paragraphs into one.

1.6.2 Input from Martin below:

Introduction
The growth of test center solutions is a logical indicator of the maturity
and adoption of ebXML
in the eBusiness marketplace.  These test hubs are being developed and
coordinated worldwide as
nucleus centers to speed standards adoption verification and expedite
deployment.

In the interoperability space, several small and large-scale ebXML test
center-related efforts exist, including:

· B2B Interoperability Testbed: Open Applications Group, Inc. (OAGI) and
the National Institute of Science and Technology (NIST)

A major focus during the initial testbed activities was to show how a
Web-based interoperability demonstration
and testing infrastructure could satisfy needs of the customers and software
vendors at various levels (for example,
message protocol, process and business content).  An ongoing key focus area
of the testbed is to test for conformance
and interoperability of semantics from a common data dictionary. NIST is
also working with the KorBIT team.

· Korean B2B/A2A Interoperability Testbed (KorBIT): POSTECH University,
Government of South Korea, et al.

The Korean team also collaborates with the Asian Interoperability Task Group
(ITG) under the eAC (ebXML Asia Committee),
particularly to obtain technical requirements.

· Asia Interoperability Task Group (ITG): Electronic Commerce Promotion
Council of Japan (ECOM),
University of Hong Kong's Center for E-Commerce Infrastructure Development
(CECID), and Korea
Institute of Electronic Commerce (KIEC), et al.

The three recent successful Asian ITG test events have focused on enabling
business solutions for small
businesses and developing countries.  The AITG interoperability tests are
similar to those developed in the ebXML IIC
Interoperability Profile, and provide additional test constructs based on
their regional requirements.

In a phased approach, the diverse KorBIT team that represents academia,
solution providers, government, industry, and
end users will provide scenario and business document validation, beta
testing and definition of business processes
and documents, and research to advance development of tools and techniquest
to support test centers  KorBIT is working
with NIST and the ebXML IIC, in advancing test development in both the
conformance and interoperability areas.

· eBusiness Board for European Standardization (eBES) ebXML Interoperability
Pilot: European Committee for
Standardization Information Society Standardization System (CEN-ISSS) and
OASIS

This accelerated test effort focused on engineering (ebMS interoperability)
and demonstration for business relevancy
(using a real-life business scenario). In the first phase, the dual team
test plan and demonstration were based on the
ebXML IIC test framework.  A second phase is being discussed for the CPA,
Registry and Business Process.

[MIKE] - Added this in the "interoperabilty space" section of related
activities

2.1, line 313, do we need to qualify this a bit more?  Are there any
restrictions or differences in associated with
either the pre-CPA or dynamic through the configurator action? May we
discuss further?  Perhaps these should be
different profiles or parameters on a profile, much as we identified for
SMTP vs. HTTP, correct? See also section 3.3.1.

[MIKE] - I agree that this section is confusing.  I would suggest removing
this paragraph, since it is more relevant and better
describe in the ebXML Test Framework document than in this document, and
should only be referenced in section 2.2 ( Test Service and
Actions ).

4.0, line 462, say that the parameters defined by the test framework
restrict rather than saying limitations.
These type nuances are primarily a constraint of the testability of the
requirements, i.e. the test vs. the specification
coverage, etc. rather than the limitation of the framework.

[MIKE] - Agreed and changed

5.1.6.2, provide a reference back to the CPPA 2b (explicit)

[MIKE] - I recommend removing this section, because this is Executable Test
Suite material, and is not really
relevant here.  Actually, this is an artifact left over from the
Interoperability Test Suite, and does not have any relevance in the
conformance
test suite.

5.2.1, does the green designate full test coverage?

[MIKE] - The green highlighting is a visual aid in identify "non-required"
test assertions ( meaning that they do not have to be implemented ).
All STRONGLY RECOMMENDED, RECOMMENDED or OPTIONAL requirements are
highlighted in green.  I added a sentence explaining
the green highlighting in the spec.

General:
In looking at the test cases include, we see patterns of execution in
several tests.  Do we see the capability to
provide a shell that can be used repeatedly across multiple test cases?

[MIKE] - I believe that we could extend the XML scripting to support
"re-use" of patterns.  The easiest way to do that might be to introduce
re-use
of Test Steps through ID reference.  It would be trivial to introduce IDs to
<TestStep> elements, and then reference them in the scripting.  Currently,
this is not defined for the Test Suite scripting schema.