Re: [ebxml-iic] Latest MS Conformance Documents

["Monica J. Martin" <monica.martin@sun.com>]

Monica J. Martin wrote:

> Michael Kass wrote:
>
>> Jacques and all,
>>  
>>    Attahed is the ZIP file of all ebXML MS 2.0 Conformance Testing 
>> Material to date.
>> Included are :
>>  
>> Conformance Test Suite Specification ( including Test Requirements 
>> and Abstract Test Suite)
>> Updated Annotated MS Specification (illustraing Test Requirement 
>> Coverage)
>> Test Case Coverage Document ( illustrating Test Case coverage of 
>> individual Test Requirements )
>> ebXML MS 2.0 Executable Test Suite
>>  
>
>
> mm1: Here are a few comments as I begin my review including the 
> section on test activities worldwide.
> mm2: Updated comments attached.  Please discard the old ones as they 
> are included in updated version.


1.7.2, line 248, recommend SHOULD rather than MUST as we want to strongly 
encourage/recommend.  We may also want to soften line 252, to show value of
conformance testing as an input to interoperability tests.
1.6, line 198 and 205, suggest we combine these two paragraphs to compare
the goals and results of conformance and interoperability testing, which
will increase understanding of the importance of the former.
1.6.2 Input from Martin below:

Introduction 
The growth of test center solutions is a logical indicator of the maturity and adoption of ebXML
in the eBusiness marketplace.  These test hubs are being developed and coordinated worldwide as
nucleus centers to speed standards adoption verification and expedite deployment.

In the interoperability space, several small and large-scale ebXML test center-related efforts exist, including:

·	B2B Interoperability Testbed: Open Applications Group, Inc. (OAGI) and
	the National Institute of Science and Technology (NIST)

A major focus during the initial testbed activities was to show how a Web-based interoperability demonstration
and testing infrastructure could satisfy needs of the customers and software vendors at various levels (for example,
message protocol, process and business content).  An ongoing key focus area of the testbed is to test for conformance
and interoperability of semantics from a common data dictionary. NIST is also working with the KorBIT team.

·	Korean B2B/A2A Interoperability Testbed (KorBIT): POSTECH University, Government of South Korea, et al.

The Korean team also collaborates with the Asian Interoperability Task Group (ITG) under the eAC (ebXML Asia Committee),
particularly to obtain technical requirements.  

·	Asia Interoperability Task Group (ITG): Electronic Commerce Promotion Council of Japan (ECOM), 
	University of Hong Kong's Center for E-Commerce Infrastructure Development (CECID), and Korea
	Institute of Electronic Commerce (KIEC), et al.

The three recent successful Asian ITG test events have focused on enabling business solutions for small
businesses and developing countries.  The AITG interoperability tests are similar to those developed in the ebXML IIC
Interoperability Profile, and provide additional test constructs based on their regional requirements.

In a phased approach, the diverse KorBIT team that represents academia, solution providers, government, industry, and
end users will provide scenario and business document validation, beta testing and definition of business processes
and documents, and research to advance development of tools and techniquest to support test centers  KorBIT is working
with NIST and the ebXML IIC, in advancing test development in both the conformance and interoperability areas.

·	eBusiness Board for European Standardization (eBES) ebXML Interoperability Pilot: European Committee for
	Standardization Information Society Standardization System (CEN-ISSS) and OASIS

This accelerated test effort focused on engineering (ebMS interoperability) and demonstration for business relevancy
(using a real-life business scenario). In the first phase, the dual team test plan and demonstration were based on the
ebXML IIC test framework.  A second phase is being discussed for the CPA, Registry and Business Process.

2.1, line 313, do we need to qualify this a bit more?  Are there any restrictions or differences in associated with
either the pre-CPA or dynamic through the configurator action? May we discuss further?  Perhaps these should be 
different profiles or parameters on a profile, much as we identified for SMTP vs. HTTP, correct? See also section 3.3.1.

4.0, line 462, say that the parameters defined by the test framework restrict rather than saying limitations.
These type nuances are primarily a constraint of the testability of the requirements, i.e. the test vs. the specification
coverage, etc. rather than the limitation of the framework.

5.1.6.2, provide a reference back to the CPPA 2b (explicit)

5.2.1, does the green designate full test coverage?

General:
In looking at the test cases include, we see patterns of execution in several tests.  Do we see the capability to
provide a shell that can be used repeatedly across multiple test cases?