RE: [ebxml-iic-msg] RE: [ebxml-iic] Latest versions of Schema andinstance documents for ebXML conformance testing

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Matt, Mike:

 

Great. We will do a thorough review of your on-coming release.

We'll certainly discuss it in Barcelona - whether or not in formal meetings

(we might have to set 1 or 2 evening sessions for this).

Those not going in Barcelona are encouraged to review too... and to post comments/questions on this list.

 

My feeling is that even though the XML material is very close to completion,

there is still quite a bit of editing to do: we need to produce a well-rounded document

that includes:

1- general approach (automation level expected,  what we assume is already tested in SOAP, HTTP...,

how tests are supposed to be conducted - remotely/locally, expected level of detail of test outputs,

handling of "recommended"...)

2- assumed test architecture/harness (AT-driver, MT-driver, how tests are executed on this,

configuration/preparation, different testing procedures supported, control of test case steps,

output trace generation and when is it consolidation, when does the condition verification occur,  

possibility to handle concurrent tests...)

3- The overall verification process (how trace items are correlated,

XML mark-up for the trace, supported operations on the trace as appear in Test Steps of test cases,

overall verification / certification function, reporting and error reporting)

4- the test suite itself (list of test reqs and test cases , how it relates to spec, suggested order of execution,

the test steps operations and conditions, the XML mark-up)

 

I will help on that (can write on 1 and 2)

 

Things we still need to address somehow:

- the CPA-based configuration.

- ?

 

So I think time is too short for the on-coming OASIS round of submissions, June 1st.

(every quarter). See email TC chairs got from Karl below:

 

>(from Karl Best) This is a reminder that the submission deadline for the next quarter is

>1 June. If your TC will be submitting an approved Committee

>Specification for consideration as an OASIS Standard I must receive it

>by that date.

 

My feeling is we should be done end of june with a well polished MS Conformance test suite and document,

that goes way beyond the MS core level. We will submit for TC review and vote at this time.

Once voted as a TC spec (mid-July?) we may release it for review / implementation to MS developer/user community,

so we still put a stake out there without waiting next OASIS round.

All this will also synergize with the Interop test suite that should follow very closely.

Both test specs should be ready for the OASIS submission round of September 1st,

and ideally we would even have some partial implementation (e.g. the verification/reporting function

working from the trace)

 

Let me know your opinion on this plan...

 

Regards,

 

Jacques

-----Original Message-----
From: Matthew MacKenzie [mailto:matt@xmlglobal.com]
Sent: Tuesday, May 14, 2002 10:40 PM
To: Jacques Durand
Cc: 'ebxml-iic-msg@lists.oasis-open.org'
Subject: Re: [ebxml-iic-msg] RE: [ebxml-iic] Latest versions of Schema and instance documents for ebXML conformance testing

Jacques,

We will be making our first official release to the group on Thursday or Friday containing all of the schemas, some documentation including HTML and flowcharts, "Level 1" test requirements, and hopefully some examples. That is certainly my deadline, as my flight to Barcelona departs on Saturday. I think Mike is doing that now, but its down to the wire as we have both been making changes to the TestCase schema (relatively major changes) today.

I think you will be able to post our output directly on the ebxml-iic page at OASIS so that everyone can have ample chance to look it over, make suggestions and so on in preparation for a vote to publish.

-Matt

On Tuesday, May 14, 2002, at 08:16 PM, Jacques Durand wrote:

Mike / Matt:

Started to review you latest version (early May).

- Would that be possible to get about 10 test cases fully described, so that we have an idea if the mark-up is OK?

(especially the Test Steps)

- just noticed in the example you gave (r1.2.9) that this particular test (consistency of conversation ID) is tricky,

because the conv ID, unlike the messageID,  is formally set by the application (so if test fails it may

just be that the app goofed off). So normally the test should also include an app driver on the candidate MSH

feeding the MSH with a response message with right conv ID (+ right RefToMsgId)...

(so in case of failure, that really means MSH is at fault).So this test is more complex than it looks...

- the overall Test Case format looks nice - mixing clear English statements with the formal test conditions.

That makes it possible to someone to use these tests even if they do not have yet the automated drivers to

process them.

Cheers

Jacques 

 -----Original Message-----

From: Michael Kass [mailto:michael.kass@nist.gov]

Sent: Monday, May 06, 2002 8:51 AM

To: ebxml-iic@lists.oasis-open.org

Subject: [ebxml-iic] Latest versions of Schema and instance documents for ebXML conformance testing

To all,

   The link below describes the latest versions of schemas and instance documents reflecting ebXML MS level1 conformance testing.  This covers testing requirements, test cases and test trace format and instance documents, including changes based upon discussion the previous two weeks.

http://xsun.sdct.itl.nist.gov/~mkass/ebXML/ebXML%20Test%20Suite%20Design.htm

Regards,

Mike