OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]

Subject: [ebxml-iic] VanLydegraf 10/2/2002: [ebxml-iic-interop] interop update

Great, Eric.

(1) Use guidelines for test driver:
<evl>I think this is an excellent point.
     would this be in a test framework specification though? </evl>

(2) <evl>I think we have to clean up the model of the Test Driver in
general so
that common concepts between conformance and interop can be kept and
differentiating concepts can be clarified.<evl>

(3) Where Test Driver begins and Test Service ends?

mm1: Bring up in Monday's meeting. #2 may take care of #3 in part at

Can we have some discussion from the floor?

-----Original Message-----
From: Eric VanLydegraf [mailto:ericv@kinzan.com]
Sent: Wednesday, October 02, 2002 7:45 PM
To: Monica Martin; Eric VanLydegraf; Jacques Durand
Cc: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: VanLydegraf 9/30/2002: [ebxml-iic-interop] interop update

Comments <evl>...</evl> below
-----Original Message-----
From: Monica Martin [mailto:mmartin@certivo.net]
Sent: Monday, September 30, 2002 11:44 AM
To: Eric VanLydegraf; Jacques Durand
Cc: Monica Martin
Subject: VanLydegraf 9/30/2002: [ebxml-iic-interop] interop update

I would propose three items that will assist us to move forward with
(that affects our synergy with conformance effort as well):

*	Pre-requisite of conformance test for interoperability testing:
Drive a decision whether this is a STRONGLY RECOMMENDED or a SHALL.  The
pre-requisite is desired over the long-term but may limit
interoperability framework adoption in the near-term if a SHALL.

<evl>I would go with STRONGLY RECOMMENDED as that still leaves an out,
discourages the practice</evl>

*	Focus on 'use' guidelines for the Test Driver as it may differ
between Conformance or Interoperability.  May be as simple as a matrix
that identifies the different roles that the Test Driver may assume and
whether if fits into Conformance, Interoperability or both.

<evl>I think this is an excellent point.
     would this be in a test framework specification though ?     

*	Drive to a decision on the Conformance Level: I make a proposal
that we accept the Conformance Levels that have been discussed and
'suggest' what level will be most used by what audience.  I think this
will also benefit the Interoperability effort to understand how to
position our long-term goal to have Conformance as a pre-requite to

<evl> it would help to break out the interop tests themselves by levels
or grouping by conformance levels would seem to make the most

*	As for integrating the trigger (start of the test) of the Test
Driver within a Test Step, I would propose we allow the flexibility of
either.  Perhaps if the Test Driver is 'started' outside of the Test
Step, this could be another implementation adapter such as the others
Eric has defined (although ensuring the function and independence of the
concept we have defined).
<evl>how about an abstract definition of the Test Driver and either
the messaging (wire) or in-process trigger as possible ways to implement
such a driver or keep the interop spec pure and have appendix, extra
material describing implementation guidelines and keep trigger as very
<evl>I think we have to clean up the model of the Test Driver in general
that common concepts between conformance and interop can be kept and
differentiating concepts can be clarified.

It seems the model is

A] Trigger (start of testing process)
B] Initiator (start of test)
C] TestCase validation (conditions have been met to proceed to testing)
D} Test Monitor (sniffing,log tracing, event trapping)
E] Test verification (expected results occured)
F] Test Result (log test result)
G] Test Report (aggregate all test results and provide feedback/score)

I'll have to think some more about where Test Driver begins and Test
ends and where they meet in the middle ;)
Also how different interop vs conformance architectures are and where
are the same.


-----Original Message-----
From: Eric VanLydegraf [mailto:ericv@kinzan.com]
Sent: Thursday, September 26, 2002 3:45 PM
To: 'Jacques Durand'
Cc: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

Well thanks jacques,
    in that case here are my comments on the 0.40 document
Introduction 1 states conformance testing (should be interoperability
correct ?)
It seems the document needs to have an interoperability definition and
motivation statement.
The term is used everywhere but only defined in Appendix A (which BTW
follows Appendix B ???) and properly differentiated from conformance.
I would specifically add a Pre Requesite heading section stipulating the
conformance to have passed prior to running interop.
I was also thinking over the Test Service & Test Driver framework and
wondering about
having the Test Service be initiated with ebXML messages itself and
having notifications and events signalled via messages.
This has the advantage of keeping the Test Driver and Test Service
separate and using ebXML messagging as the logging and tracing.
It also means we can configured the Test Service entirely through a
message - with the test cases as payload. The conversation_ID with the
Test Driver
determines the Test Case being referenced. With timeouts the Test Driver
can determine test failure. So the Test Driver is both initiatior and
logger of the test run, but the actuall testing occurs between candidate
it seems we need a template, as the tests will be very similar and only
certain tests will be excluded rather than different.
It seems more appropriate to list all tests and have a PreCondition
table stating for what protocols it applies to.
That way all testers step through each Test Case and test if needed
otherwise skip to next one.
Or alternatively write up the Test Cases in one section and make a table
with checkmarks on which protocols apply to the Test Case.
Test Case 1.1 No payload basic exchange 

Protocols [HTTP,SMTP]
Test Case 1.12 Synchronous Unsigned Acknowledgement  exchange

Protocols [HTTP]
I think with some more discussion and thought about the test framework -
namely the test driver/test service we could more thoroughly
go through the test steps.

-----Original Message-----
From: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Sent: Tuesday, September 24, 2002 11:14 AM
To: 'Eric VanLydegraf'
Cc: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

Yes, latest draft is 0.4 you got by mail. Not posted yet on our site.
We need to review in particular Section 3:
- the detail of the test steps for each test case.
- the notion of having two "basic interop profiles", concretized by a
test suite for HTTP, 
and another test suite for SMTP. (no "option" inside a profile.)
- the content of these basic interop profiles (as drafted, they roughly
match what remains
from the DGI tests, after removing (a) conformance tests, (b) optional
tests, (c) specific tests like large payloads.)

-----Original Message-----
From: Eric VanLydegraf [mailto:ericv@kinzan.com]
Sent: Monday, September 23, 2002 6:06 PM
To: 'Jacques Durand'; 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

Just to confirm 0.40 is the latest interop document ?

-----Original Message-----
From: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Sent: Monday, September 09, 2002 5:45 PM
To: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: [ebxml-iic-interop] interop update

Steve, Hatem, Eric: 

Here is an incremental update of the Interop suite document, for your
based on improvement we discussed in past conf calls as well as F-2-F. 

- Section 1.2 (concept of operations) is expanded. 

- References to third-party tests (here, DGI) has been removed, as this
is a general, abstract test suite, 
that will be used as a reference by third parties. Comparisons with
UCC/DGI test suite will be done separately, 
as we expect for other existing test suites (ECOM). (We'll need to do
such "comparative evaluations" 
if only as a basis for discussion when negotiating with these entities,
so that they understand how "compliant" 
they are with our tests, and also so that they can give feedback on what
may be missing in ours. 

- list of Actions of  Test Service (Section 2.2) is slightly updated
(configurator, also the message sending behavior 
of all these actions is the same whether the Test Service is associated
or not with a Test Driver.) 

- still need to refine the way a Test Driver will drive the test cases
by directly invoking the Test Service "Initiator" action.

- only one interop profile test suite described here for HTTP (Section
3), based on Steve's list of test cases 
 as we went through in F-2-F. 
It is roughly equivalent to: { DGI, minus conformance tests, minus
optional tests, minus HTTP/S } 
Detail of test cases (test steps, CPA...) still needs to borrow from
Mike's material. 
Still to discuss: idea that there is an interop profile for HTTP,
another vfor SMTP. (see Section 3.3) 




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]

Powered by eList eXpress LLC