OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic-interop message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]

Subject: RE: [ebxml-iic-interop] interop update

Title: interop update
-----Original Message-----
From: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Sent: Thursday, September 26, 2002 4:37 PM
To: 'Eric VanLydegraf'
Cc: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

My comments - (Steve and Hatem may add theirs especially if they disagree!)
-----Original Message-----
From: Eric VanLydegraf [mailto:ericv@kinzan.com]
Sent: Thursday, September 26, 2002 2:45 PM
To: 'Jacques Durand'
Cc: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

Well thanks jacques,
    in that case here are my comments on the 0.40 document
Introduction 1 states conformance testing (should be interoperability correct ?)
<Jacques> right
It seems the document needs to have an interoperability definition and motivation statement.
The term is used everywhere but only defined in Appendix A (which BTW follows Appendix B ???) and properly differentiated from conformance.
<Jacques> correct. I ahve not seen a "formal-enough" definition so far, so I tried to craft one some time ago.
Here it is. Let me know if you know of any other.
we can use it if you guys agree.
I think the notion of "contract" is essential - allows you to test things
that are out of scope of the spec, like interop with "large payloads", or interop with options from spec that are
narrowed down (according to the "contract"):
Interoperability Testing: Process of verifying that two [Eric VanLydegraf] (or more) implementations of the same specification, or
 that an implementation and its operational environment, can interoperate according to the requirements
of an assumed agreement or contract. This contract does not belong necessarily to the specification,
but its terms and elements should be defined in it with enough detail, so that such a contract, combined
with the specification, will be sufficient to determine precisely the expected behavior of an implementation,
and to test it.
We may need to talk a bit about the expected methodology (matrix testing, vs. hub-and-spoke testting against
some "reference" implementation)
I would specifically add a Pre Requesite heading section stipulating the conformance to have passed prior to running interop.
<Jacques> absolutely.
I was also thinking over the Test Service & Test Driver framework and wondering about
having the Test Service be initiated with ebXML messages itself and having notifications and events signalled via messages.
This has the advantage of keeping the Test Driver and Test Service separate and using ebXML messagging as the logging and tracing.
It also means we can configured the Test Service entirely through a message - with the test cases as payload. The conversation_ID with the Test Driver
determines the Test Case being referenced. With timeouts the Test Driver can determine test failure. So the Test Driver is both initiatior and logger of the test run, but the actuall testing occurs between candidate MSH's.

<Jacques> tricky issue. My main concern about initiating test cases via messages sent by the Test Driver (like
we do in conformance testing - but for conformance that is not a problem), is that this introduces a third end-point
on the communication. In a deployed environment, where network access may be subject to all kinds of restrictions,
this may be a problem (and we need be able to test interop at any time, on a periodical basis.)
 You may not be able to send easily messages to your own MSH. In addition, it may craete
"noise" in the log, send back messages or errors, you need to create an additional conversation, there may be security problems,
you don't want your MSH to interact with your driver using same CPA as the ones (several different) used for interop test cases,
so you'd need to switch to a special CPA to be installed, etc. 
All that may get messy and restrictive, even if the architecture looks attractive because just a building block game.
I think in the f-2-f we came-up with the idea that :
we would have an integration  Test Driver / Test Service, i.e. a bundled version of the test driver would be able to directly
invoke the Initiator service action (from inside same process), without going through the wire/MSH to do so.
That seems much less problem-prone, and leaves the communication and MSH layer untouched - no side effects, no
intrusion, no reconfiguration, etc. The test driver will have a mode where instead of crafting messages to send
a payload to the local Test Service (Initiator action), it will just directly invoke the Initiator action
(via the same internal interface that is used by the MSH) and directly pass the payload to it.
For the notification, same thing: actions will notify message material (header data + payload) to test driver directly through
its interface.
[Eric VanLydegraf] See Monica's email about describing both but from the concept level we only need to specify the role/requirement of the trigger function of the Test Driver
it seems we need a template, as the tests will be very similar and only certain tests will be excluded rather than different.
It seems more appropriate to list all tests and have a PreCondition table stating for what protocols it applies to.
That way all testers step through each Test Case and test if needed otherwise skip to next one.
Or alternatively write up the Test Cases in one section and make a table with checkmarks on which protocols apply to the Test Case.
Test Case 1.1 No payload basic exchange
Protocols [HTTP,SMTP]
Test Case 1.12 Synchronous Unsigned Acknowledgement  exchange
Protocols [HTTP]
<Jacques> I guess it is mostly a matter of specification style - I 'd still like to clearly and unambiguously
identify each Interop Profile (i.e. their corresponding test suite) as "optionless".
I.e. each test suite is totally deterministic (interop "contract" is fully defined within, e.g. with the precise protocol you
will use, e.g. HTTP), and ther eis only one way to execute it.
So this still allows for using a parameterized "profile template" if you will - but we would clearly distinguish two instances of
it (e.g.in our case, one for HTTP, one for SMTP), and consider them as two different interop profiles .
Would that be consistent with what you are proposing?
e.g.. :
TemplateTestSuiteXYZ ( parameters: protocolname,  protocoltype )
Test Case 1.1 No payload basic exchange 
Test Case 1.12 Synchronous Unsigned Acknowledgement  exchange 
Interop Profile A:
TemplateTestSuiteXYZ ( HTTP, synchronous)
Interop Profile B:
TemplateTestSuiteXYZ ( SMTP, asynchronous)
I think with some more discussion and thought about the test framework - namely the test driver/test service we could more thoroughly
go through the test steps.
[Eric VanLydegraf] Yes your example conveys the idea, we can bang up on it in the document to find the best format.
-----Original Message-----
From: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Sent: Tuesday, September 24, 2002 11:14 AM
To: 'Eric VanLydegraf'
Cc: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

Yes, latest draft is 0.4 you got by mail. Not posted yet on our site.
We need to review in particular Section 3:
- the detail of the test steps for each test case.
- the notion of having two "basic interop profiles", concretized by a test suite for HTTP,
and another test suite for SMTP. (no "option" inside a profile.)
- the content of these basic interop profiles (as drafted, they roughly match what remains
from the DGI tests, after removing (a) conformance tests, (b) optional tests, (c) specific tests like large payloads.)
-----Original Message-----
From: Eric VanLydegraf [mailto:ericv@kinzan.com]
Sent: Monday, September 23, 2002 6:06 PM
To: 'Jacques Durand'; 'ebxml-iic-interop@lists.oasis-open.org'
Subject: RE: [ebxml-iic-interop] interop update

Just to confirm 0.40 is the latest interop document ?
-----Original Message-----
From: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Sent: Monday, September 09, 2002 5:45 PM
To: 'ebxml-iic-interop@lists.oasis-open.org'
Subject: [ebxml-iic-interop] interop update

Steve, Hatem, Eric:

Here is an incremental update of the Interop suite document, for your review
based on improvement we discussed in past conf calls as well as F-2-F.

- Section 1.2 (concept of operations) is expanded.

- References to third-party tests (here, DGI) has been removed, as this is a general, abstract test suite,
that will be used as a reference by third parties. Comparisons with UCC/DGI test suite will be done separately,
as we expect for other existing test suites (ECOM). (We'll need to do such "comparative evaluations"
if only as a basis for discussion when negotiating with these entities, so that they understand how "compliant"
they are with our tests, and also so that they can give feedback on what may be missing in ours.

- list of Actions of  Test Service (Section 2.2) is slightly updated (configurator, also the message sending behavior
of all these actions is the same whether the Test Service is associated or not with a Test Driver.)

- still need to refine the way a Test Driver will drive the test cases by directly invoking the Test Service "Initiator" action.

- only one interop profile test suite described here for HTTP (Section 3), based on Steve's list of test cases
 as we went through in F-2-F.
It is roughly equivalent to: { DGI, minus conformance tests, minus optional tests, minus HTTP/S }
Detail of test cases (test steps, CPA...) still needs to borrow from Mike's material.
Still to discuss: idea that there is an interop profile for HTTP, another vfor SMTP. (see Section 3.3)




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]

Powered by eList eXpress LLC