[ebxml-iic-interop] FW: Feedback from eBES on our Interop Test suitedraft

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: RE: IIC specs

Our interop sub-team may be interested in this feedback

from Christopher Frank , architect in the eBES interoperability pilot project.

(I answerred it partially)

 

Christopher had also a question on whether there was implementation work plans

for our test framework. We need to be able to answer such questions, even though

that's beyond the IIC charter.

 

Regards,

 

Jacques

 

-----Original Message-----
From: Jacques Durand
Sent: Monday, October 21, 2002 4:48 PM
To: 'Frank. Christopher'
Cc: Jacques Durand
Subject: RE: IIC specs

Hi Christopher:

 

I'll forward your comments to the IIC team.

See inline comments.

 

We are (in IIC) definitely interested in keeing in touch with you and eBES,

and in getting such feedback.

 

Cheers,

 

Jacques Durand

 

-----Original Message-----
From: Frank. Christopher [mailto:C.Frank@seeburger.de]
Sent: Thursday, October 17, 2002 2:00 AM
To: Jacques Durand
Subject: AW: IIC specs

Hello Jacques,

 

Thanks for the quick respond.

 

I assume that we are able to decide on the core structure of our test spec this week (I wrote a proposed draft therefore).

 

My personal comment after studying the IIC specs (req, test framework, MS conformance, MS interop):

In general while reading the IIC drafts I had several issues when trying to relate the spec content to real-life test implementation (therefore I asked for the physical sources).

One of the basic targets of the test framework is to not request too much effort on implmenting the test framework on a physical system (where I agree a 100%;-). Reading the service description and its very detailed action description, I doubt that can come to truth (depending on the ebMS implementation out there, of course).

But this is may because I am nitpicking.

 

I tried to be more "tolerant" when interpreting the Actions of the test service (which I see as the major physical impact), what I found is this: Interpreting the Actions as "activity description" (thereby ignoring several details in the Actions description), I found them very, very useful for defining Test Cases.

 

Use this as an example (where all "activity" marked words are representing a activity (actually sometimes a test framework action and sometimes a core ebMS functionality):

 

TestCase n_00

(The table is to read from left to right, top to bottom and is then equal to the time-line while the Requestor and Responder work independently.)

 

Step    Requestor                                                Responder

1        "Send" ebXML20(mpld_00)                       

2        "TimeoutVerification"                                 "Receive" ebXML20(mpld_00)

3                                                                       "Reflector"

4                                                                       "Send" ebXML20(mpld_00)

5        "Receive" ebXML20(mpld_00)                    "TimeoutVerification"

6        "PayloadVerify"

 

<Jacques> we could indeed use such an activity diagram (or "swim line") that shows events on each side,

if only for more intuitive description of the test case (I think we should.)

When it comes to a more formal description, because the test case is entirely controlled by the Test Driver

(and because we only want to trace it in the Test Driver only, for convenience), we define it uniquely in terms

of sequence of events that occur in the test Driver (which will be, say, on the Requestor side in your example).

What is happening on the other side, is only a "reaction" to such events (e.g. an Action is triggered

that sends back an event), so we only observe its effect on the Test Driver (e.g. a messsage has been received

or not, or is corrupted, etc.). The test outcome will be decided on these observable "effects".

 

those test cases then could be easily extended (or adapted) by doing:

 

5        "Receive" ebXML20(mpld_00)                    "TimeoutVerification"

6        "PayloadVerify"

7        "Send" ebXML20(%error%)

8        "TimeoutVerification"                                 "Receive" ebXML20(%error%)

 

I would actually not care (or define) "how" Requestor or Responder perform the "Reflector" (as you in your current spec do not care (for good reason) "how" they perform a "send" or how they "parse" a ebXML message. As long as the rules and conditions of the "Reflector" activity are completly desribed (of course) . Where I found the test services "action" description too implementation-oriented (and I am saying this as technician;-).

 

<Jacques> we certainly do not intend to go further in implementation details...

while we do not specify how the Action code should be implemented, it is true

that our description is detailed about a few things, notably:

- the actual content of the messages that the action generates.

This is because in the test suites that use these Actions, there will be formal descriptions of

test Conditions that will make precise assumptions on the content of the message they generate.

- notification aspects (this specifies what / when the Test Driver should get notified.

(Was that the kind of details you were concerend about?)

 

Additionally (now speaking for the Interop Test spec only):

With a few years of EDI/B2B experience, I would strongly recommend to extend the Test cases (or flavors of them) to define "stress" testing in regard to "a-lot-messages-at-one-time" and "large-messages" (and both in combination, of course;-)

 

<Jacques> We are fully aware that this kind of scalability testing is also required...

We will look into this after we are done with the "basics" of interoperability, and

would then welcome your experience to help in this..

By end of this month, we plan to wrap-up a more complete draft of our "basic interoperability profiles", that should

only be considered as the baseline for ebXML interoperability - but one that all users could agree with,

independently from more specific requirements they might have in addition. The idea is that more advanced

"interoperability profiles" will be added after that (each of these concretized by an interop test suite - or

a sequence of test cases) .

 

Hope that gives you an idea of my impressions and you may find it useful as input to your current work.

 

I will keep you posted on the result of the discussions in the European pilot.

<Jacques> Yes please. We'd like to know what are your plans and milestones for this pilot.

 

Regards,

 

Jacques

 

Thanks again.

 

Best Regards,

Christopher

 

-----Ursprüngliche Nachricht-----
Von: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Gesendet: Mittwoch, 16. Oktober 2002 01:28
An: Frank. Christopher
Cc: Jacques Durand
Betreff: RE: IIC specs

Hi Christopher:

Thank you for your attention.
Yes, some of our members are planning to implement
a reference implementation of the Test Service, as
well as of the Test Driver.
We are lining up resources for this - while still working
at completing our test specs - as there is definitely demand
for such tools in B2B user forums, not only in US (OAG, NIST...),
but Asia (ECOM) and Europe as well.

As ebXML IIC will support and advise on the deployment
of its test architecture, we are naturally interested in learning
more about your specific requirements, and project plans.
So we could remain in touch.

Also, we are currently working on our MS Interoperability test suite,
and should have an updated draft by end of October - so please understand
 that the current draft is not stable yet.

Best Regards,

jacques Durand
ebXML IIC Chair

[Jacques Durand] -----------------------------------------------

 

The CEN eBES and OASIS started a "European Interoperability Pilot Project"
(you may are already aware of). I am member of the Messaging Team currently
putting together a core spec for our pilot project.

Our pilot will focus on a real-life business case implementation, but we are
trying to refer to the work of the IIC (especially the ebMS interop spec) as
much as possible.

After studying the IIC specs trying to determine the agreements and
disagreements to the European pilot, I was wondering, whether you have or
plan to have any physical implementations of the TestService
(ebXML_IIC_Testing) (and its actions) as desribed in the TestFramework spec?

Best Regards,
Christopher Frank
(SD Research/Protoyping)

--
SEEBURGER AG, Edisonstrasse 1, D-75015 Bretten, Germany
Fon:+49(0)7252 96-1185 Fax:+49(0)7252 96-2400 <http://www.seeburger.de/>