RE: [ebxml-iic-msg] MS Conformance checklist: comments

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: RE: [ebxml-iic-msg] MS Conformance checklist: comments

Matt:

>That is the plan.  We have one person devoted to doing this stuff,
>including some test case definition for these conformance issues, and I
>hope to donate this stuff where possible to kickstart our work here.

Great. Once we have this in our "test req" format, we can post it
on our site for review.

>Why not just have a library of ebXML Message templates that we want to
>send, and use a simple preprocessor and a tool like netcat to send the
>messages to the MSH?  I don't think we care to test a candidate MSH's
>client side API, just the behaviour of the MSH itself.

That seems fine for sending test messages to the candidate MSH,
("send-and-receive" test scenario)
but in order to fully test the behavior of the MSH, don't you think we need also to
drive it, in some cases, from the app side (how are we going to get the candidate
MSH generate the kind of message we need to check? )

>We were thinking of defining a journal or log format that candidate
>msh's would write during conformance testing.  We would define the
>format of the log, and all of the points where a log must be written. 
>There would also be a Conformance service that the MSH would implement
>allowing the conformance test suite to "download" the journal after a
>test run. e.g.

<journal start="2002-8-6T12:43:45">
        <entry time="2002-8-6T12:43:47" type="msg_recv">
        <![CDATA[ message here ]]>
        </entry>

        <entry time="2002-8-6T12:44:00" type="ack_sent">
        <![CDATA[ ack here ]]>
        </entry>
</journal>

I agree that would make our life much easier, as conformance testers,
but my concern is that we cannot require implementors to produce such a log:
that is not required by the standard (we may want to make it an implementation
guideline item - but even so, our test procedure should be able to handle
MSHs that do not write such logs...)
Other MSH implementors may have strong opinions about what kind of log
they want to produce. 

>> 4. We may have a similar approach for defining the CPA under which a
>> test is to be
>> performed (i.e. we define an XML format that encodes the subset of CPA
>> needed to
>> "configure" the MSH).

>A simple property file will likely suffice.

Or we could reuse a CPA XML doc itself (and extract what we need)
so we do not have ot define a format that I was suggesting above.
I'm concerned that a property file may be too much implementation-specific.
(same issue as the log trace above.)

>The message data format will certainly be useful, and could be
>integrated with the requirements definition, however, you must remember
>that the requirements document simply states the requirements, not
>necessarily give you the details of how to test compliance to the
>requirements. 

That was just a suggestion - you are right it is better not to overload
this req doc with "operation" stuff.

>Nonetheless, being able to pass this kind of data onto
>the test framework is probably very useful.  I'll let Michael weigh in
>here, as he is definitely the expert on this sort of thing.

That may be passed in other ways indeed.

Regards,

jacques

----------------------------------------------------------------
To subscribe or unsubscribe from this elist use the subscription
manager: <http://lists.oasis-open.org/ob/adm.pl>