Re: [ebxml-iic-msg] MS Conformance checklist: comments

[Matthew MacKenzie <matt@xmlglobal.com>]

On Friday, March 29, 2002, at 05:01  PM, Jacques Durand wrote:

> Matt, Michael:
>
> Looks like the "test requirement" material is almost finalized.
> You seem also to have detailed ideas on how best execute the test items.
>
> Would it be possible, by end of next week, to send to the group
> (in preparation to April review):
>
> (a)- a "readable" list of proposed test items for MS conformance
> (I guess pretty close to what XMLG has sent out)

I worked on this for a couple hours today, so an initial list based on 
the items donated by XMLG will be sent to the list by the end of the 
week.  Its a big data entry job, basically translating the XMLG list 
into the more useful Requirements XML format.  The good news is that 
Michael has been developing a stylesheet along with the schema, which 
does a good job of making a readable list of the requirements.

> (b)- the subset of these items for "core features" (Level 1), cast into 
> your XML format.

See above.  The XMLG list contains things beyond the level 1 scope, but 
I am classifying them all as level one so that I can get something out 
faster.  What I will do is create a level two when I am done and move 
the appropriate items into it later on.

> (c)- some overall design of the testing procedure(s) and scenarios that 
> would be used to
> execute these test items?

Michael?

>
> Matt: just some comments on your comments:
>
> >I think a "Conformance service" is probably a better way to request 
> that
> >the MSH create messages and that sort of thing to be tested.  It seems
> >impractical to me to have custom code for each vendor in the 
> conformance
> >test suite.
>
> If vendors accept to implement this conformance service,
> that will indeed be the shortest path to conformance testing
> (and also to interoperability test later). So it seems like something
> to explore further. (but we need to publish this design ASAP in Impl.
> Guidelines, if we push for it, so that implementors are aware of it.)
> I assume then that vendors who do not want/plan to implement this
> "conformance service" will still have the option to "drive" their MSH 
> through
> its API, for test scenarios - their choice?
>

Ok, I will give this some thought and send a note to the chap doing the 
impl guidelines.


> >I'm not suggesting we require vendors to adopt a certain log format,
> >what I am referring to here is more of a tracing format.  In the 
> absence
> >of a reference implementation, we need to do this, or employ network
> >sniffing.  Maybe I have my head in the mud, please set me straight if I
> >am missing an easier way to capture a complete exchange/transaction.
>
> What I had in mind is: the candidate MSH could send its messages to an 
> URL that
> would just happen to be our testing end-point listening to this HTTP 
> port
> (not even an MSH - just a servlet able to dig in the MIME envelope and
> analyze themessage parts).
> Again that may not be sufficient - we need to look at all your test 
> cases.
> So we may all have heads in the mud until that's done :)

I am going to deflect this to Michael, as he probably has a better idea 
of how the tests will be implemented.

>
>
> >Either way, it doesn't really matter.  Extracting from an actual CPA is
> >a breeze.
>
> I would favor that, as again there might not be agreement on a common 
> config file format
> across implementations (some MSH could actually directly use/fetch CPA 
> docs, if I recall).
> (also, CPA data will actually be more dynamic than config files: new 
> CPAs may
> show up any time - and expire - during the liefetime of an MSH 
> instance, based on new
> collaborations)

Maybe something simple that uses XPath would be in order.  That way, 
changes to CPP/A are easy to handle.

Regards, and Happy Easter.

--
Matthew MacKenzie
XML Global R&D
PGP Key available upon request.