OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic-framework message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Subject: [ebxml-iic-framework] Test Framework Comments


Well I feel like I'm jumping in the middle of a herd of racing elephants - lots of changes guys and tracking them down and resolving is challenging for me.
But hopefully my comments will apply or have been take care of and my confusions easy to clarify :)
 
I will say good job on working on so many fronts of the document in rapid sucession. At some point we need to roll back up to the overall document and verify all the sections work together. I think once it's all folded in and change tracking removed it will look quite nice.
 
I've just read over and reviewed Version 0.92 (Jan 20, 2002) - apologies if I've missed updates since.
 
Some overall comments:
 
To driver-mode or non-driver mode ?
----------------------------------------------------
The "driver-mode" for Test Driver and Test Service is confusing to me.
 
Test Service Driver Mode - whether it notifies the Test Driver of test events or not is clear enough and sound to me.
 
Test Driver - driver mode is confusing - since you would have a TestService in driver mode and could have a Test Driver in non-driver mode.
I suggest using conformance mode and interop mode instead and make no mention of driver/non-driver mode for the Test Driver. The role of the test driver is clearer
and we don't re-use the drive mode term from the Test Service which has a quite different semantic meaning.
 
Though I'm still trying to resolve this with Connection and Service mode - I don't know if they are the same thing or orthogonal and speak more of the communication between Test Driver and Test Service rather the type of testing being done.
 
Are we missing a section ?
----------------------------------------
We talk about General Methodology and we talk about Test Framework components and we talk about Test Suite chemas and instances but we have no description
of the runtime of the testing.
 
What I mean is how to trigger the testing to begin with - we have figure 4 we show a "Control Interface" but I can't find a description for it.
In other sections we talk about Configuration messages and Initiator messages but we don't described how that gets called. I guess I'm looking for the ignition key to the this whole engine.
 
 
Should we break up the actions and messages into 2 channels - driver control versus testing ?
=====================================================================
It seems to me there are 2 channels in the test framework.
Control: trigger, configuration, test result reporting, notifications, .. etc
Test: messages for the purpose of asserting test requirements.
 
 
++++++++++++++++++++++++++++++++++++++++++
No onto the nit picky stuff :)
 
Header: date says April 2002 (not matching document date)
 
101 - I think we should come up with a URI placeholder - so when we do an errata sheet the first thing we don't need to do is update this document :D (Catch-22)
we can always have a page at the URL that states no errata has been created yet but in the future one could be.
 
194 - brackets around the MIME,XML,SOAP
I believe SOAP with Attachements is usally written [SOAPATTACH]
 
Page 10,11 (367-393) all end with periods or none (consistency)
 
466-473 all the bullets are related to 464-465 - either indent them one more tab or change 464-465 into a non-bullet phrase so the rest match up to it.
 
Page 17 (Fig 5)suggest breaking them up into Fig 5a 5b or 5 & 6 and then refering them by that in 499--508
 
will make it a lot easier to read - I also suggest moving those 2 figures above the paragraphs that describe them.
 
545-551 hints a triggering from a message but not how the message itself is sent. (goes to my overall point above)
 
612 elts => elements
 
671-675 verificationstatus implies that the Action is doing the verifying -  where really the Test Driver will be doing that - so really it just needs to send the error it trapped back to the test driver. The Driver will know if that is correct or not.
How about <testDriverMessage> or <testNotification> or <actionResult>
 
same for 697-701, 727-734
 
829 level and expected to be general enough to [allow configuration of] most implementations, ...
 
907 fig number is out of sequence 4 should be 7, fix all subsequent figures
 
1032-1033 Delete [The ebXML MS Conformance Test Requirements instance file can be found in Appendix E]
Reason: Appendix E is firstly Test Report Schema and secondly the Test requirements is in a separate doc
 
1156 This document will drive the Test Harness ...
reason: fix grammar
 
1186 a Test Case are either large XML files or non-XML message payloads.
 
1194-1224 missing figure number
 
1258 ... in a Test Case MUST evaluate to "true" for a Test Case result to "pass".
 
1301-1304 Test Driver as driver/non-driver with interop and conformance in paranethesis - seems clearerd if we get rid of dirver/non-driver terms
 
1351 delete "the" between "parsing" and "a"[after parsing a simple Message Declaration]
 
1626 - 1640 XML attributes are Uper Cased (e.g. Id, Type, Algorithm) which is different than used every where else.
suggest consistent UpperCase for Elements, camel (sp?) lowerCase first Word upper Case remaining for attributes
 
1800 Content-ID change to ContentId
        Content-Location ContentLocation
keep it consistent with previous Id usage
remove dashes to match MimeHeader and in general XML elements
 
1806 Dsign => DSign
 
DSign needs attributes camelCased as well
 
1948 requirement
An enumerated list of requirement types of either MUST or MAY
 
2118 - 2135 CPPA=> CPA
 
we aren't talking the specification we are talking the actual document
 
btw what is [BASECPPA] ?
 
3578 [ebTESTREQ] refers to CPPA spec ?
3584 [ebTESTSUITE] refers to BPSS spec ? 
 
3607 [ebCPP] = either [CPPA] or if we stick with [ebCPPA] style than we have to search and replace [CPPA] in document and use the reference type here - I guess the prefered way is a good ebXML JC question eh ?
 
Good job folks
 
Now onto the latest interop doc :D


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Powered by eList eXpress LLC