OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic-conform message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Subject: Re: [ebxml-iic-conform] comments on latest issues


To all,

     Attached are my comments to your suggestions/questions/comments 
regarding the ebXML MS Conformance Test Requirements and
Annotated MS Specification documents.
      I've incorporated those suggestions into changes in both of those 
documents, and have now posted them for a vote this week.
Attached are "my comments to your comments", reflecting those changes.

Thanks for all of the great input,
Mike

PS- Changes can continue to be made this week as we move toward
resolution in voting by Friday.

Monica,

  I went through Eric's comments, and changed/modified accordingly.
I came across some more potential "spec/requiremnet gap" items that can be
added to the list to submit to the MS TC.

  They are:

urn:semreq:id:10 - Spec states: If provided, the MIME charset attribute MUST NOT contain a value conflicting with the encoding used when creating the SOAP Message.. 
yet says both must have same value. So UTF-8 MIME charset and ASCII SOAP encoding is illegal?

urn:semreq:id:145 - Very vague specification section... "The ultimate destination of the error message is informed of the failure by some undefined means."

urn:semreq:id:154 - Spec says SequenceNumber status attribute SHALL be there, but XML Schema says that it does not have to be present. Is this a schema design error?


At 02:35 PM 8/1/2002 -0700, Monica Martin wrote:
-----------------------------------------------
urn:semreq:id:37
What determines if a conversationID is out of context ?
The CPPA 2.0 lists concurrentConversations and invocationLimit
is this what the test requirement aludes to ?
Also what is the error to generate ? (The spec doesn't say the error
code to
use ?)
 

[MIKE] - See comments below.



urn:semreq:id:155
It implies there should be an error if this is not the case, but the
spec
doesn't say it.
It would seem the receiving MSH would throw back an error - like illegal
reset or hold the communication open until it's clear to reset.

I think the test requirement covers the specification as written- but I
think the specification missed something here.
---------------------------------------------


[MIKE] - See my comments below.

 
Michael,
Need to be added to ebMS TC gap questions...do you want me to do so and
resend?
Thank you.




[MIKE] - Outside of the two requirements above,  what I see below however are
fixable typos, naming, grammar, sequencing problems.  I will fix them this
weekend, and re-submit to the OASIS webmaster on Monday.
 
 

	-----Original Message----- 
	From: Eric VanLydegraf 
	Sent: Mon 7/29/2002 10:49 PM 
	To: 'Michael Kass'; ebxml-iic@lists.oasis-open.org;
ebxml-iic-interop@lists.oasis-open.org;
ebxml-iic-conform@lists.oasis-open.org 
	Cc: 
	Subject: [ebxml-iic-conform] TestReqs Comments
	
	

	Well as many times as I have read the ebMS document, it seems
you truly
	never know it.
	I have to commend the authors on the annotated ebMS spec for the
thorough
	highlighting and
	the test requirements generated from it. Translating
specification speak to
	actual test requirements is not always obvious.
	
	Upon review here's some comments, questions and corrections I've
found.
	
	Page 3 (Test Requirements Doc)
	Table Fields
	...
	"Name: A unique name for the Test Requirement."
	
	these ids have identical names
	73,74
	158,159,160
	163,164,165
	

[MIKE] - Done

	==================================
	Refering to
	urn:semreq:id:10
	Assertion
	They shall have the same value
	
	annotated_ebms_v2_0.doc
	576-578
	If provided, the MIME charset attribute MUST NOT contain a value
conflicting
	with the encoding used when creating the SOAP Message
	
	My understanding of the specification is that you can list
different value
	encodings, as long as the encodings are compatible (I don't
think they're
	are many such possibilities, but the spec authors didn't
disallow it)
	
	If an implementation were to have different encoding values but
one is a
	compatible subset of the other (e.g. UTF-8 and ASCII). How would
this test
	requirement play out in the testing driver. According to the
assertion they
	would fail, but they would be in compliance with the spec ?

[MIKE] - This statement in the spec seems to contradict requirement #10, which states that
the MIME charset value shall be equivalent to the SOAP encoding value.  So it would seem that a 
UTF-8/ASCII combination would be illegal.... even though this spec assertion seems to 
indicate that such a combination would be OK, as long as they don't "conflict". 
This requirement should be added to the "spec/requirement gap list".



	
	Refering to
	urn:semreq:id:18
	Has the same situation
	
	Would there be any value in just making one encoding test
requirement,
	covering
	MIME Header and XML Prolog ? It seems these 2 test requirements
are covering
	the same situation. Just spelled out differently.

[MIKE] - I read #10 and #18 the same way. I think that we only need one here.
I am going to delete #18.



	-----------------------------------------------
	urn:semreq:id:37
	What determines if a conversationID is out of context ?

The only way to call a conversationID "out of context" would be if it is 
has no context relevant to any CPAId.

	The CPPA 2.0 lists concurrentConversations and invocationLimit
	is this what the test requirement aludes to ?
	Also what is the error to generate ? (The spec doesn't say the
error code to
	use ?)

[MIKE] - This is another spec ambiguity that needs to be better defined.
Also, there is no specified error to be generated if a ConversationID is
determined to be out of context.
I think that this requirement should be removed from the list, since it
has no real reference in the specification.

	
	------------------------------------------------
	Would it provide value to list test requirement dependencies ?
	e.g.
	urn:semreq:id:79
	  ^--- urn:semreq:id:77
	      ^--- urn:semreq:id:78
	
	They struck me as building up from each other. So you could say
	urn:semreq:id:78 is Ack was not received plus (urn:semreq:id:77
	testrequirement)
	...
	It would also give you an escalting test scenario in the test
driver itself
	or reuse of test artifacts.

[MIKE] - Monica brought up this question in another post.  I believe that this
is something we can discuss at the F2F... enhancing the test harness to incorporate
dependencies between test cases.

	-----------------------------------------------
	urn:semreq:id:104
	May want to spell it that Reference is "XMLDsig Reference"
element and not
	"ebXML Manifest Reference" element.
	
	urn:semreq:id:105 is more clear concerning the reference -
another
	possibility is to switch 104 & 105 so the XML Signature is
listed first.

[MIKE] - I followed your first suggestion, and added the "XMLDsig" clarification.

	---------------------------------------------
	urn:semreq:id:110
	Suggestion:
	The DuplicateElimination element [must|is] not [be|] present.

[MIKE] - Done

	---------------------------------------------
	urn:semreq:id:130
	PreCondition:
	[Suggestion for clarity - the phrasing was little hard to
follow]
	(For each reliably received message, having a SOAP actor URI
targeting the
	receiving MSH and the AckRequested element is present)
	Assertion:
	[Correction]
	fvalue => value
	MessageIde => MessageId
	[Suggestion]
	An Acknowledgement Message is generated with RefToMessageId
having the
	MessageId
	value of the message being acknowledged.


[MIKE] - Done
	--------------------------------------------------------
	
	urn:semreq:id:137 & 138
	(dangling quote)
	I think you meant to enclose "original message"
	138: "Acknowledgement Message"
	or just remove the end quote

[MIKE] - Done

	--------------------------------------------------------
	urn:semreq:id:141
	PreCondition:
	[... AND the syncReplyMode is [not] set to none ...]


[MIKE] - Done

	-----------------------------------------------------
	urn:semreq:id:142
	PreCondition:
	{this must have been a tough one to decipher - back to boolean
algebra :&)
	[... AND [if] the syncReplyMode is [not] set to none and the CPA
indicates
	that an application response is NOT to be included]



[MIKE] - Done

	-----------------------------------------------------
	urn:semreq:id:145
	remove this one ?
	
	I wonder if we should include this one - while the spec says it
-
	it has no meaning for testing purposes. I mean hell I could
fedex you a
	error log and that would satisfy the specification requirement.
Doesn't
	really offer any value - this is more like a documentation item
for say
	implementation guidelines.



[MIKE] - We will leave it in, but annotate the specification with a coverage of
"none", due to spec ambiguity. This will be added to the list of spec/requirement
gaps.


	------------------------------------------------------
	urn:semreq:id:150
	dangling quote
	------------------------
	urn:semreq:id:150-151 should be combined (they are both required
together)


[MIKE] - I agree that the spec indicates that the "status" attribute is required
for requirements #150 and #151.  They are only "split" for now because it is not clear
whether the test harness will support aggregate assertions.  If so, then 150 and 151 will
be merged.

	urn:semreq:id:152 (is there a need to differentiate first
message versus
	reset
	as far as the assertion (status="reset" and sequenceNumber="0" )
they are
	the same test.

	
	seems like all 3 could be just one testrequirement - I don't see
any reason
	to make the semantic difference highlighted when the test is
identical.

[MIKE] - #151 tests the case of a "first message" in a conversation, while
#151 tests the scenario where sequence number is reset in a conversation.  
The spec sees these as semantically different cases, and so we wrote them
that way.



----------------------------------------------------------------------
	urn:semreq:id:153
	Remove this requirement

	
	I understand ebMS2#9.1.1 that status="reset" is with the
sequenceNumber="0"
	only in cases 1 & 2 and not the numbers following it. And thus
is the same
	as 152.
	
[MIKE] - You are right, that the status is not "Reset" in this case, but
should have a value of "Continue", along with a sequenceNumber of "0"
according to the spec.  So I will modify this requirement to reflect that.

	------------------------------------------
	urn:semreq:id:154
	Assertion:
	[... AND attribute status="Continue" (if attribute status is
present?)]


[MIKE] - This is a case where the ebXML schema says one thing, while the semantics
of the specification says another.  As you mentioned in the case of #150 and #151,
the spec requires both the sequenceNumber and status attribute to be present... and
uses the word SHALL when referring to the status value, but the schema says that the
status attribute is optional.  When I see it explicitly spelled out in the semantic
text of the spec, I give the specification wording precedence over a schema document, 
and bring it to the attention of the MS TC that their schema does not enforce the wording 
of the specification. This is another item for the "spec/requirement gap" list.


	-----------------------------------------
	urn:semreq:id:155
	It implies there should be an error if this is not the case, but
the spec
	doesn't say it.
	It would seem the receiving MSH would throw back an error - like
illegal
	reset or hold the communication open until it's clear to reset.
	
	I think the test requirement covers the specification as
written- but I
	think the specification missed something here.


[MIKE] - This requirement examines the state of the MSH prior to sending a Message with a
SequenceNumber attribute of "Reset".  I do not see where there is an implied "Error" message
to be sent if all previously sent messages have not been accounted for.  The specification
does not indicate any such behavior.

	---------------------------------------------
	
	urn:semreq:id:186,187,188,189,190,192,193,194,195
	[PreCondition]
	an party =>  a party
	
	188,189,190, 192,193,194,195,198 (redundant "the the")
	
	191 messaged => message

[MIKE] - Done
	
	---------------------------
	Couldn't urn:semreq:id:196 be a generic  test requirement that
all optional
	features that are unsupported
	must prove this assertion ?

[MIKE] - We can write one if the specification states that all optional features that are unsupported 
will return an Error element with an errorCode of "NotSupported" and a highestSeverity attribute set to "Error".
I could not find that in the specification.
	
	-----------------
	urn:semreq:id:206
	GenerateAckBasedBasedOnActor
	
	Should Based be repeated ?
	
[MIKE] - Thanks. Fixed.

	------------------------------
	
	That's all I found on this pass - phew - lots of stuff here ;)
	
	
	----------------------------------------------------------------
	To subscribe or unsubscribe from this elist use the subscription
	manager: < http://lists.oasis-open.org/ob/adm.pl>
	

Attachment: CT_issues_Monica.doc
Description: MS-Word document



Reqts Level 2
urn:semreq:id:2 - Separate two conditions (1) MIME/multipart and text/xml

[MIKE] - Agreed.  Done

urn:semreq:id:15 - Separate assertion - may need to be two requirements tested individually to determine the 
unrecognized MIME headers are ignored and that no error message is sent.

[MIKE] - Agreed.   Will split this into 2 if no assertion aggregation is permitted.
[MIKE] - Agreed.  Done

General: With the sequential numbering of conditions you are not able to see - other than in the XML which
conditions go with which semantic requirement. 

[MIKE] - Agreed, will ask Matt if he feels this is significant issue for test harness.  I agree it is textually
confusing not being able to associate conditions with a parent requirement id.   
Then again, conditions can be "re-used" via reference in other requirements 
( some are, although not currently done by reference to id, they are just "repeated").
 
urn:semreq:id:41 - Couldn't this be tested by a profile where you actually link a series of discrete test cases?

[MIKE] - Yes, I think that you could do it this way.  I hope we can explore these options when we start building 
the test harness.  This could add more power and flexibility to testing.

urn:semreq:id:47 - misspelled word in condition 54 - ff not if.

[MIKE] - Done

urn:semreq:id:43 - misspelled word in condition 49 - and not an.

[MIKE] - Done

urn:semreq:id:64 - erroneous character in condition 71 and 72 - ·

[MIKE] - Done

	
urn:semreq:id:13 - erroneous character in condition 15 - manifest᾿ 


[MIKE] - Done


urn:semreq:id:65, 69 - Can't we at least partially test by logging the error? Regardless of the other actions - resolve
by other means, etc?


[MIKE] - I think that this will be determined by how much logging capability will be defined in the testing service. To 
be determined by Jacques.

urn:semreq:id:75 - misspelled word in assertion - behaveas not behaves.

[MIKE] - Done

urn:semreq:id:77, 79, 82 - Shouldn't this assertion be broken into two parts?
(1) The message is kept in persistent storage
and (2) processsed as if the interruption had not occured.

[MIKE] - Done

Occurred is misspelled.

[MIKE] - Done

urn:semreq:id:91 - How are we verifying it is consistent with the CPA in the conditions?
This would apply on any like requirements regardless of 1,2 or 3, and conditions.

[MIKE] - As I understand it, the CPA ( or CPA-like document ) will be available for comparison ( XPath evaluation ) to
check consistency of a message with CPA configuration

urn:semreq:id:94 - Shouldn't the assertion be broken into discrete test cases?
(1) The MSH generates an Error of type Inconsistent
and (2) severity = Warning

[MIKE] - Yes, I think that it should. I Will split this. Will split if no assertion aggregation is permitted
[MIKE] - Agreed.  Done

urn:semreq:id:105 - Same

(1) Any Reference elements included in a generated Acknowledgment element are namespace
qualified to the XML Signature namespace
and (2) conform to the XML Signature specification.

[MIKE] -  Agreed.  Will split it into 2 requirements if no assertion aggregation is permitted.


urn:semreq:id:123 - condition 154 - How are we verifying what happens in application?
Reference "the message is presented once and only once to the application"

[MIKE] - I will have to defer to Jacques and Matt regarding what the test harness will and will
not be able to do regarding logging such events.


urn:semreq:id:130 - misspelled word in assertion "fvalue as the MessageIde"

[MIKE] - Done

urn:semreq:id:152 - Shouldn't the assertion be broken into discrete test cases?


(1) The SequenceNumber element has value of 0
and (2) the status attribute of the message is set to Reset.

[MIKE] - Agreed. Will split into 2 requirements if no assertion aggregation is permitted.

urn:semreq:id:170 - Shouldn't the assertion be broken into discrete test cases?
(1) The generated XML Signature Reference element includes a child Transform element
and (2) which in turn includes a first Transform element with an Algorithm attribute of value
"http://www.w3.org/2000/09/xmldsig#enveloped-signature";.

[MIKE] - Agreed. Will split into 2 requirements if no assertion aggregation is permitted.

urn:semreq:id:187 - Shouldn't the assertion be broken into discrete test cases?
(1) In the returned StatuResponse element,  the RefToMessageId element child of the MessageData element specifies the MessageId of the message containing the associated StatusRequest element. 
and (2) In addition the RefToMessageId element child of the StatusResponse elements always contains the MessageId of the message whose status is being queried.

[MIKE] - Agreed. Will split into 2 requirements if no assertion aggregation is permitted.

urn:semreq:id:190 - Did we resolve if we understand what unauthorised means?
Reference in condition 258 - "the message is received from an party deemed to be unauthorised"
I ask because we had a query back to TC and we designate this requirement as full.

[MIKE] - This does sound very hard to test "fully", or even "partially", since it seems very
application dependent.  I would recommend changing this requirement to "none", unless Jacques can 
explain how authorization will be determined through the test service.






[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Powered by eList eXpress LLC