OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic-conform message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Subject: CTTF 7/10/2002: [ebxml-iic-conform] MS Level 2 & 3 test reqs


Just fine...as long as the rigor and criteria are understood.  Thanks,
Michael.
 
Monica

	-----Original Message----- 
	From: Michael Kass 
	Sent: Wed 7/10/2002 9:43 PM 
	To: Monica Martin; Matthew MacKenzie; Jacques Durand 
	Cc: ebxml-iic-conform@lists.oasis-open.org; Monica Martin 
	Subject: Re: CTTF 7/10/2002: [ebxml-iic-conform] MS Level 2 & 3
test reqs
	
	

	Monica and all,
	
	   I think that the "atomic" approach will be the right approach
until it
	is determined
	whether or not the test harness will incorporate the reporting
of results
	for aggregated
	Assertions.  Right now, it is not clear that it will do that.
If it does,
	then the example below could be
	put into a single test Assertion, and the harness would report
failure for
	either A or B  or both.
	  However, as Jacques mentioned, some test Assertions, by their
nature will
	be germane/generic/broad-scoped,
	and the rigor to test them at the "atomic" level is impractical.
	   I think that the requirement level ( REQUIRED,
OPTIONAL...etc.. ) is
	less important as a criteria
	for aggregating an Assertion.   I believe that the reporting
capability of
	the test harness and the
	rigor involved in splitting an Assertion into parts should be
the criteria
	that we use right now.   So my vote is
	to split the ( non-rigorous) requirements until it is shown that
we can
	support aggregation, and aggregated
	test reporting.
	
	Regards,
	Mike
	
	At 04:23 PM 7/10/2002 -0700, Monica Martin wrote:
	>Everyone,
	>Three cheers for the team!!!
	>
	>As for Jacques feedback on combining/segregating assertions, I
can live
	>with either way we go, but think we have to have a few
parameters that
	>tell us whether we aggregate the assertion or break into
separate test
	>cases - some has to do with the rigor required of the test.
Perhaps the
	>discriminator is a REQUIRED test requirement should not be
aggregated
	>(not advocating, just citing an example) or aggregation only
occurs if
	>the two elements and their result can be seen or evaluated.
	>
	>Mike, Matt, and Jacques are the experts.  I am just the lacky
(sp?).
	>
	>Monica
	>
	
>-----------------------------------------------------------------------
-
	>---------------
	>r2.2.6a: re-wording: (consolidated with r2.2.7a)
	>[precondition]: sending message with MessageOrder element,
which is
	>first for a
	>conversation ID
	>[assertion]: REQUIRED: the sequenceNumber element must have
value 0,
	>and its Status attribute = "Reset".
	>[MIKE] - DONE
	>
	
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
	>++++++++++++++++
	>[mm1: IN THIS CASE (AND OTHERS), DO WE ACTUALLY HAVE TWO
ASSERTIONS,
	>JUST LIKE WE HAVE
	>0...N CONDITIONS/PRECONDITIONS - CAN WE ACCOMMODATE THIS - AND
WITH A
	>RESULT TO TRUE OR FALSE?]
	
>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
	>++++++++++++++++
	>[MIKE3] - This is always the problem with "consolidating"...
you "group"
	>assertions ( and expected results )
	>together, so if there is a failure in one part of the
Assertion, you
	>fail the entire test.. So unless the test harness provides
	>a way to uncover exactly which part fails ( the sequence number
not
	>equal to "0" or the status not equal to "reset" )
	>you cannot tell an implementer why their implementation failed,
only
	>that it failed one or the other Assertion.
	>I am going to "decouple" the sequence number from the status,
and split
	>each test into two tests. It is less
	>"pretty", but it removes any ambiguity. We have to be
consistent on
	>this, and I haven't been in all cases.
	
>-----------------------------------------------------------------------
-
	>-------------
	>
	>
	>
	>         -----Original Message-----
	>         From: Matthew MacKenzie
	>         Sent: Wed 7/10/2002 5:17 PM
	>         To: Jacques Durand
	>         Cc: 'ebxml-iic-conform@lists.oasis-open.org'
	>         Subject: Re: [ebxml-iic-conform] MS Level 2 & 3 test
reqs
	>
	>
	>
	>         Jacques,
	>
	>         I would be disappointed if the requirements didn't
change after
	>my
	>         initial submission as the reason I submitted them was
to get
	>them by a
	>         wider group of pros.  You guys have done an awesome
job on the
	>editorial
	>         side.  It is quite gratifying to see such a large
chunk of work
	>come
	>         together in such a short time, eh?!
	>
	>         I have to apologize for not being as "gung ho" on the
work items
	>the
	>         past couple of weeks.  I've been juggling this with a
proposal
	>I'm doing
	>         for Regrep 3 which has a deadline similar to this one,
only with
	>less
	>         participation.  I'll make it up by doing all of the
nasty
	>merging work
	>         that needs to happen at the end of the process.  Maybe
I'll
	>cheat and
	>         use my company's transformation tool :-)
	>
	>         Mike: let me know when you consider the CVS copy
golden.  I'll
	>get to
	>         work right away.
	>
	>         -Matt
	>
	>         On Wednesday, July 10, 2002, at 11:06  AM, Jacques
Durand wrote:
	>
	>         > Of course, ebXMLGlobal (Matt) contribution to the MS
Test Reqs
	>should
	>         > not be forgotten:
	>         > Matt provided the first cut at a list of Test
Requirements,
	>hope he
	>         > does not mind the follow-up massaging :)
	>         >
	>         > Once the test requirements are finalized, that is
good enough
	>for MS TC
	>         > release,
	>         > but not yet for an MS Conformance test Suite: we
will need to
	>add the
	>         > Test Cases definitions, and profiles definitions...
	>         > I think that for Version 0.1 (internal TC review) we
don't
	>need all
	>         > test cases: just a representative sample.
	>         >
	>         > While Mike / Matt finish-up the Test Reqs, I'll give
a first
	>shot at
	>         > Test Cases and submit to the list (anyone interested
too?)
	>         > I just want us to agree on the format of Test Cases,
before
	>going
	>         > further.
	>         >
	>         > Regards,
	>         >
	>         > jacques.
	>         >
	>         >
	>         >
	>         > -----Original Message-----
	>         > From: Michael Kass [ mailto:michael.kass@nist.gov]
	>         > Sent: Wednesday, July 10, 2002 7:30 AM
	>         > To: Jacques Durand
	>         > Cc: 'ebxml-iic-conform@lists.oasis-open.org'
	>         > Subject: RE: [ebxml-iic-conform] MS Level 2 & 3 test
reqs
	>         >
	>         > At 08:12 PM 7/9/2002 -0700, Jacques Durand wrote:
	>         >
	>         > Mike:
	>         >
	>         > Great milestone, you have been key to all this !
	>         > and thanks to all who gave comments.
	>         > I see the following cosmetic remaining changes
before we
	>         > have something to submit to MS TC (do we all agree
here?):
	>         >
	>         >
	>         >
	>         > [MIKE] - Thanks also to XML Global for providing the
initial
	>test
	>         > requirements that
	>         > we have been massaging :)
	>         >
	>         >
	>         > - consolidate all these Levels in a single test req
document
	>that
	>         > ignores the notion of levels, yet keeps the
organization of
	>test reqs
	>         > by spec modules,
	>         > only for the sake of ease of browsing.
	>         > (levels/profiles would be defined separately, not
needed for
	>MS TC
	>         > submission)
	>         >
	>         >
	>         >
	>         > [MIKE] - I agree.
	>         >
	>         >
	>         > - renumber each test req item (ID) (Matt can help on
this he
	>said?
	>         > Note: if we use a pure sequential numbering, we
should not
	>change them
	>         > later
	>         > as these will be referenced everywhere, and yet we
should not
	>expect
	>         > them to remain
	>         > contiguous in the test req doc, as we may have to
add/remove
	>some test
	>         > reqs later based
	>         > on feedback.)
	>         >
	>         >
	>         >
	>         > [MIKE] - I will leave the "merging/restructuring" of
the 3
	>test
	>         > requirements documents and
	>         > XSL stylesheet to Matt, as he has volunteered to do
the
	>resequencing of
	>         > the test requirements.
	>         >
	>         >
	>         > - remove from this final copy the "Coverage"
attribute (would
	>go to the
	>         > annotated spec)
	>         >
	>         >
	>         >
	>         > [MIKE] - This is a simple stylesheet change, as far
as
	>rendering in
	>         > HTML.  However, the "coverage"
	>         > attribute is a valuable piece of information we may
want to
	>keep in the
	>         > original XML document...
	>         >
	>         >
	>         > Is that fine with everyone?
	>         >
	>         > Regards,
	>         >
	>         > jacques
	>         >
	>         > -----Original Message-----
	>         > From: Michael Kass [ mailto:michael.kass@nist.gov]
	>         > Sent: Tuesday, July 09, 2002 2:46 PM
	>         > To: Jacques Durand
	>         > Cc: 'ebxml-iic-conform@lists.oasis-open.org'
	>         > Subject: RE: [ebxml-iic-conform] MS Level 2 & 3 test
reqs
	>         >
	>         > Jacques and all,
	>         >
	>         >     Here are the latest versions of levels 1,2 and 3
ebXML MS
	>         > Conformance
	>         > Testing Requirements.
	>         > Incorporated are changes ( and a few not ) based
upon
	>comments.  Further
	>         > modifications based
	>         > upon discussion is possible ( but we seem to be
iterating to a
	>         > conclusion :)
	>         >
	>         >     I am also  attaching comments to Monica's and
Jacques
	>comments (
	>         > already sent out comments
	>         > for Michael Wang's post ).
	>         >
	>         >     All my comments begin with [MIKE3]
	>         >
	>         > Regards,
	>         > Mike
	>         >
	>         >
	>         --
	>         Matthew MacKenzie
	>         XML Global R&D
	>         PGP Key available upon request.
	>
	>
	>
----------------------------------------------------------------
	>         To subscribe or unsubscribe from this elist use the
subscription
	>         manager: < http://lists.oasis-open.org/ob/adm.pl>
	>
	>
	>
	
>----------------------------------------------------------------
	>To subscribe or unsubscribe from this elist use the
subscription
	>manager: < http://lists.oasis-open.org/ob/adm.pl>
	>.o.o
	
	
	



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Powered by eList eXpress LLC