[ebxml-iic-conform] RE: merging test assertions

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Monica & Mike:

 

Whichever solution you choose is OK wirh me on this issue of "merging" assertions (see cut&paste below).

I just wanted to say that I do not see a problem in "merging" such test assertions, when they are germane

(i.e. can be seen logically as subparts of the same test, and always relevant to the same level/profile of conformance)

only if there is a risk they belong to different profiles of conformance, they need be kept separate.

 

Take for example:

r0.1.1: the message envelope must validate against the envelope Schema.

Clearly this is an extensive test, that may fail for very diverse reasons.

Yet we won't decompose it (e.g. validate message subparts separately ), but

we can still provide enough detail in report of failure.

In the case of r2.2.6 below, I see the grouping  as being of the same nature: a logical test entity

on the right usage of the sequence number ("Reset" must always accompany a seq=0 when starting a conv.).

 

When it comes to write the Test Cases, because there is a cost in setting them up,

we might also decide to consolidate at this level. But its only an implementation trick.

So the outcome of a Test Case may report different results for each of the test Reqs it refers to.

 

Regards,

 

jacques

 

---------------------------------------------------------------------------------------

r2.2.6a: re-wording: (consolidated with r2.2.7a)

[precondition]: sending message with MessageOrder element, which is first for a

conversation ID

[assertion]: REQUIRED: the sequenceNumber element must have value 0,

and its Status attribute = "Reset".

[MIKE] - DONE

 

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

[mm1: IN THIS CASE (AND OTHERS), DO WE ACTUALLY HAVE TWO ASSERTIONS, JUST LIKE WE HAVE

0...N CONDITIONS/PRECONDITIONS - CAN WE ACCOMMODATE THIS - AND WITH A RESULT TO TRUE OR FALSE?]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

[MIKE3] - This is always the problem with "consolidating"... you "group" assertions ( and expected results )

together, so if there is a failure in one part of the Assertion, you fail the entire test.. So unless the test harness provides

a way to uncover exactly which part fails ( the sequence number not equal to "0" or the status not equal to "reset" )

you cannot tell an implementer why their implementation failed, only that it failed one or the other Assertion.

I am going to "decouple" the sequence number from the status, and split each test into two tests. It is less

"pretty", but it removes any ambiguity. We have to be consistent on this, and I haven't been in all cases.

-------------------------------------------------------------------------------------

 

 

>