Comments on previous comments on (1) IUT + TA, (2) pre-requisites

["Durand, Jacques R." <JDurand@us.fujitsu.com>]

 

Some comments on two points:

 

1. IUT and TA:

 

"A best practice technique is to list the IUTs to which a TA applies using words

like 'applies to ...'

Consensus that IUT should not be required as part of the individual TA"

 

I believe there are here different concerns, that may need to be addressed separately in the guide:

 

(1) IUT vs COnformance Target

The two should probably be presented as orthogonal - generally not of the same level of granularity,

but could overlap.

In WS-I Profiles, you have for example TAs that apply to a [Web service] INSTANCEs

which clearly match to real world products, that could definitely also be the target of

conformance clauses. But you also have TAs that are targeting much finer granularity entities

such as [message] ENVELOPE.

 

(2) Not having to mention the IUT for many TAs.

I believe that an IUT is an essential part of a TA (otherwise WHAT is the testing about ?)

BUT the IUT may not need be mentioned in every single TA, assuming that it can be

made somehow "global" to many. As long as the user has a means to figure out what the IUT is

(what class of objects) for a TA, then we are fine.

 

(3) A same spec req may apply to different IUTs.

Right - but I feel that a TA cannot afford the same fuzziness.

For example, if a spec requirement says "error messages must contain a reference to the faulty

message", then this can be interpreted either as a requirement for either (a) the error message

artifact itself, or (b) the message handler that produces such messages.

It think it is OK to write two TAs here, one for each IUT, or just one, depending which IUT is of interest.

 

 

2. Sequences of Test Assertions, and pre-requisites

 

"Make it the rule to keep each test assertion singular and atomic but recognize that

there are exceptions

when an assertion depends on another."

Rule: keep assertions singular and atomic.

 

Yes, singular and atomic in that the expression that evaluates the test outcome does not computationally depend on other TA outcome (i.e. can be evaluated without having to resort to the test outcome expression of another TA)

I think that was the prime concern from Patrick and Sun.

Concretely, if TA1 is a pre-requisite for TA2, this means that the test case that exercises TA2 can still be executed independently from test case for TA1, but that the outcome of TA2 may not be meaningful - i.e. the logic of TA2 testing

was assuming that TA1 was successful for the same IUT.

E.g WS-I TAyyyy on presence of some elements in SOAP header, has pre-req TAxxxx on SOAP extension compliance of the header.

If TAyyyy fails for IUT 123 you can infer that 123 was not conforming/adhering to the requirement addressed by TAyyyy

ONLY IF TAxxxx was successful for this IUT.

 

-Jacques