OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

tag-comment message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: RE: [tag-comment] Comment: include a means to reverse logic of a test assertion



We acknowledge the comment you made below.


Here is my personal take on your comment:


There are indeed cases where it is easier to demonstrate a “failure” than to demonstrate conformance.


> This would allow for assertions which are more easily stated by their negative.

You do not provide examples of this, so let me provide some in my view and start from there:

Let us consider the normative statement “all widgets must be painted blue.” Let us also assume that as far as testability goes, it is common knowledge that test equipment for widgets can only test for red and yellow colors, i.e. for “failure to be blue”, but no conclusive test exists for being blue.

Your request makes sense in situations where (a) there are known limits to testability of some test target for some requirement, that make it only possible to detect a subset of failure cases, but not to assert conformance to the requirement, (b) one wants to design test assertions that reflect this reality (instead of merely reflecting the property required in the normative statement, and let the test suite designer deal with the test implementation issue.)

I must say that in our methodology so far, we made the choice to have the TA predicate faithfully reflect the normative statement, so (b) is not really an option we favor.

However, nothing prevents us to use a combination of more than one TA to address a normative statement, while “interpreting” the normative source to indicate what aspect of the normative statement each TA is addressing, i.e. here:


-          normSource (with “interpretation”): “an aspect of being “blue” is to NOT being red”

-          predicate: “the widget is NOT red” ,


-          normSource (with “interpretation”): “an aspect of being “blue” is to NOT being yellow”

-          predicate: “the widget is NOT yellow”

Note that these are not “reverted” TAs. They follow the expected logic: if predicate is false then the normative source interpretation is adhered with, otherwise is violated.

Then we can write a TA fully addressing the original normative statement , yet one we know will only be partially “testable”:


-          normSource (with equivalent “interpretation”): “widget being “blue” is same as NOT being of any other known color.”

-          Predicate:  (TA”notRed” = ‘pass’) AND  (TA”notYellow” = ‘pass’) AND (widget is not of any other color)

Here the job of the TA writer is done: s/he has faithfully matched the normative requirements, yet in a way that will help the test suite writer. The test suite writer knows that in the current state of affair, TA”Blue” cannot be fully implemented YET can be partially implemented, thanks to the testability of “red” and “yellow”: its predicate will definitely fail the test if widget is red or yellow, yet cannot be evaluated by known test technology in other cases.

So the test suite writer , recognizing his/her inability to fully implement the TA”Blue”, should write for it a test case that will either return:

-          Fail (if either TA”notRed” or TA”notYellow” fail)

-          Inconclusive otherwise.

In other words: I take your suggestion Stephen as relevant to best practices (TA guidelines) and indeed a case that is worth addressing in the “guidelines” but not in the model so far. The current model appears to be able to address the “partial testability” concern as I understand it at least. I am afraid the “reverse logic” feature would open a can of worms that affect more deeply the model fundamentals…  If we need to go that route I think we need first a convincing set of use cases and if we can’t address it with TA model 1.0 then roll our sleeves for 2.0.


The TAG TC will nevertheless examine your comment and ways to address it.


Best regards,


Jacques Durand

TAG TC chair


From: Stephen D Green [mailto:stephengreenubl@gmail.com]
Sent: Thursday, August 02, 2012 7:38 AM
To: tag-comment@lists.oasis-open.org
Subject: [tag-comment] Comment: include a means to reverse logic of a test assertion


A comment on Test Assertions Model V1.0:


May I suggest (perhaps for a future minor version) that there be a

means in the model to indicate a reversal of the logic such that

a test assertion predicate can be expressed negatively where a

test outcome based on such a reversed test assertion is positive

when the test fails. This would allow for assertions which are

more easily stated by their negative. I suggest one way to do this

might be to extend the list of prescription levels to include negated

values (MandatoryNOT, PreferredNOT) but this does not work if

the prescription level is optional, so another means to indicate

negation of the semantics of a test assertion would be preferred.


Best regards,


Stephen D Green


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]