About Prescription levels

["Durand, Jacques R." <JDurand@us.fujitsu.com>]

So far we have the following values for the prescription level:

 

- mandatory: cover statements that make use of keywords like : MUST , SHALL, REQUIRED

- preferred: cover statements that make use of keywords like : SHOULD , RECOMMENDED

- permitted: cover statements that make use of keywords like : OPTIONAL, MAY

 

These values are an abstraction of these popular keywords. Their intent is to reflect the level to which the feature is prescribed by the specification. They are an indication for the test case writer of what the outcome means in terms of conformance.

 

We need to decide of what to do with "negative statements" like MUST NOT,  SHOULD NOT, etc.

 

So far, if keeping the above keywords, by default we have:

 

MUST NOT,  SHALL NOT, (and MAY NOT?) --> mandatory, i.e. a predicate must be expressed so that it is "true" when the statement is fulfilled. E.g.: "the button MUST NOT be red" --> predicate = not ([button] is red).

"Mandatory" prescription means generally that if the predicate is "false" the specification is violated.

 

same for:

 

SHOULD NOT, NOT RECOMMENDED --> preferred

 

NOT REQUIRED --> permitted

 

Now, if we want to have more freedom in expressing the TA predicate, and avoid having to use the negative form, as Stephen suggested we could say instead:

 

 "the button MUST NOT be red"

predicate = [button] is red.

prescription = prohibited

 

and:

 

 "the button SHOULD NOT be purple"

predicate = [button] is purple.

prescription = discouraged (or "tolerated" ...)

 

===================================================================

Questions are:

===================================================================

 

-----  Question (1):  is there in some cases a difference worth of being reflected in the TA, between:

 

predicate = [button] is red.

prescription = prohibited

 

and:

 

predicate = not ([button] is red).

prescription = mandatory

 

I frankly would prefer not, but I may be wronged by a convincing example...

 

 

----- Question (2)

 

Regardless of  answer  to (1) we have stated in the past that the predicate outcome interpretation actually is more subtle than "fulfills / violates", e.g.:

 

(A)

outcome"true" = spec requirement is fulfilled

outcome"false" = spec requirement is violated

 

should rather in many cases be understood as:

 

(B)

outcome"true" = spec requirement NOT violated so far

outcome"false" = spec requirement is violated

 

Question is:

 

Should we do this distinction at TA level, or should we leave these subtleties to test cases?

(In favor of testcase level: the TA only states a criterion of adherence to a spec statement, while the test case is the one making a more committing statement on a SUT.)

 

If distinction at TA level, that means that "NOT violated so far" is not always same as "fulfilled". That means logically a 3rd value "undetermined" is needed. In that case we need a new interpretation element DIFFERENT from prescription, that says:

 

outcome"true" = fulfilled / violated / undetermined

outcome"false" = fulfilled / violated / undetermined

 

or the like.

 

NOTE: Even if ansering "no" to (2) we know that in some cases, some TA predicates cannot match exactly the specification requirement, for testability reasons (e.g. WS-I). A new "interpretation" element would allow describing such predicates that only allow partial tests against the normative statement.