RE: [oic] Test Assertions and Test Cases

[robert_weir@us.ibm.com]

"Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 04/07/2009 
03:33:12 PM:

> 
> It seems to me that we need to be specific about test assertions and
> prescription levels.   I also think we need to consider prescription 
levels
> and/or test assertions that are not based on normative requirements in 
the
> specification.  We need to differentiate those where interoperability
> conditions are to be assessed where the ODF specification does not 
establish
> a measurable determination of conformance.
> 

I was thinking that would just be a difference in "Normative Source".  So 
something that is a "shall" according to the ODF standard would have 
prescription level "mandatory" and would reference the ODF standard as the 
Normative Source.  But something that was a recommendation of the ODF 
Accessibility Guidelines would have prescription level "preferred" with 
"ODF Accessibility Guidelines" as the normative source.  In other words, 
the prescription level is relative to the normative source.   That way we 
can filter the complete set of test assertions.  Ones where the normative 
source are the ODF standard and the prescription level is mandatory would 
comprise the conformance test. 

Note also the importance to TAG of the "prerequisite" item.  The name lead 
me astray at first.  I thought it referred to test setup procedures.  But 
it is really an indication of when the test is relevant.  So you can 
define a TA so that it applies in only certain situations.  This is 
important for ODF because most elements are optional.  But where they are 
used they may have stated syntactic or behavioral constraints.  So the TAG 
framework allows us to express this. 

Do you get the impression that the Test Assertion Guidelines TC is 
actively working?  If so it might be worth while establishing a liaison, 
or at least closely observing their work.  We might be able to pilot some 
of their work and give feedback. 

-Rob