Re: [oic] "Test Suite": A Rose by Any Other Name ...

[robert_weir@us.ibm.com]

> Please respond to dennis.hamilton
> 
> I've been training myself not to use test and test suite when 
> speaking about what we are producing as the work of the ODF 
> Interoperability and Conformance TC.  My purpose is to avoid any 
> suggestion of comparison, assessment, acceptance mechanisms, and the
> use of text fixtures and evaluative suites of some sort. 
> 
> There are multiple reasons for this, in part because what we are 
> addressing is far removed from what the particular manifestation 
> might be in a viewer and especially in a processor.
> 
> So far, my favorite expression is feature-confirmation documents, 
> feature-confirmation exercises, and feature-confirmation 
> demonstrations.  I'm very big on confirmation tests (but am training
> myself to avoid test in that case and say confirmation case).  My 
> inclination is to speak of feature-confirmation cases and 
> interoperability-confirmation cases. 
> 
> Have others found it useful to defang "test" and, if so, what sort 
> of expressions are used to avoid the popular confusion around tests 
> and suites and provides something perhaps different if not more precise?
> 

You can always fall back on the language of the TC's charter:

"To collect the provisions of the ODF standard, and of standards 
normatively referenced by the ODF standard, and to produce a comprehensive 
conformity assessment methodology specification which enumerates all 
collected provisions, as well as specific actions recommended to test each 
provision, including definition of preconditions, expected results, 
scoring and reporting;"

But note that even there we use the word "test" and "assessment".  It is 
certainly within scope, and certainly the intent of the charter to allow 
the TC to publish written instruments that allow a third party 
(implementor or otherwise) to test the conformity of an ODF 
implementation.  Calling this a "test suite" does not seem to be an abuse 
of the term, at least not to me.

-Rob