Re: Test case metadata is not in scope (was: Re: Reasons to wait...)

[david_marston@us.ibm.com]

In an attempt to avoid obfuscating, I will start over in presenting the 
environment surrounding the proposal to have a TC set a standard for Test 
Assertions (TAs).

I. Specs are written by documentation-type people. They write prose, and 
much of it (from OASIS, W3C, et al., at least) is loaded with testable 
sentences. Some specs, such as XPath, also have numbered and individually 
ID'd "productions" in a formal grammar. Spec writers would be the logical 
group to write TAs, which are a more formal version of the testable 
sentences they currently write. They await some standards for TAs.

II. Test cases are written by software-quality-assurance-type people. Some 
of them also use test case metadata, either in portable standalone files 
(e.g., XQuery Test Suite) or managed by a tool such Rational TestManager. 
These people are the ones who write test case metadata, and they are doing 
so without standards. But they would benefit if there were standards for 
test case metadata, at least setting a floor.

A good summary of the state of the art for test case metadata as used in 
portable test suites is at
http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/
Note in particular one item of data for each test case is the SpecRef; see 
section 2.6 for details. The SpecRef would point to the TAs used in the 
test case, if they existed. The SpecRef is distinct from the Purpose 
(section 2.3) and Description (section 2.4) fields of the metadata. 
Depending on the form that the standardized TAs take, they may become 
associated with Preconditions (2.7), Inputs (2.8), and/or ExpectedResults 
(2.9).

III. Code is written by software developers. They will use specs, test 
cases, TAs, and whatever else they can get as they attempt to write 
bug-free code that behaves as the specs dictate. There are several 
commercial software tool suppliers and projects like Eclipse that attempt 
to make these people as productive as possible by providing Integrated 
Development Environments (IDEs) and the like. If there were a standard for 
TAs that defined them in a way that enabled a set of TAs to be poured into 
an IDE, and if that helped the developers to be more productive, great 
appreciation would be heaped upon those who set the TA standard.

In summary, TAs flow from Group I to Groups II and III. Group II would use 
TAs in conjunction with test case metadata. An individual can be a member 
of more than one group; the groups define "roles" rather than, say, 
personality traits.
.................David Marston