Test Assertion Guidelines

[robert_weir@us.ibm.com]

There has been some work in this area in the W3C and within OASIS.  In 
particular I'd like to draw your attention to the latest draft from the 
Test Assertion Guidelines TC:

http://www.oasis-open.org/committees/document.php?document_id=30633&wg_abbrev=tag

Figure 1 on page 7 illustrates their model of the relationship between 
normative statements in a standard, test assertions and test cases in a 
test suite. 

The test assertion sits between the standard and the test case.  They give 
an example on page 11.  If the standard says, "A widget MUST be of 
rectangular shape", then a test assertion might say:

TA id: widget-TA100-1 
Normative Source: ?widget specification?, requirement 100
Target: widget
Predicate: [the widget] is of rectangular shape
Prescription Level: mandatory

Note that this is not really a test case yet, since an application cannot 
process the TA directly.  But it is easy enough to create a test case that 
would express such TA in directly testable form.  The idea is that a test 
assertion is not tied to any specific embodiment.  So, in our case with 
ODF, a single test assertion might make a statement that could be 
expressible as an ODT file, and ODS file or an ODP file.  So a single test 
assertion could result in multiple test cases.

A more worked out example is what the W3C has done with HTML 4.01:  
http://www.w3.org/MarkUp/Test/HTML401/current/assertions/assertions_toc.html

I'd like to see us consider creating a set of TA's for ODF, as well the 
directly-consumable test cases.  This appears to be the emerging best 
practice.  In fact, if we want to concentrate on a single area first -- I 
notice Bart has signed up for Packaging -- we could use that as a trial. 
We could create TA's as well as test cases for that section and get a 
better feel for how they can work together.

-Rob