Another way to express the Test Lab's requirements and expectations

[David_Marston@lotus.com]

In earlier messages, I described certain aspects of how a Test Lab
would use our test suite in conjunction with one or more XSLT
processors and an "environment" (operating system and other code) to
produce test results. In one case, I mentioned these four phases:
1. Setup for a test run
2. Run the (applicable) test cases through the processor
3. Compare results of this run against the reference output
4. Cleanup of temporary stuff
But the above is about what happens in a test run. We also need to
consider how well a Test Lab can use our whole package. To broaden
the discussion, I'm initiating two checklists of expectations.
Consider these to be complementary to the "requirements" list that
Carmelo sent earlier.

WHAT THE TEST LAB SHOULD GET FROM OASIS:
Instructions for installing the test suite
Instructions for rendering the suite for a particular test run
Instructions for canonicalizing and comparing output
Instructions particular to each "operation" scenario
Guidance on running single test cases that check the setup
Guidance on the uses of individual data items in the catalog
Guidance on the directory structure (e.g., read-only parts)
Information about discretionary items and the questionnaire
Information on the W3C specs and errata we deal with
(When the time comes) instructions about updating the suite

WHAT THE TEST LAB NEEDS TO GET FROM THE PROCESSOR DEVELOPER(S):
Specifications of the necessary operating system (or JRE)
Specifications of other software needed to run the processor
Instructions for installing the processor
Command-line and/or API arguments; invocation sequence
Information about how errors and xsl:message are handled
Information from our questionnaire about discretionary items
Information about languages and locales supported

If you have additions, send them to the whole list. Later, each of
the above line-items can be exploded into a detailed sub-list.
.................David Marston