Summary of where we stand on catalog data

[David_Marston@lotus.com]

This week, we settled most of the name-management issues that we needed
to tackle before issuing the prototype. Significantly, the "case-name"
string is back, and the troublesome Dublin Core names have been
replaced. Here are the name parts available:
1. test-catalog/@submitter names the directory for one submission
2. file-path gives the intermediary directories to locate a case
3. case-name or "id" is the name of a case for display purposes
4. input-file and output-file elements contain names of *all* files
   used in a test case
Note that (1) and (2) are only separate for our purposes, and that all
test lab uses need to use the concatenation of the two (with a / or
appropriate separator between).

Using the above name parts plus naming conventions suggested by the
testing guide, a test lab should be able to create all the following
in an automated way:
A. script to canonicalize all InfoSetized reference output
B. script that wraps the process of rendering the test suite for a
   given processor and errata level
C. script to check that all the input files needed by a test case
   exist in the correct directories (notable details to come)
D. line in a script to run a test from the command line, feeding in the
   name of the principal output file, and ensure that the output is
   either generated in or moved to the correct directory
E. variation of (D) where console output must be captured and moved to
   an appropriate directory
F. script to run all tests using a mixture of (D) and (E), and also
   make a log file that names the cases being run (could also display
   "purpose" strings or spec-citations if desired)
G. programs similar to (D) and (E) that use API calls
H. scripts to InfoSetize and then canonicalize the output from the
   test run, using different directory trees for each
I. script to compare all canonicalized test-run outputs against the
   corresponding reference outputs
J. script to delete files no longer needed after a successful run
K. HREF-clean version of fully-qualified name (change / to _)
L. Anything else?
In the above items, "script" means a batch file, shell script, or
whatever. These would be generated off the catalog data by an XSLT
transformation with text-method output. I've already had success with
such transformations. Notice that C-K must be run on the rendered
catalog, not the full catalog.

I think we can and should supply at least one stylesheet as either an
example or template to help the test lab succeed in generating their
scripts. By having such an example, we will give ourselves a target for
the XSLT sufficiency test that the lab will need. Supplying a template
means that we have a stylesheet that says "replace this string with
your command that invokes the processor" etc.

Once again, the exact mechanism for the external-params scenario is
being deferred. The rough idea (starting point for discussion) is that
there will be a supplemental input file containing parameter-setting
info. Whatever it is, assume it's in XML format so that it can be
transformed into additional tokens on a command line.

Referring to the Design Questions:
1. Outputs in parallel trees (allows submitted trees to be read-only)
2. Filetype extensions not mandated, but advised
3. InfoSetized and canonicalized output suggested to be segregated by
   both name and directory, and we will do that on our piece of it
4. Naming schemes preserve launchable extensions, I/C in middle
5. All input-file and output-file entries in catalog should have full
   names, with filetype extensions
6. case-name is explicit in catalog (attribute of <test-case>)
7. Submitter's catalog has outer <test-suite> element, one <test-catalog>
   element, and all the <test-case> elements inside it (one DTD)
8. We will fill in "submitter" string
9. file-path suggested to not have leading or trailing /, but our
   transformations will be able to deal with them being there
10. Catalog can use any character-set encoding
11. InfoSetized and canonicalized outputs will be UTF-8
12. Controversial Dublin Core names are replaced by more descriptive
    strings; non-controversial ones changed to all lower case
13. Date of submission applied to each <test-catalog> by us
14. There will be guidelines for submitters and test lab
.................David Marston