More about naming needs

[David_Marston@lotus.com]

This message follows up on my posting from earlier today, and includes
some thoughts stimulated by what Kirill and Carmelo just sent. In the
face-to-face portion of our July 11 meeting, we discussed how the data
in the test catalog would be used by the test lab to accomplish their
testing.

The detail tasks of constructing the names of various files and
directories operate in support of making scripts work for the four
major phases of the testing effort. Once again, these phases are:
A. Setup script to put inputs in the proper place (or confirm that they
   are in place and ready)
B. Run script to run test cases
C. Compare script to process and compare actual and reference output
D. Cleanup script to move or delete output and log files

In Carmelo's "requirements" list, sent earlier today, data about
individual cases can be taken from the test catalog in support of:
    4) The test name.
    5) The nature of this test (positive, negative, implementation-defined,
etc) .
    6) The expected output to this test (if any).
    7) How do I run the tests?
    8) Any secondary files needed by the tests.
    10) How do I interpret the results? <== For comparing against reference
output
    18) Which information do I need to provide to any of the tests?
Of course, data in the <discretionary> element supports the prior
phase of rendering the test suite according to developer choices.

NOW, THE DETAILS
Each numbered entry below is a detailed task. We show how the task
uses the combined suite-name and file-path (called "file-path" for
simplicity) and/or the filename supplied in input-file or
output-file. If the Source (a.k.a. case-name) continues to exist, we
show how that could be used. Recall that the formula for deriving the
case-name is: take the basename of the input-file designated as the
primary stylesheet input, or the primary data input for those cases
that have no stylesheet.

1. Display case-name in log of test run: must show case-name derived
from primary input, or could be sloppy and just show the primary input
file name as found. (Use the Source if still available.) For full
identification, the log must show the file-path data, but it could
show that each time it changes rather than for every case.

2. Fetch a primary input file: use the input-file name along with the
assembled file-path to find the file.

3. Fetch/verify a secondary input file: Take the file-path and append a
slash followed by the input-file name. Map / to another character if
required by the environment. If the command shell cannot interpret .
and .. in the middle of the path, perform a path normalization.

4. Find the data about setting external parameters and convert it into
the necessary invocation options: to be designed later.

5. Construct the name of the primary output file: Use the output-file
name in the catalog, along with any relative pathing that you wish to
apply. If you wish to maintain outputs in a separate file tree, use
the file-path data to construct the necessary parallel structure (or
map / to _ and make long file names). Advice: it may be safer to just
create the output file in the current directory, then move it after (if
the test runs to completion and if the file is non-empty). If you want
to have your own file extensions, you can trim off the existing ones or
just suffix beyond them, but consult the compare attribute rather than
the current extension to know which filetype is expected.

6. Construct the names of secondary output files: take the names from
the catalog and apply the same pathing as in (5) and also path
normalization as in (3) if required.

7. Designate directory to become current while the test is run: use
the file-path information in combination with the top-level path for
the system on which the tests are run.

8. Designate directory into which raw, InfoSetized, or canonicalized
output file goes: use file-path and/or basename of primary input in
combination with locally-designated data. (Can use the Source if still
available.)

9. Designate directory where log-file goes: the data from (8) can be
used if you wish.

10. Fetch the reference output: combine the file-path and output-file
along with the string constant designated by OASIS.

11. Construct the command line for processing or arguments to API:
use the primary input and output file names as given. (NOTE: obtain
invocation syntax from vendor.)

12. Pre-generate the canonicalized form of the reference output:
use file-path and/or basename of primary input in combination with
locally-designated data. (Use the Source if still available.)
.................David Marston