Re: Notes about name management, and questions not yet resolved

["G. Ken Holman" <gkholman@CraneSoftwrights.com>]

At 01/07/13 16:33 -0400, David_Marston@lotus.com wrote:
>Based on the face-to-face part of Wednesday's meeting, it's probably
>worthwhile to produce another iteration of the test-catalog design and
>apply it for the prototype. Likewise, Ken's data for customizing the
>now-generic test catalog to XSLT would change. We can make that happen
>in a timely fashion with some email voting.

I have time to work on this this week.

>3. Interestingly, we observed no particular need for the submitter to
>"name" the cases if (2) is enforced. For appearances, a case name can be
>derived by taking the basename of the input-file designated as the
>primary stylesheet input, or the primary data input for those cases that
>have no stylesheet (embedded stylesheets within data). The likely uses of
>the case-name are in log-file entries and certain directory schemes where
>a directory named for the case-name contains the results of multiple runs.

A name (be it derived or specified) is going to be useful in Carmelo's 
result HTML documentation files for the test suite for hyperlinking purposes.

>4. Discussion revealed some conflicting uses for the output-file name(s).
>Our short-term plan was to say that the name in the catalog is the name
>for both the output as generated and for the "raw" version of the
>"correct" output file. The InfoSetized and canonicalized versions of both
>probably have matching basenames again, but canonicalized-file names are
>actually the concern of the Test Lab.
>
>5. Given (4), we need to put the InfoSetized "correct" output files, and
>the raw "correct" output if we deliver that as planned, in a safe place.
>Kirill proposed putting a subdirectory under each directory that contains
>test cases, which means that we would potentially be grafting directories
>onto the submitter's tree, and/or asking submitters to put raw correct
>output files there when they submit them. The OASIS Committee must assign
>a fixed name for this directory!

I am uncomfortable with the above.

>An alternative is to have one or two
>parallel trees, named by us, of correct output for all the cases in the
>merged suite (two trees if we want to provide raw and InfoSetized in
>separate trees).

Yes, I like the above, but do we have enough piecemeal information to 
recreate the directories?  I will try some MSDOS tricks for making 
subdirectories as that is the platform I will be using myself.  The end 
result doesn't matter to the lab, our subdirectories of infosetized results 
would be mirrored to the submitter's sacrosanct directories.

>6. The Title (or "suite-name") and Identifier ("file-path") are always
>used together when generating directory names and fully qualified file
>names (the latter also needs more data). The only reason they are
>separate is that the OASIS Committee sets the Title while the submitters
>name and organize directories in their sub-tree as they wish. A test lab
>will need to get the Title for each test case, which is most convenient
>if it's a datum within each test-case element in the merged catalog.

I don't think it is onerous to get it from an ancestral attribute.

>We
>could put it there in the merging process or require submitters to put it
>there.

Since the submitters don't know which value we use, and we may need the 
flexibility of changing the value, I don't think the submitter should even 
think about it nor even have it's value present in the individual tests.

>8. Submitted catalogs have an outer <test-catalog> element with many
><test-case> elements as children. Our merged catalog could have the same
>structure or could place the <test-catalog> elements as children of an
>outer <merged-catalog> element. The latter does allow Title to be an
>attribute of <test-catalog> instead of being in every <test-case>, but it
>means that the DTD differs between the submitted and merged catalogs.
>Some choices for (6) also mandate different DTDs.

I'm envisioning the merged catalogue DTD incorporating the submitted 
catalogue DTD.

>DATA REQUIREMENTS: If we agree to the double use of the output-file names
>as discussed in (4) above, then Iron/Germanium man has at least identified
>all the name pieces we want from submitters.

The double use of the same name is supported if each use is in a different 
directory structure.  Your suggestion of the parallel directory structure 
with only infosetized results accommodates this.  Any scheme involving 
renaming at the  file name level is inherently unstable ... the double use 
of the same output-file name (once in the submitter's subdirectory for the 
raw output file and once in the parallel infosetized subdirectory for the 
infosetized version of the output file) won't run into that problem.

>The new data items are the
>designators of the filetypes and primary/secondary status.

I will accommodate this in the next prototype.

>First draft of questions for voting:
>1. Should we require submitters to make an explicit <input-file> or
><output-file> entry for every file involved in every test case?
>2. Should we require the submitter to be explicit about all filetype
>extensions of all files in the catalog?
>3. Are we comfortable with a catalog that does not have a case name
>supplied for each test case, given that there is a formula for
>deriving one from required data?
>4. Should the names of InfoSetized output files be derived by adding a
>prefix, adding a suffix, or changing the extension (existing suffix)
>of the raw output file?

"or preserving the name without change and maintaining the file in a 
parallel subdirectory"

>5. Where should we deliver the correct outputs? Is the name of an
>InfoSetized output derived from the name of a raw output by adding a
>prefix, adding a suffix, or changing the existing extension (if any)?
>(In thinking about this, keep in mind that having a known extension
>like .xml is a convenience for launching apps in some environments.)

How is this different that Q4?

>6. Should the submitter have the Title (suite-name) in their catalog as
>submitted? Or have an empty placeholder for it?
>7. Should the file-path element have a leading slash? Trailing slash?

"Neither?"

>8. How important is it for the DTD of the merged catalog to be exactly
>the same as it is for the submitted catalogs? Should the <test-case>
>elements be at the same depth in both?
>9. Should the test catalog be submitted with only ASCII characters?
>10. Should we replace the Dublin Core name Title with "test-suite"?
>11. Should we replace the Dublin Core name Identifier with "file-path"?
>12. If Title and Identifier are dropped: Do you want to retain the
>non-controversial Dublin Core names Creator, Date, and Version?
>
>Interlocks among questions:
>(2) and (4) should be voted in a consistent fashion.
>(6), (8), (10) should be voted in a consistent fashion.
>(10) and (11) could be voted the same (keep/drop Dublin Core names) or
>not. Only if you vote to drop Dublin Core on both will (12) be raised.
>
>Other questions, not discussed in any depth at the meeting:
>13. To what extent should we provide guidelines to submitters about the
>current directory at the time the test is run? We can anticipate that
>test labs will want guidelines from us.

Really?  My gut reaction is this is out of scope.  How tests are run are up 
to the lab.  Perhaps the only guideline is to ask submitters not to use 
absolute URLs when referring to files.

>14. If a submitter wants to update their test suite, which could mean
>adding, dropping, or changing test cases, how do we want them to catalog
>the revision?

I think this can be deferred.  If we are asked to wholly replace an old 
submission with a new submission I would not want to do piecemeal 
fidgeting.  The only savings is not having to review the same test a second 
time, which we should be able to do with path and file names and the 
date/time of the test case (part of the tracking information of the review 
process).

>15. For error cases, we could provide compare data of two types: an
><elaboration> string explains the error for human understanding, and/or
>a <substring> element is something to grep for in the console output.
>How much help should we provide to test labs in this area?
>16. Do we want to track a Date of a whole submission in this catalog?
>17. We are continuing to defer design of an input file that carries the
>parameter-setting data. The mention of a secondary-params type of input
>file is strictly meant as a placeholder in case we decide that (a) this
>data should be conveyed in a file, and (b) the file can be treated the
>same as other inputs for purposes of file management.

All valid questions to ask, David, and I'm prepared to answer most of them.

I would like to proceed with prototyping with some preconceived answers so 
we can all look at some working code instead of just prose.  Any 
preconceived answer is welcome to be changed by a result of the vote.

During the development phase I think we should call this a "canvas of 
opinion" rather than a "vote" and save the votes for issues of policy 
during our formal meetings.  If any member (myself included!) kicks up a 
fuss about a consensus or near consensus opinion we can try formal email 
voting, but you are just trying to get development direction in these 
questions.  I'm willing to prototype any answers that reflect a majority 
opinion of those on the list.

Time is of the essence folks ... if you have an opinion on the questions 
that David posts as a result of our feedback to the formation of these 
questions, please do not hesitate.  I would hate to waste development time 
prototyping an apparent agreement only to find late opinions swaying the 
outcome.

Dave, at your earliest convenience could you please incorporate comments 
received about your questions and ask them again soliciting feedback?

Thanks!

.......................... Ken


--
G. Ken Holman                      mailto:gkholman@CraneSoftwrights.com
Crane Softwrights Ltd.               http://www.CraneSoftwrights.com/s/
Box 266, Kars, Ontario CANADA K0A-2E0     +1(613)489-0999   (Fax:-0995)
Web site:     XSL/XML/DSSSL/SGML/OmniMark services, training, products.
Book:  Practical Transformation Using XSLT and XPath ISBN 1-894049-06-3
Article: What is XSLT? http://www.xml.com/pub/2000/08/holman/index.html
Next public instructor-led training:      2001-08-12,08-13,09-19,10-01,
-                                               10-04,10-22,10-29,02-02

Training Blitz: 3-days XSLT/XPath, 2-days XSLFO in Ottawa 2001-10-01/05