[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Subject: My answers to Design Questions #2 and #3 (Action Item for TC)
>Design Question #2: How should filetype extensions be used on the >raw output files and the corresponding correct-output files? >... >A. All principal transformation outputs have the same extension... >B. Principal transformation outputs have an extension pertaining to >the type.... I favor "B" as an advisory position. (Disclosure: the Lotus suite currently implements "A" and it would be hard to change.) We should take whatever we can get for a submission, assuming that it's correctly cataloged with the extension/tag included. I think this is similar to what Ken and Lofton have already said: don't force the submitter to adopt a given scheme, but design all our stuff so that usefull filetypes are present (facilitating auto-launching of the correct associated application), and advise submitters to choose meaningful extensions if they ask. >Design Question #3: Should the canonicalized and InfoSetized outputs >be distinguished by naming, directories, or both? >... >X. ...parallel directory trees...but filenames...the same. >Y. ...parallel directory trees...and a naming scheme... >Z. A naming scheme...but they are in the same directory. I think separate directories is definitely a good idea, especially when you picture a test lab having results for several processors, different run dates, etc. so Z is out for me. I also like the use of a naming scheme for safety reasons, as Lofton mentioned, so I'd go with Y. I'd like the naming scheme to preserve meaningful tags, and I prefer work only with suffixes so that the "basename" is always on the left, which boils down to favoring an extra piece in the middle for the InfoSetized and canonicalized forms. We don't necessarily have to worry about it, but I would note that some whole-directory diff utilities will work best if filenames match exactly, which refers to the canonicalized forms in our case. For that reason, I'd like the same names between the reference output and test-runs results, with naming changes only applying along the raw/InfoSetized/canonicalized direction. Here's a complete picture of what I favor: we ship blah/blah/submitted-output/blah/LRE001.xml (raw reference output) blah/blah/reference-output/blah/LRE001.I.xml (InfoSetized version) Upon setting up the test harness, they canonicalize on-site blah/blah/compare-output/blah/LRE001.c.xml (canonicalized version) (Notice that they create a new set of directories on-site.) The lab runs the test case, and the catalog tells them to create test/run/identifier/blah/LRE001.xml (raw reference output) which they then transform into test/run/InfoSet/blah/LRE001.I.xml (InfoSetized version) which they then canonicalize into test/run/compare/blah/LRE001.C.xml (canonicalized version) Having done that for all tests, they run a multi-file diff on blah/blah/compare-output/* test/run/compare/* and start tabulating the differences. Filenames in the above two directories would match exactly. .................David Marston
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Powered by eList eXpress LLC