Re: 0.6 of the way to genericizing the test case catalog (Was: Halfwayto...)

[David_Marston@lotus.com]

>>At our June meeting, we decided that <Identifier> would include
>><Source> at the end, and hence would have that / between.

>Then I still have the problem of determining the subdirectory in which
>the source file is found.  I think that is going to be important for
>file management.

Source is not a file, necessarily. It is definitely the name of the test
case, and is probably the name (sans tag) of the input files. If the
latter turns out to be a problem, we could require that <input-file>
elements always be present. However, I think you're referring to the
different problem of how to identify the directory one should "cd" to
before running the test. That should be on the agenda for Wednesday.

>What is the filetype tag?

This is the next trouble spot. Do we want the submitter to specify the
name of the output file? If so, is that with tag (.xml, .html, .txt, and
.log for console messages) or without? When xsl:document comes along,
we probably will have no choice but to allow that. But then I ask: what
about a relative path (e.g., ../../outdir/foo.txt)? Where do we draw
the line? One thing's for sure: the catalog data must contain enough
information so that the Test Lab can find the output(s). (Note that the
Test Lab may have to combine catalog data and their own file-path data
to truly locate the output.) Right now, the Test Lab can say
  -OUT test.out
on every single XSLT test case, then move the file where they want to,
but that's not true for the generic test catalog, and it won't be true
for XSLT much longer.

>Should we add a compare= attribute to <output-file>?  I just did in my
>local copy of the prototype.  That means, I think, that we *don't*
>need the attribute in <scenario>, so I've removed it.

I already addressed that notion in my previous message, which crossed
paths with the one quoted above. Again, we can discuss it Wednesday,
or pass it to the comparing-output subcommittee. The prototype may
shed more light, but it probably won't exercise this area too
strongly.

Re: bringing up Macintosh in filetype discussions
In the MacOS, you don't just create foo.txt to make it be a text file.
You put certain stuff in the resource fork of the file.

>>GENERATING SCRIPTS
>I do not agree we should supply a Java invocation sequence ...
That's not what I was suggesting. The java part was to illustrate
that ultimately, a recognized command like "java" must be issued to
a command processor. My suggestion is that we provide an example that
transforms the rendered catalog into a text file full of command lines
(could act as both batch file under DOS and script under UNIX) where
the command is "runcase" literally. Then the Test Lab merely has to
envision how they will get the command line they really want.

>perhaps just a generic "run" batch file...
That's what I meant (as stated above), but I'm more concerned about
giving them a good stylesheet as a starting point to produce the
master file that runs all the tests.

>and could choose to run Java if desired for the tool being tested.
Sure. Another aspect is that the master transformation could
produce a Java/C/whatever program that invokes each test via API
calls. Yes, it probably has to do some form of "reset" after each
test case.

>>Think about this, because we
>>want to ensure that the catalog has all the right pieces that a Lab
>>would need...

>Sure, but can't we have a Lab tell us it cannot be done?

That's equivalent to the Lab telling us that they can't use our
catalog, which leads them away from using our suite, which leads to
this being a waste of time. I want our test suite to be useful!
My remarks were aimed at ensuring that the catalog data is more
than text to be transformed into a report, that it is the basis
for setting up to run the tests. At Wednesday's meeting, we should
push deeper into the issue of "what the Test Lab will need to know"
about each test case and about the suite as a whole.
.................David Marston