Responses to Ken's comments accompanying his votes.

[David_Marston@lotus.com]

Overall observation: we've expended so much effort on this, that I'm
hoping it can be used for multiple purposes. You'll se what I mean below.

>> is me, last time
> is what Ken just sent

>I think all of those in section "Discretionary Items
>Acknowledged But Not Testable" would go into the section "Out of Scope
>Discretionary Behaviour ... reading the first paragraph of each section
>would imply the same thing is being discussed.

This is could be done if the votes don't pass it as-is. We had our
nuances of meaning there, some of which will emerge below.

>>Those
>>items that won't affect *our* suite can still be used by the developer
>>in running their own vendor-specific (not conformance) tests in
>>combination with our suite.

An alternative to consider: take all the items in either non-testable
or out-of-scope and re-divide them according to whether the test lab
needs the answer. Many of the others are interesting for reporting to
the consumer, but not crucial to test administration.

>But is elaboration required from our perspective?  I ask because if so,
>then does it belong in the configuration instance and do we include in the

>configured output created by Carmelo the elaboration as captured in the
>configuration instance?

I just re-read all the ELABORATE items, and I would say that they *are*
required from our perspective when the test is Testable or Out-of-Scope
but needed info (as mentioned above). Of particular note is the default
encoding: if the test lab doesn't know what the default is, then we
have to make sure 99.99% of the tests specify an encoding, which in turn
means we either edit submitted tests or send them back and require the
submitters to do that, or else we use the answers on default encoding to
guide the test lab and possibly also paramterize our post-processing
tools. The ELABORATE flag is guidance to the test lab, but we would have
reduced the info to binary questions had that been possible. Thus, I
don't expect that the elaborations can be used to mechanically isolate
irrelevant tests. We can pass along the elaboration as text if you wish.

>>There are some tests that can use "foreign"
>>attributes or top-level elements in a negative way that could be in
>>scope, in which case the tests would not bear the discretionary flag
>>and hence would always be included in the rendered suite.
>In the section's second paragraph, since the phrase "must not affect"
>is used, then couldn't we test it isn't affected?

That's part of what I'm saying: the test case that shows it *isn't*
affected is always in the suite; its catalog entry does not carry the
"discretionary" flag for this item. If we get a test case that uses a
foreign attribute or top-level element, we'd probably reject it outright
but this flag lets the developer use our test system for themselves.

>I'm curious if it will be important to developers to split up
>add-namespace-to-non-element into four separate discretionary items.  I
>see there are four "It is an error..." descriptions, though only two have
>"implementations may".

The two that have "implementations may" resulted in two items on the
discretionary list. In general, we've already split as much as we can,
keeping in mind that these error situations should apply to the
creation of the result tree, not the (later) serialization thereof.

>In "number-not-positive" perhaps add the two words "when rounded" to the
>end of the question to handle those positive numbers less than 0.5 as
>indicated in the prose.

Fine with me, as long as this doesn't force a re-vote we could
otherwise avoid. Anyone object to calling it a "minor fix"?

>Perhaps, then, the
>question is "which of the allowed default encodings is used?" with the
>only answers being "UTF-8" and "UTF-16" wherefrom we can determine that a
>processor not implementing either does not conform.

That's what we originally intended to do, then we saw that word "should"
in there. Any test lab that will report results to the public certainly
could raise a fuss if the default is something else, but it conforms.

>Why not just have the two
>questions "disable-output-escaping-value-of" and
>"disable-output-escaping-text" with values "ignore" and "raise
>error"?  How would the test suite configuration be altered by the "Does
>the processor support disabling output escaping?" question?

We didn't see a separate discretionary grant for xsl:text and
xal:value-of. The structure came from an attempt to express all
questions as binary. So the effect of the questions would be as
follows:

support-disable-output-escaping="yes"
   Include lots of tests that have disable-output-escaping
   Include tests where d-o-e used incorrectly
     Filter those according to answers to non-text-d-o-e and
       unrepresented-character-d-o-e to pick just error or ignore
   Exclude tests that expect d-o-e to raise an error
   Only the latter would have unsupported-d-o-e answers
support-disable-output-escaping="no"
   Exclude lots of tests that have "functional" d-o-e
   Exclude tests where d-o-e used incorrectly
     Filtration of these tests becomes irrelevant; all are out
   Include tests that expect d-o-e to raise an error
   Use unsupported-d-o-e answer to pick just error or ignore
Whew!

>I think "stylesheet-parameter" is out of scope. However the value is
>assigned from the external environment does not impact on the
>conformance of the use of a top level parameterized variable. I don't
>see how our test suite would be impacted by an answer to this question.

Note that the actual item is about *whether* external parameters can
be set. The only reason it's even in the "Postpone" group is that we
aren't too near consensus on how to express parameter-setting in the
catalog of test cases. The test lab needs the setting data from the
catalog plus the elaboration from the vendor to run such tests.

>Ditto on the above for the "result-tree-as-bytes". Important to a testing
>organization, but I don't think to the test suite itself. Actually, if the

>answer is "no" then our comparison methodology is inapplicable, so it
>really wouldn't impact us at all.

This document is serving multiple purposes. I think you've identified why
we have to ask the question! It may be out-of-scope if we say that a "no"
answer means our test suite is useless. I'm not so pessimistic; I think
most tests can be run if the test lab figures out how to compare outputs,
but a few won't be relevant. Hence, it still has filtration value.

>I think "obey-xsl-output" needn't be postponed. If the answer is no, that
>will eliminate a number of tests for anything that isn't serialized as
XML.

As noted in the document, the above two items interlock. A "no" answer
opens a lot of possibilities, so if we don't postpone, we might begin by
just saying that every single test that has xsl:output automatically gets
this discretionary flag attached to it.

>I think "converted-RTF-disabled-output-escaping" needn't be postponed ...
>it seems quite well defined.

Probably true. This is the most recent change, and maybe we were too
cautious here.

>...but I think we will have to underscore this is a
>supplemental list of questions unrelated to conformance testing and
>provided by the committee for the information of users.

I agree with providing that perspective, but I think that "unrelated" is
an exaggeration. More precisely: some questions include/exclude test
cases, while others tell you important info about what the processor is
going to do, and sometimes how to do things with it. If you need the
answer in order to be able to *use* the processor, then you also need
the answer in order to be able to *test* the processor.

>I'm wondering about the use of absolute XPointer addresses instead of
>addresses relative to the closes ancestral element with a unique
>identifier....
>My rationale for relative addresses is the resilience to future versions
>of the recommendation that may (but of course not guarantee to) preserve
>these relative values in many places.

I presume that you're talking about "Editions" like XML had. I'm happy to
use the most robust pointer available. Again, if all else is voted
affirmatively, I'd like to consider this a minor revision.

>Does it make sense to sort the items, or add to the end of the
>report a sorted index?

I think the index, or even two indexes, would be the way to go. It's an
HTML document, because that's the way Tony wrote it, but we could
either XMLize it or load it up with HREFs pointing back and forth.

Maybe we should just vote on the content, and leave some details of
internal structure to be filled in. I think we have a lot of actions
waiting on this. We could publish the HTML version on our Web pages
(part 10.1, with a pointer at 6.1), and say that a heavy-duty
version with the same content will come later.
.................David Marston