ACTION ITEM: Vote on Discretionary Items report by end-of-day Monday(4/23)

[David_Marston@lotus.com]

Here is a new copy of The Catalog of Discretion, with internal fixes:


Please review this and vote on it, per the questions below. Please
respond (to the Committee list) by midnight Monday, April 23. Once it
passes, a number of other actions can take place, not the least of
which is that we can begin to poll the vendors. As you can see, this is
a catalog of all the areas of discretion that we can find in both XSLT
and XPath and their errata, but some won't affect the test suite. Those
items that won't affect *our* suite can still be used by the developer
in running their own vendor-specific (not conformance) tests in
combination with our suite. We also do not address the available
discretion regarding the text of error messages.

There is open discretion in language support. In this edition of the
catalog, we propose "convert-number-English" and "sort-English" as the
language-specific questions. If you know that we can get test cases for
other "interesting" languages (German, Spanish, Swedish, Japanese have
been suggested), let me know and I can add questions. We only want to
ask about languages that would really be represented in our test suite.

As we got into details, it became clear that we couldn't limit our
questions to multiple-choice and still capture all the information that
a test lab would want. Therefore, you'll see ELABORATE flags where
needed.

Here are some highlights:
A. At "add-attribute-to-non-element" you see the first and worst case
   where questions are interlocked. We should be able to proceed with
   our questionnaire and test cases, even if this should also go to the
   XSL Working Group at the W3C for possible errata.
B. At "unsupported-encoding-error" you see how we dealt with a choice
   that is more than two-way and also involves a "should" instead of
   "must" provision. We use interlocks to limit the range of answers.
C. At "support-disable-output-escaping" you see how a question can
   sometimes become moot through interlocks. In a real questionnaire,
   we would probably allow "moot" as a choice, but a test case would not
   be cataloged that way. Thus, the set of "valid choices" refers to the
   test case catalog rather than the questionnaire.
D. Expandable questions about language support could spawn subsidiary
   questions per language, as indicated. These would only apply when the
   language has more than one way to sort or stringify a number.
E. If "result-tree-as-bytes" is answered "no", it may not even be
   possible to use our output-comparison tools, but we should leave it
   to the test lab to do what they can.
F. At "converted-RTF-disabled-output-escaping" you see a case where it
   wasn't clear how many different facets of discretion are being given,
   so we keyed off their use of the word "both" and produced two items,
   rather than five or one. I need to submit a question or errata
   nomination to the Working Group regarding disabling output escaping
   for text that is used as the xsl:message content.
G. The preamble to the "Out of Scope" section explains why the three
   items are there. There are some tests that can use "foreign"
   attributes or top-level elements in a negative way that could be in
   scope, in which case the tests would not bear the discretionary flag
   and hence would always be included in the rendered suite.

GENERAL PRINCIPLE: These questions are to help customize the test suite
within the bounds of allowable discretion. When all processors must act
the same way on a given input, the test case does not need to carry the
discretionary flag.

===================== Cut 'n' Vote ==================================
1. Do you agree that all the items listed as "Testable" belong there?
   If not, state your objections.
2. Do you agree that all the items listed as "Testable Discretionary
   Items to Postpone" are testable? Do you agree that they should be
   postponed? If not, state your objections.
3. Do you agree that all the items listed as "Acknowledged But Not
   Testable" belong there? (These items would go on the questionnaire,
   but associated test cases would always be excluded. There are some
   tests that can use generated IDs, nodes from multiple documents, or
   output HTML character entities without running afoul of these items,
   so those cases would not be flagged.)
   If not, state your objections.
4. Do you agree that all the items listed as "Out of Scope" belong
   there, given clarification (G) above? If not, state your objections.
5. Do you agree that we have asked for enough information for test
   cataloging, suite rendition, and use of the suite by a test lab?
   If not, please ELABORATE.
=======================================================================
ACTION: Send your answers to questions 1-5 above to the Committee list
by midnight Monday, April 23.
.................David Marston