Questions on the Submission and Review Policies

[Cris Cooley <ccooley@overdomain.com>]

Greetings everyone,
I need your input to the following questions, so I can turn our Policy
bullets into Policy prose.  I've read / re-read the material on the website,
which answered a few questions, but left most unanswered.  I'll be on
vacation tomorrow afternoon through all the next week, so I'm hoping to come
home to a mailbox full of responses!  Once I get your answers, I'll plan to
call if I have questions, if Carmelo has a list complete; if not, perhaps
you can list your phone number.  (I think for clarification, phone is much
more efficient.)

A lot of the questions may seem basic (well...they are basic!) but I think
your answers will help me to communicate to  others less expert what we're
doing and what we're expecting reviewers and submitters to do.  Please, have
at it!  I promise it won't take more than a few minutes.  Thanks.

Cris


Questions (please type your answer after the "A:" before the square
bracket).

[Q: Has anyone from the committee submitted anything in addition to these
bullets?  (I haven't seen anything, so I assume no...)
A:    ]


1. Submission Policy (bullets with questions)

[Q: What suggestions do you have for an introductory paragraph?
A:    ]


The following ideas were captured regarding submission process and practice:

  - prefer atomic tests for 1.0
[Q: What are atomic tests?  Who prefers them?  Prefers them to what?
A:    ]
Q: What does 1.0 refer to?  The test suite?  Is this defined?
A: 	]

    - target specific language issues, not composite issues
[Q: What language?  XSLT?
A:
Q: What are composite issues?
A: 	]

    - consider others later
[Q: Other what?  issues?  What other issues?  When should they be
considered?
What boundaries exist on what we should consider and when we should consider
it?
A: 	]

  - committee reserves right to exclude any test submitted
[Q: What possible reasons might the committee have for exclusion?  Is there
any
formal process for notifying the submitter of exclusion?
A: 	]

  - prefer no "should" decisions for 1.0 suite of tests
[Q:What does this mean? How does this impact submitters?
A:	]

    - target only explicit processor choices, not unspecified areas of
Recommendations
[Q: Does this refer to W3C Recommendation on XSLT?  Does this mean the test
suite will only target choices made by the XSLT processor vendors?  Why?
(limitation on scope?  other?)
A:	]

  - test identification:
[Q:What is test id?
A:	]

    - use test file hierarchy
[Q: What is the test file hierarchy?  Use it for what? (for id?)  Who should
use
it?
A:	]

    - base hierarchy on root directory of submitter
[Q: What hierarchy?  (Assume "base" is a verb...?)  By "root directory" you
must mean the directory designated by submitter on their server (regardless
of whether it is actually the root of any server drive) ?
A:	]

    - submitter welcome to arrange subdirectories as they wish
[Q: Is there a connection between the test file hierarchy & the subdirectory
hierarchy?  Are they the same?  Different?  How?
A: 	]

    - each test will have a unique identifier as well
[Q: What is the identifier?  As well as what?  By test, do you mean each
test file or each submission or test performance, or something else?
A: 	]

    - each submitter will be assigned a unique identifier
[no question ]

    - final test identifier will be concatenation of submitter and test ids
[Q: What is the difference between the test and the final test?  By
"identifier" do  you mean file name?  element name?  something else?
A: 	]

  - test scope will be identified by Recommendation and date
[Q: What is test scope?  Does "Recommendation" = W3C xslt Recommendation?
date of what?  Submission?  Other?
A: 	]

-	of recommendation itself
[Q: What does this mean?  (I don't see what it connects to...)  Is this
talking about the W3C xslt Recommendation?  (not capitalized = something
different?)

-	of modified date of errata document
[Q: What does this mean?  What is it connected to?  What is errata document?
Why is the date modified?
A: 	]

[Q: What else should be included in the Submission Policy?
A:    ]


2. Review policy

[Q: What suggestions do you have for an introductory paragraph?
A:    ]

  1 - judge the eligibility of a test by:
[Q: What is eligibility?
A:	]

      - accuracy of test
[Q: What does accuracy mean?  What is the baseline for determining it?  What
is
the means for measuring it?
A:	]

      - scope of test
[Q: (Already asked) what is scope?
A: (above)	]

      - clarity of test
[Q: What is clarity?  How is it measured?
A:	]

      - clarity of aspect of recommendation being tested
[Q: What is clarity of aspect?  Does this refer to W3C Recommendation?
A:	]

      - should/shall use in the recommendation
[Q: What does this mean?  What recommendation?  Who should/shall?
A:	]

      - is the test testing a discretionary item?
[Q: What is a discretionary item?  Defined where by whom?
A:	]
      - atomic/molecular nature of the test
[Q: What does atomic mean? (asked above) What does molecular mean?  What is
meant by nature (specifically)?
A:	]

  2 - judge each eligible test through a process
[Q: Who is judging?  Who is being judged?  What process?  Where defined?
A:	]

      - run thorugh multiple processors
[Q: xlst processors?  Whose?  Which one is the benchmark or baseline, or is
there one?
A:	]

        - any differences imply examination by committee
[Q: Differences between what & what? (submitter expected output and user
actual output?)  What does this mean "imply"?  Who is the committee?  This
xslt conformance committee?  What sort of examination?
A:	]

        - consensus opinion to accept the test, reject the test, or defer
deciding on the test while the issue is forwarded to the W3C for
clarification
[Q: What does this mean
A:	]

        - possible actions:
        - reject test and update errata and errata exclusion
[Q: What does it mean to reject a test?  In what form is rejection
communicated?
What is included in the rejection message?  What errata?  What exclusion?
A:	]

          - reject comment with advice to go to W3C if the submitter is not
convinced
[Q: What does this mean "reject comment"?  Who advises the submitter?
Convinced
of what?
A:	]

          - forward to W3C for clarification
[Q: What is forwarded?  By whom?  Clarification of what?
A:	]

      - accommodate external comment from the community at large
[Q: Who will make this accommodation?  How?  Comment on what?  Who is the
community-at-large (specifically)?
A:	]

        - committee publishes consensus opinion of response to comment with
justification from Recommendation (not just precedence of how a processor
has acted)
[Q: none]

[Q: What else should be included in the Review Policy?
A:      ]


  3 - game plan for tests
      - a member will report to the list the hierarchy of tests undertaken
for comparison with multiple processors
      - tally of tests will tracked on a visible web page for the committee
      - members report that all tests in a given hierarchy have been
examined, incl. a summary of findings of tests not to be included in the
resulting suite
      - a given hierarchy is not considered complete until reports from at
least two members have been submitted

  4 - public review
      - initial suite of a very small set of files will be used to test
procedures and scripts and stylesheets
      - committee will publish draft work periodically, starting with very
small set
      - committee will solicit comments on usability of the product
      - committee will publish a disposition of comments
      - committee progresses on the testing of files until all hierarchies
covered


~~~~~~~~~~~~~~~~~~~~~~~
Crisman Cooley
Overdomain, LLC
ccooley@overdomain.com
805-683-0938 tel
805-570-5474 cel
www.overdomain.com
~~~~~~~~~~~~~~~~~~~~~~~