Re: Revised Review Policy

[David_Marston@lotus.com]

Assorted comments:

>...All accepted tests are intended to test conformance; when a processor
>can fail the test and still produce the anticipated result, that test
>should be excluded....

Um, that looks twisted around, since we normally judge failure by the
result produced. Did you mean that we neutralize all permissible
variations in the result, such as order of attributes, so that any
conformant processor will produce an output that comes "acceptably
close" to our reference result?

>To the extent possible, Committee Reviewers should remove
>tests exhibiting interpretive behaviors.

How about: ...tests whose reference result constitutes interpretation
of the spec, unless the test is cataloged with a Committee-approved
gray-area designation.

>1. At least two Reviewers will check off on each test.  Only the
>assessment of a single member is required for the test to be included
>in the draft release.

Is it: one reviewer's Accept is sufficient? One reviewer's Reject or
remand is sufficient to cause exclusion, but two Accepts to accept?

>7.3 Reviewers report that all tests in a given hierarchy have been
>examined, including a summary of findings of tests not to be included
>in the resulting suite.

As mentioned in earlier 1-1 email, I think our system should allow
each reviewer to sent a batch of decisions (in XML?) after each
time they do some reviewing. In other words, piecemeal updates.

>8.5 The Reviewers will continue testing the files until all the
>hierarchies are covered.

I think it's "reviewing test cases until all categories are covered."
.................David Marston