RE: Revised Review Policy

[Crisman Cooley <ccooley@overdomain.com>]

>> Revised Review Policy
> David


> -----Original Message-----
> From: David_Marston@lotus.com [mailto:David_Marston@lotus.com]
> Sent: Monday, September 17, 2001 3:17 PM
> To: xslt-conformance@lists.oasis-open.org
> Subject: Re: Revised Review Policy
>
>
> Assorted comments:
>
> >...All accepted tests are intended to test conformance; when a processor
> >can fail the test and still produce the anticipated result, that test
> >should be excluded....
>
> Um, that looks twisted around, since we normally judge failure by the
> result produced. Did you mean that we neutralize all permissible
> variations in the result, such as order of attributes, so that any
> conformant processor will produce an output that comes "acceptably
> close" to our reference result?

I don't recall the genesis of that sentence; but it's meaning doesn't seem
especially clear to me either.  I suggest we strike everything after the
semi-colon and replace it with "to the Specification on the basis of
output."

>
> >To the extent possible, Committee Reviewers should remove
> >tests exhibiting interpretive behaviors.
>
> How about: ...tests whose reference result constitutes interpretation
> of the spec, unless the test is cataloged with a Committee-approved
> gray-area designation.

Yes, that sounds like an improvement in clarity.

>
> >1. At least two Reviewers will check off on each test.  Only the
> >assessment of a single member is required for the test to be included
> >in the draft release.
>
> Is it: one reviewer's Accept is sufficient? One reviewer's Reject or
> remand is sufficient to cause exclusion, but two Accepts to accept?

I don't believe that has been determined yet.  Sounds like another Committee
call.  I vote to leave the sentence appropriately vague until we know
specifically how it works.

>
> >7.3 Reviewers report that all tests in a given hierarchy have been
> >examined, including a summary of findings of tests not to be included
> >in the resulting suite.
>
> As mentioned in earlier 1-1 email, I think our system should allow
> each reviewer to sent a batch of decisions (in XML?) after each
> time they do some reviewing. In other words, piecemeal updates.

Yes, that makes sense.  I'll change it so we're not appearing to lock
Reviewers into examining all tests at once.  I don't think that was the
intent.

>
> >8.5 The Reviewers will continue testing the files until all the
> >hierarchies are covered.
>
> I think it's "reviewing test cases until all categories are covered."
> .................David Marston

Thanks, I'll change the wording.

>
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>