Re: [cgmo-webcgm] Initial draft of test suite reviewer instructions.

[Lofton Henderson <lofton@rockynet.com>]

Hi Rob,

Looks good to me.  I think people who have test review assignments
can start using it to guide their reviews, even if we do some minor
tuning.

Just a couple comments...

1.) "should not invent new styles" -- at least, they shouldn't
diverge *arbitrarily* from previous tests of similar subject
matter.  There may be cases where a new style is in order (I don't
know now if there are any such).  So maybe append "without good
cause."

2.) I don't have a strong feeling for standardizing on a particular DTD
for HTML or XHTML (although I generally lean towards the latter.)  We
probably have a mixture now in 2.0.  (A spot check shows a new20
test uses HTML 4.0.)  Thoughts, anyone?

3.) The 2nd-last bullet is probably clear to everyone, but if not ...
what we mean is that the displayed result of running the test should have
sufficient information to interpret the results, and especially to judge
pass/fail, etc.  (I.e., ideally one should not have to consult
another document to interpret the result.)

4.) "tests should be atomic..." -- I would suggest
s/tests/tests or their subtests/.  We have allowed a single test to
step interactively though a sequence of a few atomic subtests in past
work.  That seems reasonable, as long as it is not taken to
extreme.

Thanks,
-Lofton.

At 05:07 PM 10/7/2008 -0600, Robert Orosz wrote:

> Hi all,
>
> Lofton and I had a discussion last Friday about
> what should be included in the test suite reviewer's guidelines. He also
> sent me a "Test Builder's Manual" which he wrote for the W3C
> SVG WG, and I've liberally plagiarized from that. Based on those things,
> I've written and attached an initial draft of test suite reviewer
> guidelines for WebCGM. My editorial comments and questions are
> highlighted with a yellow background.
>
> Comments and criticisms are welcomed.
>
> Rob