Think about updates when writing Submission Policy

[David_Marston@lotus.com]

In "Notes about name management..." I wrote >> and Ken replied >

>>14. If a submitter wants to update their test suite, which could mean
>>adding, dropping, or changing test cases, how do we want them to
>>catalog the revision?

>I think this can be deferred.  If we are asked to wholly replace an old
>submission with a new submission I would not want to do piecemeal
>fidgeting.  The only savings is not having to review the same test a
>second time, which we should be able to do with path and file names and
>the date/time of the test case (part of the tracking information of the
>review process).

Lotus reserves the right to organize our test suite differently in the
future. If you want our future submission to not force a review of tests
that are unchanged, tell us that review of test cases is burdensome
work and that you want data that will avoid such work where possible.

Even for the first submission, the Submission Policy should hint at the
idea that you hope to get future submissions, including repairs to
individual tests as well as massive expansions, and how you intend to
detect changed/unchanged in the massive submissions.

Which reminds me: we (Lotus) already use some of Carmelo's test cases
in our testing. The Submission Policy should also request that we
filter out tests that we know will be submitted by their creators.
.................David Marston