Re: Tracking system

[David_Marston@lotus.com]

In thinking about test review, I've observed some situations and issues
that we need to represent in our tracking system.

When a submission is received, we will date-stamp it as discussed in
the thread about catalog design. It would be very helpful to take the
catalog as tweaked by OASIS and generate entries/records in the test
tracking system, starting each test in the "received" state.

Chunks of the test suite will then be assigned to reviewers. Each test
may be determined by each reviewer to have a status of accept (can be
included in the OASIS test suite), reject (is not really a conformance
test), defer, and possibly others. From our last teleconference, it
occurred to me that "defer" status can happen for two orthogonal
reasons. The obvious one is that we are only looking at xsl:number
cases at the moment, so if a test comes in but actually does not use
xsl:number, we would defer it. (Some of the 4xslt tests fall in this
category.) The other dimension is use of xsl:number for languages
that we are not ready to include in the test suite, or using external
parameter setting. Thus, the tracking system may want to have data
that tracks reasons for deferral, allowing us to find all the tests
deferred for a particular reason when we are ready to expand the
capabilities of the test system.

Do we also want to track the "remanded" status when a test case has
been earmarked for repair by the submitter?
.................David Marston