1. For each
feature of the target specification (e.g. each sub-section of its document),
we
need to tell people how well these are covered by our set of
Test Requirement items.
Indeed, test
reqs may not cover fully a spec feature due to the constraints associated
with testing,
difficulty to reproduce all
situations,
etc.
So
Mike and I started to do that with
by associating this coverage measure with the
Test Reqs
themselves - but the point is, that does not
work well in some cases where
there is not a 1-to-1 mapping between test Reqs and
spec features.
(e.g. the same spec sub-section may be fully
covered by three Test Reqs, yet each
would
participate only partially into
the verification of the features inside the
sub-section.
Or, the same Test Req might actually address two
different spec features, differently.)
2. One
solution is to associate this
coverage indicator
with
the spec features
themselves.
(Please see in the latest version of
ebXMLTestFramework doc
that we posted on our site,
the description in Section 4.3 "Specification Coverage".)
If we agree on this,
that means:
- we should remove
the "coverage" field in current Test Reqs document, and
instead provide a
list of all sections/subsections numbers of the MS spec,
and for each,
specify the degree of coverage we estimate our Test Reqs
provide.
- so, when we communicate our Test Requirements to the spec
owners (or spec originators)
for review, we provide a document with two sections:
(a) Test Reqs
(without coverage element),
(b) List of spec sections / sub-sections
reference numbers with, for each ,
(b1) the list of Test Reqs that cover it,
and (b2) how well
the section spec requirements
are covered by these (full, partial,
none).
Let us know if any problem with
that.
Jacques