RE: [tag] Groups - Intro Material for TA Guide (Rationale.html)uploaded

["Serm Kulvatunyou" <serm@nist.gov>]

-----Original Message-----
From: Dave Pawson [mailto:dave.pawson@gmail.com] 
Sent: Thursday, September 20, 2007 3:51 AM
To: jdurand@us.fujitsu.com
Cc: tag@lists.oasis-open.org
Subject: Re: [tag] Groups - Intro Material for TA Guide (Rationale.html)
uploaded

On 20 Sep 2007 00:31:23 -0000, jdurand@us.fujitsu.com
<jdurand@us.fujitsu.com> wrote:
> F2F presentation: Intro Material for TA Guide (Patrick Curran)
>
>  -- Mr Jacques Durand
>
> The document named Intro Material for TA Guide (Rationale.html) has 
> been submitted by Mr Jacques Durand to the OASIS Test Assertions 
> Guidelines
> (TAG) TC document repository.

Comments.
Each test assertion is an independent, complete, testable statement of
requirements in the specification.

The 'complete' bothers me. Either define it or remove it. With the other
words it may be unnecessary. It stands pretty well without it.

<serm> Not only the 'complete' bother me, but also the 'independent' because
we are talking about prerequisite. Maybe the right term is 'modular', but
maybe it is better to define whatever term we want to use, e.g., "change in
one TA does not require change in another TA". 'Testable' could be
problematic too because there is no test cases implemented yet, it is only
an anticipation. Maybe we could add a separate sentence that says "TA should
be written with sufficient information that executable test cases can be
derived.", but that may be too risky. I think "each test assertion is a
normative statement of requirements in the specification that assists other
activities in the testing process." can be sufficient. Acknowledging from
the discussion that we are going in the direction of allowing different
levels of formality of the test assertion. </serm>

Re benefits.

I'm uncomfortable with the relationship between coverage and mapping.
Classic of a high level requirement which needs 23 tests. Shown in the
coverage but has a one to many mapping, requirement to tests. Each test is
independent, but the coverage of the requirement isn't complete until all 23
have been run?
Does this need stating? Possibly not, but it is implicit.


Coverage analysis

Define coverage goals for each section.
Each section of what? The test suite? or the specification. Unclear as it
stands.

For each partition,
is this a section?

What percentage of assertions are covered by at least one test?
Should this be requirements? In the requirements document?

Need a way of making it clear which document is being spoken of.

<serm> Is it possible that we are talking about 2 different but connected
levels of coveraage - specification coverage by test assertions and test
assertion coverage by tests (test cases). </serm>


I can see where it's going, I've previously called this the test plan.
I think it needs re-work to aid clarity?

Generally a very useful document. IMHO due a place fairly high in the output
of this SC.

regards




--
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk