Re: [tag-discuss] the role of an XML mark-up for TAs

["Dave Pawson" <dave.pawson@gmail.com>]

I hope I'm not nitpicking.

On 27/11/06, Durand, Jacques R. <JDurand@us.fujitsu.com> wrote:
> > 3. Test cases model and metadata.
> At least this is what I captured from recent exchanges between David M.
> (11/23) and Abbie, but I may have conjectured too much.

For my money, its too much of an assumption that metadata will be used.
Meta to what? The test? No.
If you say metadata, many will presume an XML solution. Is that
what we're assuming?


>
> Agree with you that is hard to see within scope other than as examples. I'm
> ready to write off (3) anytime.

I didn't understand it, so I can't comment.

>
>
>
> Probably we should be more precise when we talk of:
>
> (a)- test assertion
>
> (b)- test case
>
> (c) – test

Yes, very precise for a spec :-)

>
>
>
> I suggest we avoid saying just "test".
The word is widely understood, but I'd need a context before deciding
on appropriateness?


> For (a) and (b), Here are some definitions that I have borrowed from a few
> sources:
>
>
>
> Test Assertion: a Test Assertion is a statement of behavior for an
> implementation: action or condition that can be measured or tested. It is
> derived from the specification's requirements and bridges the gap between
> the narrative of the specification and the test cases. Each test assertion
> is an independent, complete, testable statement on how to verify that an
> implementation satisfies a conformance requirement.

Since (I assume) we're talking about specifications, proposed mod:


> Test Specification: a Test Specification is a statement of expected output for an
> implementation: an action or condition that can be measured or tested. It relates
> the specification and the test cases. Each test assertion
> is (as far as possible) an independent, complete, testable statement
> of how to verify that an implementation satisfies a requirement.

Though when I re-read that, it reads as a test specification!
I was required to write the test specification and test planning matrix directly
from the system specification (or requirement), which would be the Oasis
spec in this case. Mmm. The test case actually uses the phrase test spec.
I've updated my definition to reflect that and it reads OK to me. What
do you think?


> Test Case: Consists of a set of a test tool(s), software or files (data,
> programs, scripts, or instructions for manual operations) that checks a
> particular requirement in the specification to determine whether the results
> produced by the implementation match the expected results, as defined by the
> specification. Typically, a Test Case exercises (and is derived from) one or
> more test assertions. Each Test Case includes: (1) a description of the test
> purpose (what is being tested - the conditions / requirements / capabilities
> which are to be addressed by a particular test, (2) the pass/fail criteria,
> (3) a reference to the requirement or section in the standard from which the
> test case is derived (traceability back to the specification), or to related
> test assertion(s).

Oh dear nit-picking again. I don't like 'checks a particular
requirement in the spec'
if what is being tested is the implementation?
Which is it to be? I think the implementation is the realisation of the spec,
so that is what can be tested; our aim is to make sure that the spec author
reviews the spec to ensure a test spec can be written? That helps to make
sure the implementation can be tested against the spec.
Another re-write. Please don't presume I'm knocking the input. I liked it.


 A single test: Consists of a set of a test tool(s), software (data,
 programs, scripts, or instructions for manual operations) that tests an
implementation for compliance to a single requirement in the specification.
The result of the test is checked against the expected result, as defined by the
 specification. Typically, a Test Case exercises (and is derived from)
a  test specification
item. Each Test includes:

(1) a description of the test purpose (what is being tested
   - the conditions / requirements / capabilities which are to be addressed
  by a particular test,

(2) the pass/fail criteria,
(3) A reference to the test specification paragraph being implemented.
(4) a reference to the requirement or section in the specification
from which the
 test is derived (traceability back to the specification), or to related
 test assertion(s).


That does it for me. Is it an improvement? items 3 and 4 provide the test matrix
usable to ensure coverage of the specification.


Thanks Jacques

That clears it up for me.




-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk