OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

tag-discuss message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: RE: [tag-discuss] Terminology


I think we have a thread here that needs to be pursued to produce an
agreeable definition of "Test Assertion" (or test specification).
I think we are close :-)
Quite important for an unambiguous definition of a TC deliverable /
charter scope.
So I renamed the subject of the thread "terminology". 

Dave:

Not sure  I get the difference between test assertion (TA) vs. test
specification, if there is any. Seems we are talking of the same thing,
though not sure yet.
Is there any definition somewhere for " test specification", to compare?
I suspect TA is more commonly used, though. What do other think?

And [at least some] of your "nitpicking" makes sense ;-)
Yes, a TA is targeting an implementation, should make that clearer. 

Regards,
-Jacques





-----Original Message-----
From: Dave Pawson [mailto:dave.pawson@gmail.com] 
Sent: Tuesday, November 28, 2006 12:05 AM
To: tag-discuss@lists.oasis-open.org
Subject: Re: [tag-discuss] the role of an XML mark-up for TAs

I hope I'm not nitpicking.

On 27/11/06, Durand, Jacques R. <JDurand@us.fujitsu.com> wrote:
> > 3. Test cases model and metadata.
> At least this is what I captured from recent exchanges between David
M.
> (11/23) and Abbie, but I may have conjectured too much.

For my money, its too much of an assumption that metadata will be used.
Meta to what? The test? No.
If you say metadata, many will presume an XML solution. Is that
what we're assuming?


>
> Agree with you that is hard to see within scope other than as
examples. I'm
> ready to write off (3) anytime.

I didn't understand it, so I can't comment.

>
>
>
> Probably we should be more precise when we talk of:
>
> (a)- test assertion
>
> (b)- test case
>
> (c) - test

Yes, very precise for a spec :-)

>
>
>
> I suggest we avoid saying just "test".
The word is widely understood, but I'd need a context before deciding
on appropriateness?


> For (a) and (b), Here are some definitions that I have borrowed from a
few
> sources:
>
>
>
> Test Assertion: a Test Assertion is a statement of behavior for an
> implementation: action or condition that can be measured or tested. It
is
> derived from the specification's requirements and bridges the gap
between
> the narrative of the specification and the test cases. Each test
assertion
> is an independent, complete, testable statement on how to verify that
an
> implementation satisfies a conformance requirement.

Since (I assume) we're talking about specifications, proposed mod:


> Test Specification: a Test Specification is a statement of expected
output for an
> implementation: an action or condition that can be measured or tested.
It relates
> the specification and the test cases. Each test assertion
> is (as far as possible) an independent, complete, testable statement
> of how to verify that an implementation satisfies a requirement.

Though when I re-read that, it reads as a test specification!
I was required to write the test specification and test planning matrix
directly
from the system specification (or requirement), which would be the Oasis
spec in this case. Mmm. The test case actually uses the phrase test
spec.
I've updated my definition to reflect that and it reads OK to me. What
do you think?


> Test Case: Consists of a set of a test tool(s), software or files
(data,
> programs, scripts, or instructions for manual operations) that checks
a
> particular requirement in the specification to determine whether the
results
> produced by the implementation match the expected results, as defined
by the
> specification. Typically, a Test Case exercises (and is derived from)
one or
> more test assertions. Each Test Case includes: (1) a description of
the test
> purpose (what is being tested - the conditions / requirements /
capabilities
> which are to be addressed by a particular test, (2) the pass/fail
criteria,
> (3) a reference to the requirement or section in the standard from
which the
> test case is derived (traceability back to the specification), or to
related
> test assertion(s).

Oh dear nit-picking again. I don't like 'checks a particular
requirement in the spec'
if what is being tested is the implementation?
Which is it to be? I think the implementation is the realisation of the
spec,
so that is what can be tested; our aim is to make sure that the spec
author
reviews the spec to ensure a test spec can be written? That helps to
make
sure the implementation can be tested against the spec.
Another re-write. Please don't presume I'm knocking the input. I liked
it.


 A single test: Consists of a set of a test tool(s), software (data,
 programs, scripts, or instructions for manual operations) that tests an
implementation for compliance to a single requirement in the
specification.
The result of the test is checked against the expected result, as
defined by the
 specification. Typically, a Test Case exercises (and is derived from)
a  test specification
item. Each Test includes:

(1) a description of the test purpose (what is being tested
   - the conditions / requirements / capabilities which are to be
addressed
  by a particular test,

(2) the pass/fail criteria,
(3) A reference to the test specification paragraph being implemented.
(4) a reference to the requirement or section in the specification
from which the
 test is derived (traceability back to the specification), or to related
 test assertion(s).


That does it for me. Is it an improvement? items 3 and 4 provide the
test matrix
usable to ensure coverage of the specification.


Thanks Jacques

That clears it up for me.




-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]