RE: [tag] Groups - TA Anatomy V0.5 (AnatomyTA-v05.doc) uploaded

[stephen.green@systml.co.uk]

I agree that the conformance clause would make sense of the
test outcome, although the chances are that the conformance
clauses will have been written prior to the test assertions,
despite our urging the contrary. (The maturity necessary for
writing TAs might come after a few early adoptions and after
the standardization.) I just think we need to be careful not
to preclude anything a conformance clause might want to say
about how a set of (existing or future) TAs are interpreted.
I think that might mean not casting it in stone that the
outcome should address *only* pass and fail. I think the
TA Anatomy as it stands in v0.5 does cater for this factor.

Steve


Quoting Lynne Rosenthal <lsr@nist.gov>:

> I agree with Dave (see a below)
>
> In summary:
> a) Regardless of whether it is a positive or negative test - a pass is a
> pass; a fail is a fail.
> (note: we need to be clear regarding the results of the negative test in
> that the behavior being tested causes an error or failure, but that is
> what is expected to happen, thus, the test passes)
>
> b) If an IUT passes all the tests - then the best we can say is that
> there were no failures.  Since testing was probably falsification
> testing, you can't prove the IUT correct. However, failing a test proves
> that it isn't correct. BUT, if the conformance clause or the testing
> organization allows for partial conformance, then the door is wide open
> with respect to failing a test and what it means for the IUT.
>
> Thus, with respect to the TA, we should stay away from making statements
> like (b) - there are other factors (e.g., conformance clauses, the
> organization doing the testing, certification, etc) that provide the
> rules for an IUT passing or failing a set of tests.
>
> --Lynne
>
>
>> -----Original Message-----
>> From: Dave Pawson [mailto:dave.pawson@gmail.com]
>> Sent: Wednesday, October 03, 2007 8:41 AM
>> To: tag@lists.oasis-open.org
>> Subject: Re: [tag] Groups - TA Anatomy V0.5 (AnatomyTA-v05.doc)
> uploaded
>>
>> On 03/10/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk>
>> wrote:
>> >
>> > Quoting Dave Pawson <dave.pawson@gmail.com>:
>> >
>> >
>> > > If it doesn't fail, it may not pass.
>> > > If it doesn't pass, it may not have failed.
>>
>> This is an interpretation from the Oasis site.
>>
>> > >
>> > > It should be a boolean. If it hasn't passed it has failed.
>> > > If it has passed all the tests, it has passed.
>> > > Period.
>>
>> My view
>>
>> > >
>> >
>> > But I think we have to accept it that you *can't assume* the
>> > converse - that if it hasn't failed any of the tests it has
>> > passed.
>>
>> I don't say that Steve. "If it has passed all the tests, it has
> passed"
>> Thats the only sensible option for a pass.
>>
>> The tests may all have been negative tests designed
>> > to find faults but it is quite likely that not failing any
>> > of them will still not mean a pass as such.
>>
>> Pos or neg is immaterial. A test passes or fails.
>> ANDing them gives the overall UUT result.
>>
>>
>> > It will just mean
>> > that no failure has been detected.
>>
>> If a test hasn't passed (whether it ran or not) it is deemed
>> to have failed in my books.
>>
>> Why make it more complex than that.
>>
>>
>> regards
>>
>>
>> --
>> Dave Pawson
>> XSLT XSL-FO FAQ.
>> http://www.dpawson.co.uk
>
>
>


-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice