RE: [tag] Groups - TA Anatomy V0.5 (AnatomyTA-v05.doc) uploaded

[Lynne Rosenthal <lsr@nist.gov>]

I agree with Dave (see a below) 

In summary:
a) Regardless of whether it is a positive or negative test - a pass is a
pass; a fail is a fail.  
(note: we need to be clear regarding the results of the negative test in
that the behavior being tested causes an error or failure, but that is
what is expected to happen, thus, the test passes)

b) If an IUT passes all the tests - then the best we can say is that
there were no failures.  Since testing was probably falsification
testing, you can't prove the IUT correct. However, failing a test proves
that it isn't correct. BUT, if the conformance clause or the testing
organization allows for partial conformance, then the door is wide open
with respect to failing a test and what it means for the IUT.  

Thus, with respect to the TA, we should stay away from making statements
like (b) - there are other factors (e.g., conformance clauses, the
organization doing the testing, certification, etc) that provide the
rules for an IUT passing or failing a set of tests. 

--Lynne


> -----Original Message-----
> From: Dave Pawson [mailto:dave.pawson@gmail.com]
> Sent: Wednesday, October 03, 2007 8:41 AM
> To: tag@lists.oasis-open.org
> Subject: Re: [tag] Groups - TA Anatomy V0.5 (AnatomyTA-v05.doc)
uploaded
> 
> On 03/10/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk>
> wrote:
> >
> > Quoting Dave Pawson <dave.pawson@gmail.com>:
> >
> >
> > > If it doesn't fail, it may not pass.
> > > If it doesn't pass, it may not have failed.
> 
> This is an interpretation from the Oasis site.
> 
> > >
> > > It should be a boolean. If it hasn't passed it has failed.
> > > If it has passed all the tests, it has passed.
> > > Period.
> 
> My view
> 
> > >
> >
> > But I think we have to accept it that you *can't assume* the
> > converse - that if it hasn't failed any of the tests it has
> > passed.
> 
> I don't say that Steve. "If it has passed all the tests, it has
passed"
> Thats the only sensible option for a pass.
> 
> The tests may all have been negative tests designed
> > to find faults but it is quite likely that not failing any
> > of them will still not mean a pass as such.
> 
> Pos or neg is immaterial. A test passes or fails.
> ANDing them gives the overall UUT result.
> 
> 
> > It will just mean
> > that no failure has been detected.
> 
> If a test hasn't passed (whether it ran or not) it is deemed
> to have failed in my books.
> 
> Why make it more complex than that.
> 
> 
> regards
> 
> 
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk