Re: [oiic-formation-discuss] Code walkthrough

["Tom M." <m42tom-odf@yahoo.com>]

--- robert_weir@us.ibm.com wrote:
>
>We can't prove conformance.  We can just test some test 
>cases, which is effectively just sampling.  We can 
>disprove conformance, when test cases fail.  But we 
>can never prove, by testing, that the application is 
>conforms when presented with all possible conforming 
>documents. 

We may never implement the perfect test suite.  We will
certainly not do it by focusing only on how the application
processes known good data.  It is just as important to 
verify that the output is good data.  It is also important
to know how it deals with bad data.

>There is an art to picking test cases such 
>that you will detect the maximum number of errors 
>caused by implementation defects.  This is known to 
>all QA practitioners.  But this assumes that errors 
>are unintentional.  Finding intentional errors, 
>especially when those making the errors know that you 
>are looking for them, this is something else entirely.

What sort of intentional errors are you thinking of, Rob?

If an application has a mode of operation that conforms
more strictly to the standard, presumably by not using
their own extensions, I'm not sure that is a bad thing.  I
would hope that there is a way that any user can invoke
that mode of operation easily.

IAC, reading code is never an adequate substitute for
well-designed tests.

-- 
Tom M.