OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

oiic-formation-discuss message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [oiic-formation-discuss] Code walkthrough


--- robert_weir@us.ibm.com wrote:
>
>We have an expression in the US, among school teachers,
>that mandatory testing leads to the practice of "teach 
>to the test".  In other words, if you know in advance 
>the contents of the test, then there is a risk of 
>teaching only that exact material and losing the broader
>context. 
> 
>If an implementation knows in advance the exact test
>cases that will be given it, it is possible for that 
>implementation to change their code so only those test 
>cases work, but still have a negligible effect on 
>improving interoperability...
> 
>I think this is why, in addition to the atomic testcases,
>we also need some real world complex documents that use 
>features in combination and exercise their interactions. 

+1

In order to test for interoperability, the test needs to be
very comprehensive.  IMO, we should start with areas where
we have seen problems.  That is not enough to be able to
certify a program as being compliant, though.  That
requires extensive testing.

Designing adequate test data is not a simple task.

-- 
Tom M.




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]