[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [sca-assembly] Issue 101: Complete the Conformance Section -Comments on Proposal
> Back to answer Martin's question of whether the runtime is right or whether > the document is right when some failure occurs. If the SCA runtime passes > our tests then the first reaction should be to say that the runtime is > right and the document is incorrect. If this isn't the case, it actually > shows a hole in our SCA runtime tests - go plug the hole!! > It is not possible to plug such a hole. Such failure could mean: 1) the runtime is non-conformant OR 2) the doc is non-conformant OR 3) both are non-conformant OR 4) both are conformant (the runtime just doesn't support the things in the document) We can't be writing test for every possible eventuality, but we can aim for a large coverage. -Anish -- Mike Edwards wrote: > > Anish, > > Reply inline... > > Yours, Mike. > > Strategist - Emerging Technologies, SCA & SDO. > Co Chair OASIS SCA Assembly TC. > IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain. > Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431 > Email: mike_edwards@uk.ibm.com > > Anish Karmarkar <Anish.Karmarkar@oracle.com> wrote on 03/03/2009 07:46:07: > > Re: [sca-assembly] Issue 101: Complete the Conformance Section - > > Comments on Proposal > > Mike Edwards wrote: > > > > > > Folks, > > > > > > Some comments on the proposal: > > > > > > > > > 1) I think that making a series of SCA-related documents into > > > conformance points, as per section 12.1, is unnecessary and unwise. > > > > > > The only point of making these documents into conformance points is if > > > it is intended to write testcase(s) that will validate those > > > documents. I do not believe that we have the resources to write such > > > testcases and as a result, the conformance demands made > > > here are a waste of time and effort. > > > > > > > I would like to understand what additional tests will have to be written. > > There are already requirements on SCA runtime regarding rejecting what I > > would call non-conformant SCA documents. To test that we would have to > > create non-conformant documents, use it with a runtime and ensure that > > the runtime rejects them. What additional test artifact would have to be > > created wrt document conformance? I.e., we have a set of documents that > > we assert are non-conformant. We of course would have a set of > > conformant documents (for the positive test for a runtime). > > > > I can see us not wanting to create non-conformant (or conformant) SCA > > documents for every little feature and element/attribute that schema > > allows. But we are now talking about how detailed our test suite is > > going to be. Given that there is a requirement for runtimes to reject > > invalid SCA documents, this is already an issue. I don't think we need > > to Turing complete. > > > > The tests you describe above are simply tests of an SCA runtime - and we > have > these already, including "invalid documents". However, they only check on > the valid behaviour of an SCA runtime. > > If we make claims about documents themselves being valid or invalid, it is > my opinion that such claims are pretty worthless unless there are tests to > back them up. Such tests would actually have to be able to accept some > random document and evaluate whether that document was a valid document or > not. I even heard one person on a previous call ask for a tool of exactly > this kind. > > At the moment I don't think it is worth the time and effort of building > tests that could validate documents. As a result, I take the view that > there is little to be gained in having documents as conformance points. > The specification concentrates on what an SCA runtime does with a document > and we have the tests to check out that the runtimes are doing the right > things. > > Back to answer Martin's question of whether the runtime is right or whether > the document is right when some failure occurs. If the SCA runtime passes > our tests then the first reaction should be to say that the runtime is > right and the document is incorrect. If this isn't the case, it actually > shows a hole in our SCA runtime tests - go plug the hole!! > > > > -Anish > > -- > > > <snip> > > > ------------------------------------------------------------------------ > > / > / > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > > >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]