[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [sca-assembly] A New SCA Testing TC?
Martin: Why do we need b) ? If they show the test suite and say they pass all the language-dependent and language-independent tests, isn't that sufficient? All the best, Ashok Martin Chapman wrote: > > On the MS/Siemens issue, how about we change the conformance > requirements in the assembly spec to the effect of a) and b) in your > email. > > Currently we say in Section 13.2 > > … > > 3.The implementation MUST support and comply with at least one of the > OpenCSA Member Section adopted implementation types. > > How about replacing this with (something like): > > 3. The implementation MUST support and comply with a publically > available implementation type. A publically available implementation > type MUST: > > a) Have a publicly available version of the test suite written in the > implementation type, available for any party to download, validate and > execute > > b) Have a publicly available document(s) describing the implementation > type, which, at a minimum, describes the mapping from implementation > type artifacts to the componentType when the implementation artifact > is used as the implementation of an SCA Component. > > Note that OpenSCA Member Section adopted implementation types fulfill > these requirements. > > *From:* Mike Edwards [mailto:mike_edwards@uk.ibm.com] > *Sent:* 17 July 2009 10:01 > *To:* OASIS Assembly > *Subject:* [sca-assembly] A New SCA Testing TC? > > > Folks, > > Bob's points below really go into the area of the responses that we > owe Microsoft and Siemens > relating to their public review comments on the Assembly spec. > > It is my belief that the Assembly tests have been structured in such a > way that they are adaptable > to a any new future implementation type, although I agree that the > TestCases document needs to > be more explicit and normative about which parts of the testcases can > be changed and what > changes are allowed. > > The problem remains however, about how we can ensure that a vendor > implementing a new > implementation type and wanting to claim conformance for their runtime > using that implementation > type satisfies 2 key requirements: > > a) Have a publicly available version of the test suite written in the > new implementation type, > which can be used by all and which would be subject to scrutiny for > the validity of the translation > of the testcases. > > b) Have a publicly available document describing the new > implementation type which at a > minimum describes the mapping from implementation type artifacts to > the componentType > when the implementation artifact is used as the implementation of an > SCA Component. > > > So far, the only route we have of guaranteeing both of these > requirements is to require that > the new implementation language is described by a specification > developed under the > aegis of an OASIS TC. > > I am interested to hear if there some alternatives available which > will provide these requirements > while allowing some alternative approach to the development and > publishing of the materials. > > > > I don't think that a "Testing TC" affects these questions much at all. > I tend to agree with Bob that > the value of a separate Testing TC is questionable and there remains > an issue relating to the > implementation type specific tests that an impl language TC may want > to require as part of the > conformance requirements for their spec. To me, these tests seem to > naturally belong to the > language TC itself. Having a separate TC with different membership > seems like a recipe for > conflict and/or missing testcases.... > > > Yours, Mike. > > Strategist - Emerging Technologies, SCA & SDO. > Co Chair OASIS SCA Assembly TC. > IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain. > Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431 > Email: mike_edwards@uk.ibm.com > > From: > > > > Bob Freund <bob.freund@hitachisoftware.com> > > To: > > > > ashok.malhotra@oracle.com > > Cc: > > > > OASIS Assembly Test <sca-assembly-testing@lists.oasis-open.org>, OASIS > Policy <sca-policy@lists.oasis-open.org>, OASIS Bindings > <sca-bindings@lists.oasis-open.org>, OASIS Java > <sca-j@lists.oasis-open.org>, OASIS BPEL > <sca-bpel@lists.oasis-open.org>, OASIS CPP > <sca-c-cpp@lists.oasis-open.org> > > Date: > > > > 16/07/2009 22:03 > > Subject: > > > > Re: [sca-policy] A New SCA Testing TC? > > ------------------------------------------------------------------------ > > > > > Ashok, > I am somewhat sympathetic to the argument that it ought to be possible > for a new language to be brought in to the family, but I am concerned > about creating a centralized testing TC. > One main reason is that it is unclear how long such a TC would be > staffed or its members remain interested. A new language could come > along years from now. > IMO it would be better to describe the language interface tests in a > language independent manner so that they could be implemented in > whatever language as may arise in the future. > The problem is the determination of the correct implementation of > those "meta" tests. Perhaps there is some technical solution, but it > escapes my limited imagination. > an alternate approach might be that the implementors might self- > certify and be required to publish the tests used as well as the > results before being able to claim conformance. The customer's might > them be able to judge for themselves the degree of rigor used and thus > the quality of the self-certification. > -bob > > On Jul 15, 2009, at 4:42 PM, ashok malhotra wrote: > > > If you have been following the Assembly Testing work, you know that > > the Assembly Test Cases are written in Java. > > Mike is now preparing a BPEL version. > > > > Clearly, the Java test cases test the Java C&I to some extent and > > Bryan has raised an issue that asks whether the Assembly and Java > > tests should be more cleanly separated, but there are > > complications. The Assembly tests need some C&I to test and some > > duplication cannot be prevented. Also, if the Assembly tests and > > the language tests were separated, then who adjudicate differences > > of opinion and duplication? Further, if the Assembly tests were > > shorn of any language C&I, someone could come along, pass only the > > Assembly, Policy and Bindings tests and claim SCA compliance. > > > > So, I'm suggesting we consider forming a new TC that would be > > responsible for all SCA Testing. > > Before you go "Aaargh! Not another TC!", please consider the > > advantages: > > > > 1. Bryan's issue would be closed as there would be only a single TC > > and it would write and manage all the tests, some > > of which would, of course, cover Assembly and one or more language > > C&Is. In fact, thinking about it, many of the tests > > would cover some Assembly features, some Policy, a binding and some > > C&I. > > > > 2.. If all the tests were in a single TC there would be no need for > > someone else to settle disputes. > > > > 3. If an outside party were to come and claim SCA compliance, the > > Testing TC would have the authority to vet their > > tests and say 'yay' or 'nay'. Or to enforce which tests they should > > run. > > > > What do people think? > > -- > > All the best, Ashok > > > > --------------------------------------------------------------------- > > To unsubscribe from this mail list, you must leave the OASIS TC that > > generates this mail. Follow this link to all your TCs in OASIS at: > > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php > > > > > > ------------------------------------------------------------------------ > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with > number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 > 3AU/ > > > > >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]