Re: [sca-assembly] A New SCA Testing TC?

[ashok malhotra <ashok.malhotra@oracle.com>]

Martin:
Why do we need b) ?

If they show the test suite and say they pass all the language-dependent 
and language-independent tests, isn't that sufficient?
All the best, Ashok


Martin Chapman wrote:
>
> On the MS/Siemens issue, how about we change the conformance 
> requirements in the assembly spec to the effect of a) and b) in your 
> email.
>
> Currently we say in Section 13.2
>
> …
>
> 3.The implementation MUST support and comply with at least one of the 
> OpenCSA Member Section adopted implementation types.
>
> How about replacing this with (something like):
>
> 3. The implementation MUST support and comply with a publically 
> available implementation type. A publically available implementation 
> type MUST:
>
> a) Have a publicly available version of the test suite written in the 
> implementation type, available for any party to download, validate and 
> execute
>
> b) Have a publicly available document(s) describing the implementation 
> type, which, at a minimum, describes the mapping from implementation 
> type artifacts to the componentType when the implementation artifact 
> is used as the implementation of an SCA Component.
>
> Note that OpenSCA Member Section adopted implementation types fulfill 
> these requirements.
>
> *From:* Mike Edwards [mailto:mike_edwards@uk.ibm.com]
> *Sent:* 17 July 2009 10:01
> *To:* OASIS Assembly
> *Subject:* [sca-assembly] A New SCA Testing TC?
>
>
> Folks,
>
> Bob's points below really go into the area of the responses that we 
> owe Microsoft and Siemens
> relating to their public review comments on the Assembly spec.
>
> It is my belief that the Assembly tests have been structured in such a 
> way that they are adaptable
> to a any new future implementation type, although I agree that the 
> TestCases document needs to
> be more explicit and normative about which parts of the testcases can 
> be changed and what
> changes are allowed.
>
> The problem remains however, about how we can ensure that a vendor 
> implementing a new
> implementation type and wanting to claim conformance for their runtime 
> using that implementation
> type satisfies 2 key requirements:
>
> a) Have a publicly available version of the test suite written in the 
> new implementation type,
> which can be used by all and which would be subject to scrutiny for 
> the validity of the translation
> of the testcases.
>
> b) Have a publicly available document describing the new 
> implementation type which at a
> minimum describes the mapping from implementation type artifacts to 
> the componentType
> when the implementation artifact is used as the implementation of an 
> SCA Component.
>
>
> So far, the only route we have of guaranteeing both of these 
> requirements is to require that
> the new implementation language is described by a specification 
> developed under the
> aegis of an OASIS TC.
>
> I am interested to hear if there some alternatives available which 
> will provide these requirements
> while allowing some alternative approach to the development and 
> publishing of the materials.
>
>
>
> I don't think that a "Testing TC" affects these questions much at all. 
> I tend to agree with Bob that
> the value of a separate Testing TC is questionable and there remains 
> an issue relating to the
> implementation type specific tests that an impl language TC may want 
> to require as part of the
> conformance requirements for their spec. To me, these tests seem to 
> naturally belong to the
> language TC itself. Having a separate TC with different membership 
> seems like a recipe for
> conflict and/or missing testcases....
>
>
> Yours, Mike.
>
> Strategist - Emerging Technologies, SCA & SDO.
> Co Chair OASIS SCA Assembly TC.
> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
> Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
> Email: mike_edwards@uk.ibm.com
>
> From:
>
> 	
>
> Bob Freund <bob.freund@hitachisoftware.com>
>
> To:
>
> 	
>
> ashok.malhotra@oracle.com
>
> Cc:
>
> 	
>
> OASIS Assembly Test <sca-assembly-testing@lists.oasis-open.org>, OASIS 
> Policy <sca-policy@lists.oasis-open.org>, OASIS Bindings 
> <sca-bindings@lists.oasis-open.org>, OASIS Java 
> <sca-j@lists.oasis-open.org>, OASIS BPEL 
> <sca-bpel@lists.oasis-open.org>, OASIS CPP 
> <sca-c-cpp@lists.oasis-open.org>
>
> Date:
>
> 	
>
> 16/07/2009 22:03
>
> Subject:
>
> 	
>
> Re: [sca-policy] A New SCA Testing TC?
>
> ------------------------------------------------------------------------
>
>
>
>
> Ashok,
> I am somewhat sympathetic to the argument that it ought to be possible
> for a new language to be brought in to the family, but I am concerned
> about creating a centralized testing TC.
> One main reason is that it is unclear how long such a TC would be
> staffed or its members remain interested. A new language could come
> along years from now.
> IMO it would be better to describe the language interface tests in a
> language independent manner so that they could be implemented in
> whatever language as may arise in the future.
> The problem is the determination of the correct implementation of
> those "meta" tests. Perhaps there is some technical solution, but it
> escapes my limited imagination.
> an alternate approach might be that the implementors might self-
> certify and be required to publish the tests used as well as the
> results before being able to claim conformance. The customer's might
> them be able to judge for themselves the degree of rigor used and thus
> the quality of the self-certification.
> -bob
>
> On Jul 15, 2009, at 4:42 PM, ashok malhotra wrote:
>
> > If you have been following the Assembly Testing work, you know that
> > the Assembly Test Cases are written in Java.
> > Mike is now preparing a BPEL version.
> >
> > Clearly, the Java test cases test the Java C&I to some extent and
> > Bryan has raised an issue that asks whether the Assembly and Java
> > tests should be more cleanly separated, but there are
> > complications. The Assembly tests need some C&I to test and some
> > duplication cannot be prevented. Also, if the Assembly tests and
> > the language tests were separated, then who adjudicate differences
> > of opinion and duplication? Further, if the Assembly tests were
> > shorn of any language C&I, someone could come along, pass only the
> > Assembly, Policy and Bindings tests and claim SCA compliance.
> >
> > So, I'm suggesting we consider forming a new TC that would be
> > responsible for all SCA Testing.
> > Before you go "Aaargh! Not another TC!", please consider the
> > advantages:
> >
> > 1. Bryan's issue would be closed as there would be only a single TC
> > and it would write and manage all the tests, some
> > of which would, of course, cover Assembly and one or more language
> > C&Is. In fact, thinking about it, many of the tests
> > would cover some Assembly features, some Policy, a binding and some
> > C&I.
> >
> > 2.. If all the tests were in a single TC there would be no need for
> > someone else to settle disputes.
> >
> > 3. If an outside party were to come and claim SCA compliance, the
> > Testing TC would have the authority to vet their
> > tests and say 'yay' or 'nay'. Or to enforce which tests they should
> > run.
> >
> > What do people think?
> > --
> > All the best, Ashok
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this mail list, you must leave the OASIS TC that
> > generates this mail. Follow this link to all your TCs in OASIS at:
> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>
>
>
>
>
> ------------------------------------------------------------------------
>
> /Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
> 3AU/
>
>
>
>
>