OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

sca-assembly message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [sca-assembly] Concrete Exit Criteria for the SCA Assembly TC - Proposal

On Feb 23, 2011, at 3:06 PM, Martin Chapman wrote:

From: Jim Marino [mailto:jim.marino@gmail.com] 
Sent: 23 February 2011 19:35
To: OASIS Assembly
Subject: Re: [sca-assembly] Concrete Exit Criteria for the SCA Assembly TC - Proposal
Hi Martin,
I guess I may now be even more confused, probably because of my ignorance of OASIS rules. Why would the exit criteria for the TC require an implementation to adhere to something that is not normative, specifically running and passing the TC-supplied test suite?
<MC> Because that is why we have been writing a test suite in the TC. The intention is that two indepe! ndent runtimes pass the TCís test suite. That way we demonstrate that the spec  has been independently  implemented and that the test suite isnít (too) specific to a particular runtime
Also, who determines whether a runtime passes or not?
<MC> the Test suite, saying whether a particular test passes or not?

Hi Martin,

I think you raise two important but separate issues here: (1) demonstrating the spec has been independently implemented; and (2) that the test suite is too tailored to a specific runtime.

Regarding (1), I thought it was sufficient for an entity responsible for a runtime to declare in some public and documented way that it conforms to an SCA specification.  The entity could decide to use the TC tests, adapt them, or write their own. Policing conformance would be left to the court of public opinion.  Is it now a requirement that a runtime passes the TC-supplied test suite to claim conformance? If so, what does an entity need to do to pass the tests? Is it a requirement that the entity publish test artifacts, including source and/or binaries of the integration code required to plug the runtime into the test suite? Or can they just say they do? Are they allowed to modify the test cases to fit them into their own integration testing infrastructure? 

My concern, similar to Eric's, is that we are going down the path of over-specification. 

Regarding (2), my understanding was the test suite was non-normative and similar to a set of "guidelines". If that is the case and the suite is too tailored to a particular runtime or set of runtimes, so be it. There is always the option of taking the test suite as inspiration and developing one more suited to a particular platform. For example, if I am developing a non-Java runtime, I am likely to want to develop a test suite using the build and CI tools currently in use. I wouldn't want to be forced to write Java code that integrates my non-Java runtime into the test suite and have to run the JDK-based test client. 


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]