Re: [sca-assembly] Concrete Exit Criteria for the SCA Assembly TC- Proposal

[Anish Karmarkar <Anish.Karmarkar@oracle.com>]

On 2/28/2011 4:08 PM, Eric Johnson wrote:
>   I have a simple answer:
>
> On 2/28/11 3:17 PM, Anish Karmarkar wrote:
>> So what I'm wondering is: why would we not have adapting and passing
>> the test suite be a requirement to claiming conformance (necessary but
>> not sufficient condition) to the main spec? Wouldn't that keep
>> implementations and vendor's marketing departments honest? It always
>> happens that there are ambiguities and various interpretation of a
>> spec despite best efforts. The test suite, is a runnable artifact with
>> little doubt as to what one is supposed to do. Certainly, the test
>> suite won't cover everything and isn't intended to do so. But I think
>> a as close to an iron-clad conformance criteria that we can get to,
>> the better off we are wrt ensuring portability and interop.
>
> My take is simple - the assembly spec in particular, is about a model
> for describing services. The test suite, as such, is simply a projection
> of that model coupled to a Java-based execution environment.

It isn't coupled to the Java-based execution environment. You can use 
any implementation type. It is adaptable to use C/C++/BPEL or anything else.

> As with all
> mappings some details are lost, and some are gained as part of that
> projection. Do you know which details those are? I certainly don't.
>
> For me to understand the answer to that question, I would want a
> significant chunk of time, and I expect the same for others in TC, and
> what would be the added value? This extra level of conformance would add
> to the "marketplace" a binary flag for conformance/non-conformance? But
> who cares? For the near term, SCA end users will likely trend towards
> medium to large corporations. It won't include random developers who are
> content to kick around Ruby or Python code, download open source
> projects, and cobble them together in whatever way works. In contrast to
> those small time developers, those medium and large corporations will
> come to their own conclusions about how important conformance to all of
> SCA might be. If you run the test suite, and get 90% conformance, is
> that good enough? It might be, if the runtime in question is 50% faster
> than all the others. So I think, if you raise the conformance bar to
> include the test suite, it *still* won't be a binary answer.
>
> In addition, conforming to the spec means looking at a very limited set
> of conformance claims. Looking at the test suite, how do I know whether
> a line of code is considered "normative". What if it happens to break in
> my runtime? The authority for deciding whether the test is incorrect
> /already/ comes from the existing spec. What if a newer version of Java
> breaks some aspect of the test suite? What does that even mean?
>
> We know that Microsoft asked us to jump through hoops so that they could
> claim conformance with an implementation type not based on Java. And we
> agreed to that. Are you suggesting we go back on that?
>

No. They can adapt the test suite to their needs. All the top-level 
composites in the test suite use implementation.composite. You can use 
any impl type in the lower-level composites as long as it has the same 
component type needed by the components in the top-level composites.

-Anish
--

> Now, to change that, by requiring that they also pass the test suite
> feels like you're trying to overturn a previous TC decision. In
> addition, all previous votes on the test suite have been held with the
> expectation that the the test suite itself was *not* a conformance
> criteria, and thus for my votes at least, I've considered it a much
> lower threshold to clear.
>
> -Eric.
>