Re: [sca-assembly] Exit Criteria Incomplete

[Jeff Mischkinsky <jeff.mischkinsky@oracle.com>]

On Jul 01, 2011, at 6:50 AM, Mike Edwards wrote:

>
> Jeff,
>
> I note that the Assembly TC had a considerable debate on the Exit  
> Criteria and did discuss the matters
> that you raise in your email before it took the vote to adopt the  
> Exit Criteria.  Words similar to the ones
> that you describe failed to win support in the TC.
>
> One point to make is that the SCA Assembly specification does not in  
> itself deal with interoperability -
> this must instead be the province of the Bindings TC.
>
> Regarding Portability, I believe that the requirements of  
> Portability for SCA artifacts are met if an SCA
> Runtime conforms to the SCA Assembly specification, since that is  
> essentially what the specification
> is about.

Yes, I would agree with that. Theoretically, IF the spec is written  
correctly, that should be true. The question is how is this  
demonstrated. One way to demonstrate portability would be to  
DEMONSTRATE that a set of artifacts can be consumed by (at least) 2  
different runtimes - the ones that are being used to demonstrate that  
the exit criteria is met. If the TC don't want to use the artifacts  
defined in the test suites I don't think there would be any objections  
-- though it seems like rather a lot of extra effort.  My point is  
simply that something like that NEEDS to be defined and agreed to by  
the various TCs, as appropriate for each TC.
>
> I agree that when considering whether the Exit Criteria have been  
> met by two or more SCA runtimes,
> members of the TC will want evidence and assurance that those SCA  
> runtimes do indeed conform to
> the specification.  However, I think that this is an obvious point  
> and it serves no purpose to add
> words to the Exit Criteria.
I'm not trying to add words to that part of the Exit Criteria. I am  
however trying to get us add the necessary words about HOW one goes  
about doing that demonstration.
>  I don't think that it is wise or necessary to be prescriptive about  
> the evidence
> that the TC will accept when considering the claims relating to a  
> given SCA runtime.

I really don't understand this statement. Most of the other TCs/WGs  
set up various requirements/tables/matrices that show the results that  
are used to validate that features are implemented properly -- the  
usual red/green/yellow thing :-). We did this for ws* specs in OASIS,  
and most w3c wg's do something like this.  E.G. ws-addressing said  
something like 4 impls to have greens for all the mandatory features,  
and 2 for each optional feature. I suppose by your definition this is   
"prescriptive", but it doesn't seem unwise and in fact it seems more  
like a best practice.


All I am however trying to us to complete our charter mandated job.

Given that we've set the number at 2. I just don't see what would be  
unwise to add something like:
     The 2 implementations must provide adequate evidence that the  
following set of artifacts are properly consumed and executed by the 2  
runtimes in question. <fill in the agreed upon set>

Note that it would be a judgement call on the part of the TC as to  
whether sufficient evidence has been presented. That's what the  
adoption vote would be about.

Other TCs need to adopt similar kinds of things, as appropriate.

cheers,
   jeff
>
>
> I agree that the Assembly Test Suite is a fine piece of work and  
> evidence of passing the tests will certainly
> convince me that a runtime is conforming.  However, I do not wish to  
> exclude other forms of convincing evidence
> that may be presented relating to a given SCA runtime.

>
>
> Yours, Mike
> Dr Mike Edwards
>  Mail Point 137, Hursley Park
> <mime-attachment.gif>
> STSM
>  Winchester, Hants SO21 2JN
> SCA & Services Standards
>  United Kingdom
> Co-Chair OASIS SCA Assembly TC
>
> IBM Software Group
>
> Phone:
> +44-1962 818014
>
> Mobile:
> +44-7802-467431 (274097)
>
> e-mail:
> mike_edwards@uk.ibm.com
>
>
>
>
>
>
>
> From:
> Jeff Mischkinsky <jeff.mischkinsky@oracle.com>
> To:
> OASIS Assembly <sca-assembly@lists.oasis-open.org>
> Date:
> 28/06/2011 15:53
> Subject:
> [sca-assembly] Exit Criteria Incomplete
>
>
>
>
> hi,
>
> I don't believe the TC has completed its job as required by the
> Charter wrt Exit Criteria - though we certainly have made concrete
> progress towards that goal.
>
> The SCA Assembly Charter requires:
> "The TC shall define concrete exit criteria that include at least two
> independent offerings that implement and are compliant with the all
> normative portions of specifications and demonstrate interoperability
> and portability as appropriate. Note that these are minimums and that
> the TC is free to set more stringent criteria."
>
> Last week the Assembly TC adopted exit criteria as follows:
> The Concrete Exit Criteria for the SCA Assembly V1.1 specification are
> that there shall be at least 2 independent SCA runtimes each of which
> are compliant with all normative portions of the specification as
> described in Section 12.2 of the SCA Assembly V1.1 specification
>
> We have completed the first part and met the minimums required by the
> Charter wrt number of offerings, but we have NOT YET set criteria for
> the second part - "demonstrating interoperability and portability as
> appropriate".
>
> I, and others, have struggled unsuccessfully so far, with words that
> would be satisfactory. Nevertheless, the TC still needs to do that
> before it can proceed to vote on whether those criteria have in fact
> been met.
>
> EXAMPLES might include such things as: The easiest thing in my mind is
> demonstrate that the 2 impls pass the test suite with 100% (98% or 95%
> or nnn%) success. Other examples might be provide output that shows
> that 98.34% of TA's have been successfully tested. I'm sure there are
> lots of other possibilities.
>
>
>  NOTE: Adopting those criteria is a different act from VOTING that
> the criteria have been met. As we have discussed many, many times -
> making that decision is a judgment that each voting member of the TC
> will have to make when the time comes.
>
>   Also, please note that there is nothing in the above suggestions as
> to who does the demonstrating. It could be a the provider of the
> offering. it could be another TC member. It could be an interested
> third party.
>
> cheers,
>   jeff
> --
> Jeff  
> Mischkinsky 
>                                                                                                jeff 
> .mischkinsky@oracle.com
> Sr. Director, Oracle Fusion  
> Middleware 
>                                                                       
> +1(650)506-1975
>                 and Web Services  
> Standards 
>                                                               500  
> Oracle Parkway, M/S 2OP9
> Oracle 
>                                                                                                                                         Redwood 
>  Shores, CA 94065
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>
>
>
>
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with  
> number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire  
> PO6 3AU
>
>
>
>
>
>
>

--
Jeff Mischkinsky			          		jeff.mischkinsky@oracle.com
Sr. Director, Oracle Fusion Middleware 				+1(650)506-1975
	and Web Services Standards           			500 Oracle Parkway, M/S 2OP9
Oracle								Redwood Shores, CA 94065