RE: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite name - Propose CNA

["Martin Chapman" <martin.chapman@oracle.com>]

I guess there's a subtlety between scoping work at a point in time, and binding a TC to some future non-actions. 

In any case, if someone wants to add tests for an optional assertion, they can raise an issue and it can be dealt with in the normal way.

Martin.

> -----Original Message-----
> From: David Booz [mailto:booz@us.ibm.com]
> Sent: 25 February 2010 13:45
> To: sca-assembly@lists.oasis-open.org
> Subject: RE: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite name - Propose CNA
> 
> Martin,
> 
> The test sub-committee made the decision not write tests for optional features during it's work last
> summer where the guts of the suite were vetted.  I have been working under the assumption that this
> decision was ratified by the TC itself (since no sub-committee decisions are binding).
> Perhaps this is a faulty assumption on my part and on the part of others as well.
> 
> Dave Booz
> STSM, BPM and SCA Architecture
> Co-Chair OASIS SCA-Policy TC and SCA-J TC "Distributed objects first, then world hunger"
> Poughkeepsie, NY (845)-435-6093  or  8-295-6093 e-mail:booz@us.ibm.com
> 
> 
> |------------>
> | From:      |
> |------------>
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
>   |"Martin Chapman" <martin.chapman@oracle.com>
> |
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
> |------------>
> | To:        |
> |------------>
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
>   |"'Mike Edwards'" <mike_edwards@uk.ibm.com>, "'OASIS Assembly'" <sca-assembly@lists.oasis-open.org>
> |
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
> |------------>
> | Date:      |
> |------------>
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
>   |02/25/2010 07:10 AM
> |
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
> |------------>
> | Subject:   |
> |------------>
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
>   |RE: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite name - Propose CNA
> |
>   >---------------------------------------------------------------------------------------------------
> -----------------------------------------------|
> 
> 
> 
> 
> 
> Mike,
> 
> I understand your point more clearly now, but you seem to be making a unilateral decision for the TC.
> I understand if you personally dont want to write tests for optional items, but I hope we wouldnt
> preclude others from doing so, if they wish. Im not suggesting anyone intends to, just making a
> general point.
> 
> Martin.
> 
> From: Mike Edwards [mailto:mike_edwards@uk.ibm.com]
> Sent: 24 February 2010 16:03
> To: 'OASIS Assembly'
> Subject: RE: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite name - Propose CNA
> 
> 
> Folks,
> 
> Let me make my position clear regarding the test suite and optional conformance items.
> 
> I do not claim that optional items should not be tested.
> 
> What I do say is that the current test suite - the one we're building for the 1.1 specification -
> should not test optional conformance items.
> 
> This is a pragmatic choice, to keep the work of building the test suite within bounds.
> 
> I believe that in the longer run, it would be a good idea to extend the test suite to test all
> optional conformance items.
> However, I think that in order to do this, a more sophisticated version of the test suite will be
> required that can relate specific testcases to specific conformance statements and which can then
> discount "testcase failures" to those optional conformance items.  At present, the test suite tests
> required conformance items - and an SCA runtime either passes all the tests or not - which is then a
> very simple thing to evaluate and report.
> 
> Let's remind ourselves of the goal that we set ourselves with the SCA test suite - to paraphrase Jeff
> Mischkinsky we're not building the ultimate test suite, we're creating a test suite that is serious
> enough to "pass the giggle test".
> 
> I believe that the current test suite more than passes the giggle test and indeed it is a serious test
> of the capabilities of an SCA runtime. I think that the Fabric3 developers and the Tuscany developers
> would more than agree with that statement!!
> 
> I do not think that it is desirable at this stage to extend the test suite to check the (relatively
> few) optional conformance items.
> 
> 
> Yours,  Mike.
> 
> Strategist - Emerging Technologies, SCA & SDO.
> Co Chair OASIS SCA Assembly TC.
> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
> Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431
> Email:  mike_edwards@uk.ibm.com
> 
> 
> 
> 
>  From: "Martin Chapman" <martin.chapman@oracle.com>
> 
> 
>  To:   Mike Edwards/UK/IBM@IBMGB, "'OASIS Assembly'"
>        <sca-assembly@lists.oasis-open.org>
> 
>  Date: 24/02/2010 15:00
> 
> 
>  Subje RE: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty
>  ct:   composite name - Propose CNA
> 
> 
> 
> 
> 
> 
> 
> 
> Generic comment on Mikes last line.
> 
> Its a fallacy to suggest optional items should not be tested. I think there is some confusion about
> testing code against expected outcomes, and what tests need to be passed in order to claim
> conformance. A test for an optional assertion must be written with the mindset of if you implement
> this assertion then you must pass these tests , this is no different from any tests of a mandatory
> assertion. How we structure the test suite to trigger which optional assertions are being tested to
> match those that a vendor is claiming to support  is a different discussion IMHO.
> 
> Martin.
> 
> From: Mike Edwards [mailto:mike_edwards@uk.ibm.com]
> Sent: 23 February 2010 15:09
> To: OASIS Assembly
> Subject: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite name - Propose CNA
> 
> 
> Folks,
> 
> Assembly issue 205 (http://www.osoa.org/jira/browse/ASSEMBLY-205 ) argues that testcase ASM_12001 is
> invalid since the TestConfiguration contains a set of Contributions to use, but does not supply the
> name of any Composite to run.
> 
> My view is that the testcase is valid and does not need to be changed.
> Hence we should close this issue with no action.
> 
> Here is the configuration in the test client:
> 
>            config.testName                 = this.getClass().getSimpleName
> ().substring(0, 9);
>            config.input                         = "request";
>            config.output[0]                         = config.testName + " "
> + config.input + " service1 operation1 invoked" ;
>            // null composite supplied - the name of the composite to run comes from the sca-
> contribution.xml
>            config.composite                 = null;
>            config.testServiceName         = "TestClient";
> 
>            config.contributionNames        = new String[] { "ASM_12001",
> "General", "General" + _Lang };
> 
> The relevant things are:
> 
> a) the list of contribution names assigned to config.contributionNames
> b) the value of config.composite, which is null
> 
> In the design of all the testcases, the configuration has one or more Contributions, which are
> intended to contain the artifacts to use, plus the name of a Composite to run.  It is expected that
> the Composite exists somewhere in the supplied Contributions.
> 
> In this testcase, one of the contributions - ASM_12001 - has an SCA contribution which has a
> contribution.xml file as follows:
> 
> <contribution
>        xmlns="http://docs.oasis-open.org/ns/opencsa/sca/200912";
>        xmlns:test="http://docs.oasis-open.org/ns/opencsa/scatests/200903";>
> 
> 
>        <deployable composite="test:TEST_ASM_12001"/>
>        <import namespace="
> http://docs.oasis-open.org/ns/opencsa/scatests/200903"/>  <!-- Contributions namespace -->
>        <import namespace="http://test.sca.oasisopen.org/"/>
> <!-- WSDL namespace -->
> 
> </contribution>
> 
> Note the presence of the <deployable/> element, referencing a specific Composite.
> 
> The design of this test is intended to check that the SCA runtime will deploy the composite referenced
> by the <deployable/> element.
> Either this test is correct as written, or the argument must be made that an SCA runtime does not have
> to do anything with the <deployable/> element.
> 
> If that is the case, then the whole testcase should be removed as the function is optional.
> 
> 
> 
> Yours,  Mike.
> 
> Strategist - Emerging Technologies, SCA & SDO.
> Co Chair OASIS SCA Assembly TC.
> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
> Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431
> Email:  mike_edwards@uk.ibm.com
> 
> 
> 
> 
> 
> 
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> 
> 
> 
> 
> 
> 
> 
> 
>