From: Mike Edwards
[mailto:mike_edwards@uk.ibm.com]
Sent: 24 February 2010 16:03
To: 'OASIS Assembly'
Subject: RE: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty
composite name - Propose CNA
Folks,
Let me make my
position clear regarding the test suite and optional conformance items.
I do not claim
that optional items should not be tested.
What I do say
is that the current test suite - the one we're building for the 1.1
specification - should not
test optional
conformance items.
This is a pragmatic
choice, to keep the work of building the test suite within bounds.
I believe that
in the longer run, it would be a good idea to extend the test suite to test all
optional conformance items.
However, I
think that in order to do this, a more sophisticated version of the test suite
will be required that can relate
specific
testcases to specific conformance statements and which can then discount
"testcase failures" to those
optional
conformance items. At present, the test suite tests required conformance
items - and an SCA runtime
either passes
all the tests or not - which is then a very simple thing to evaluate and
report.
Let's remind
ourselves of the goal that we set ourselves with the SCA test suite - to
paraphrase Jeff Mischkinsky we're not
building the
ultimate test suite, we're creating a test suite that is serious enough to
"pass the giggle test".
I believe that
the current test suite more than passes the giggle test and indeed it is a
serious test of the capabilities of
an SCA runtime.
I think that the Fabric3 developers and the Tuscany developers would more than
agree with that statement!!
I do not think
that it is desirable at this stage to extend the test suite to check the
(relatively few) optional conformance items.
Yours, Mike.
Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
Email: mike_edwards@uk.ibm.com
|
From:
|
"Martin
Chapman" <martin.chapman@oracle.com>
|
|
To:
|
Mike
Edwards/UK/IBM@IBMGB, "'OASIS Assembly'"
<sca-assembly@lists.oasis-open.org>
|
|
Date:
|
24/02/2010
15:00
|
|
Subject:
|
RE:
[sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite name -
Propose CNA
|
Generic
comment on Mike’s last line.
It’s
a fallacy to suggest optional items should not be tested. I think there is some
confusion about testing code against expected outcomes, and what tests need to
be passed in order to claim conformance. A test for an optional assertion must
be written with the mindset of “if you implement this assertion then you
must pass these tests ,” this is no different from any tests of a
mandatory assertion. How we structure the test suite to trigger which optional
assertions are being tested to match those that a vendor is claiming to support
is a different discussion IMHO.
Martin.
From: Mike Edwards [mailto:mike_edwards@uk.ibm.com]
Sent: 23 February 2010 15:09
To: OASIS Assembly
Subject: [sca-assembly] [ASSEMBLY 205] TEST_ASM_12001 has empty composite
name - Propose CNA
Folks,
Assembly issue 205 (http://www.osoa.org/jira/browse/ASSEMBLY-205
) argues that
testcase ASM_12001 is invalid since the TestConfiguration contains a set of
Contributions to use, but
does not supply the name of any Composite to run.
My view is that the testcase is valid and does not need to be changed.
Hence we should close this issue with no action.
Here is the configuration in the test client:
config.testName
= this.getClass().getSimpleName().substring(0,
9);
config.input
= "request";
config.output[0]
= config.testName + " " + config.input
+ " service1 operation1 invoked" ;
// null
composite supplied - the name of the composite to run comes from the sca-contribution.xml
config.composite
= null;
config.testServiceName
= "TestClient";
config.contributionNames
= new
String[] { "ASM_12001", "General", "General"
+ _Lang };
The relevant things are:
a) the list of contribution names assigned to config.contributionNames
b) the value of config.composite, which is null
In the design of all the testcases, the configuration has one or more
Contributions, which are intended to contain the artifacts to use, plus
the name of a Composite to run. It is expected that the Composite exists
somewhere in the supplied Contributions.
In this testcase, one of the contributions - ASM_12001 - has an SCA
contribution which has a contribution.xml file as follows:
<contribution
xmlns="http://docs.oasis-open.org/ns/opencsa/sca/200912"
xmlns:test="http://docs.oasis-open.org/ns/opencsa/scatests/200903">
<deployable composite="test:TEST_ASM_12001"/>
<import namespace="http://docs.oasis-open.org/ns/opencsa/scatests/200903"/> <!-- Contributions namespace -->
<import namespace="http://test.sca.oasisopen.org/"/>
<!-- WSDL namespace -->
</contribution>
Note the presence of the <deployable/> element, referencing a specific
Composite.
The design of this test is intended to check that the SCA runtime will deploy
the composite referenced by the <deployable/> element.
Either this test is correct as written, or the argument must be made that an
SCA runtime does not have to do anything with the <deployable/> element.
If that is the case, then the whole testcase should be removed as the function
is optional.
Yours, Mike.
Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
Email: mike_edwards@uk.ibm.com
Unless
stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless
stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU