Re: [sca-assembly-testing] Assembly TestCases Complete and ready for Review

[Bryan Aupperle <aupperle@us.ibm.com>]

More Comments (through Section 8)

Testcase 6004: Lists TestComposite16
in document but uses TestComposite18

Testcase 6016: Should be listed
as a negative test.

Testcase 6020: Should be listed
as a negative test.

Testcase 6033: Lists TestComposite44
in document but uses TestComposite41.

Testcase 7004: Duplicated

Testcase 8006: Lists TestComposite60
and TestComposite61 in document but uses TestComposite54 and TestComposite55.
Document indicates Service9 and Service9Callback are used, which do not
exist in SVN and are not referenced by other artifacts.  
Testcase 8015: Service1.wsdl
should be listed.

Testcases with statically detectable
errors that need separate contributions:
4002
4003
4004
4007
4008
5004
5005
5006
5007
5008
5009
5014
5023
5024
5027
5028
5029
5030
6001
6002
6003
6005
6006
6008
6012
6021
6022
6023
6025
6028
6029
6031
6032
6033
7001
7002
7003
7007
7008
7009

Testcases that need separate contributions
if interface compatibility can be checked statically:
5010
8006
8007
8008
8010
8011

Test cases that need separate contributions
if introspections can be done when a contribution is loaded:
8009
8012

Bryan Aupperle, Ph.D.
STSM, WebSphere Enterprise Platform Software Solution Architect

Research Triangle Park,  NC 
+1 919-254-7508 (T/L 444-7508)
Internet Address: aupperle@us.ibm.com

Mike Edwards <mike_edwards@uk.ibm.com>

05/12/2009 10:57 AM

To	sca-assembly-testing@lists.oasis-open.org
cc
Subject	Re: [sca-assembly-testing] Assembly TestCases Complete and ready for Review

Bryan, 

First, many thanks for reviewing the materials - the more eyes, the better.
 I know there are "bugs" of various kinds in there - I keep
discovering them myself as I work on eliminating the 
places where Tuscany does not yet conform to the OASIS specs.

1) Artifacts containing statically detectable errors

Yes, these will eventually need to be separated out into contributions
of their own. 
My reason for not doing this so far is that Tuscany does not stop the testcases
running - it just prints warning 
messages about the static errors that it finds.  I am changing the
code in this area and this will force me to 
hive off the "bad artifacts" into testcase-specific contributions.

2) Testcase 5031 

You just reported your first bug against the testcases document  ;-)

The document is wrong - it is a positive testcase with valid output.  I've
changed the testcases doc. 



Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

From:	Bryan Aupperle <aupperle@us.ibm.com>
To:	sca-assembly-testing@lists.oasis-open.org
Date:	11/05/2009 18:58
Subject:	Re: [sca-assembly-testing] Assembly TestCases Complete and ready for Review

---

I've started reviewing the test cases (I'm through Sections 4 & 5).
and I have a couple of questions. 

ASM14002 states that a runtime SHOULD prevent deployment of a contribution
that contains statically detected errors.  There are a number of test
cases that contain statically detectable errors, but are in the main contribution.
 Examples include Text cases 4002-4004, and 5004-5006 (I have a full
list of the ones in sections 4 & 5 I believe are included in this category
if desired).  Do these need to be restructured into separate contributions?

Why is test case 5031 a negative test?  The description suggests it
should execute successfully. 

I'm sure I will have more questions as I continue my review, but I wanted
to get these on the table. 

Bryan Aupperle, Ph.D.
STSM, WebSphere Enterprise Platform Software Solution Architect

Research Triangle Park,  NC 
+1 919-254-7508 (T/L 444-7508)
Internet Address: aupperle@us.ibm.com

Mike Edwards <mike_edwards@uk.ibm.com>

05/08/2009 07:17 AM

To	"OASIS Assembly" <sca-assembly@lists.oasis-open.org>, "OASIS Test " <sca-assembly-testing@lists.oasis-open.org>
cc
Subject	[sca-assembly-testing] Assembly TestCases Complete and ready for Review

Folks, 

The TestCases for the SCA Assembly spec are now complete and are ready
for review. 

You can find the TestCases document here: 

http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/download.php/32453/SCA_Assembly_TestCases_21.odt
http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/download.php/32455/SCA_Assembly_TestCases_21.pdf
http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/download.php/32454/SCA_Assembly_TestCases_21.doc

and the Test Case artifacts are held in the OASIS SVN repository here:

http://tools.oasis-open.org/version-control/browse/wsvn/sca-assembly/TestCases/#_TestCases_

You can find the TestAssertions document here: 

http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/download.php/31915/SCA_Assembly_Test_Assertions_16.odt
http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/download.php/31914/SCA_Assembly_Test_Assertions_16.doc


A reminder of the schedule for review and approval of the TestAssertions
and the TestCases for the Assembly spec: 

Schedule
TestAssertions document is already complete. 
TestCases document (and Test Cases) complete May 1 
Subcommittee Review May 1 - May 12 
Subcommittee Vote - May 13 
Assembly TC review May 1 - May 26 
Assembly TC vote - May 26 

We are a little late on the completion of the TestCases - if anyone thinks
that the overall schedule needs changing, please post to the list.

Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU