OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

sca-assembly message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [sca-assembly] Re: reviewing ASM test assertions and testcases

Comments below <bea> in tis pen/marking </bea>

Bryan Aupperle, Ph.D.
STSM, WebSphere Enterprise Platform Software Solution Architect

Research Triangle Park,  NC
+1 919-254-7508 (T/L 444-7508)
Internet Address: aupperle@us.ibm.com

Mike Edwards <mike_edwards@uk.ibm.com>

05/29/2009 08:43 AM

"OASIS Assembly" <sca-assembly@lists.oasis-open.org>
[sca-assembly] Re: reviewing ASM test assertions and testcases


Many thanks for the review - keep it coming!!

Responses in my usual

Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

: Review of Test Assertions and TestCases

I've been through sections 4 and 5. I'm looking at SCA_Assembly_Test_Assertions_16.odt, SCA_Assembly_TestCases_22.odt and the CD03 assembly spec. Some of the following may have come up before, I don't know.

1) I think we really need to restructure or rename some/all of the artifacts because it's REALLY hard to connect all the pieces together when the names aren't meaningful...what does TestComposite4.composite do again? :-) For example; if I were King for a day, I'd rename ASM_0002_Client.java to something like Passthru.java.

I think I agree with you, but of course renaming everything now will be a lot of pain, so I'm reluctant, especially as Eclipse provides no

refactoring support for SCA artifacts (wow, there is an opportunity for some powerful tooling)

I am prepared to consider a one-time renaming as part of the Public Review process - but I don't want to make changes before Tuesday.

However, I'm not sure what kind of renaming you have in mind - at least a numeric labelling allows us to have a table with details such as

does exist in the appendix of the testcases document.

For the example you give here, ASM_ClientInterface_0002 might be more appropriate, but the number IS needed as there are 0003 and 0004

versions as well.

2) Shouldn't the composite files be in the oasis-open.org namespace? I don't think
http://oasis/tests is good enough. Same thought for the java packages of the java code.
A good point.  We're blazing a trail here - no-one else has done a test suite at OASIS before so there are no guidelines.

So what names should we be using?

We need:

1) Composite artifacts namespace

2) WSDL interfaces namespace

3) Java artifacts package name(s)

4) ....when we get there, BPEL processes name space
...etc (for other impl language types

Here are some thoughts:

3) org.oasisopen.sca.test


3) No TA for ASM40001(ok?)

I state this explicitly in the listings at the back of the TA doc.

It is not testable by Assembly alone as we dont (yet) have a language which requires componentType files.

Perhaps Bryan can do something about this, as they are required by C, C++

<bea> It should be pretty easy to create a test case for this conformance point using C++ or C.  Dave may be on the right path, there should be a TA with a prerequisite that includes a clause similar to " the implementation type of a component  requires an explicit componentType file".  Then in the same manner that test cases for bidirectional interfaces having mismatched  remotable/local attributes are being pushed down to the Java TC, the test case for this could be pushed down to the C/C++ TC. </bea>

4) ASM-TA-4001 - The spec says that constrainingType files are in the Domain, but the testcase doesn't have them in the definitions file. The constraint file is just hanging around in the General contribution. The Def'ns file doesn't have an element to hold constrainingTypes. Seems like a hole in the spec.
There is a hole in the spec here.  constrainingType is defined as a top level element in the SCA XSD.

There is then a statement relating to "Constraining type document" in Section 13.1, which implies that

they are defined as separate files in the Domain.  However, there are no statements about location and

naming of these files anywhere, particularly in section 6 which covers constrainingType.

I dont think it was ever the intention to have constrainingType declarations within a Definitions file.

What needs doing is to raise an issue to define the document file for constrainingType.  I used

"constraint" as the file extension, but others may have different views.

5) TA-4008 is not in the testcase document but it seems possible to write a test for it.

ASM_5002_Testcase covers this TA (as described in the table in section 3)

6) TestComposite43 and 44 could be more similar. One uses component1, the other uses a seemingly gratuitously different component name. I suspect this might be true of other composites in that these things evolved organically but some cleanup is needed so that the only differences between the composites is something that is central to it's purpose....and then as I said above, rename it to reflect it's purpose also.

Agreed on this point - again a result of changing design decisions over time.

I now know that it is best to try and uniquely label all components, using the composite name in the component name is

a good practice.  Otherwise when you look at error traces you can get confused.

The problem here is the simple on of the sheer effort in redoing all the artifacts.  If someone wants to help, then it might be made to happen.

7) property/@file never appears in a TA or TC, same for property/@many=true

I suggest an issue against the TA document is appropriate for this.

8) I don't think @autowire is ever used on a component or composite, but I might have missed something in my search (searched both TA and TC documents)

I think this is because there is no normative statement relating to @autowire on a composite or component.

5.4.2 describes the inheritance but never makes a normative statement about it.  It probably should.

This requires an issue against the Assembly spec.

9) ASM50007 - has a typo, it refers to service when it should say reference.

No.  Time to raise an issue against the Assembly spec.

10) ASM_5010_TestCase claims to be related to ASM-TA-5005, but that's a typo, should be ASM-TA-5006

Thanks for spotting this.  Fixed.

11) ASM50005 has a 3rd case that I don't see covered by the TAs (I see only TA-5007 and TA-5008) - Component overrides bindings present in componentType

This is the result of a typo.

actually relates to ASM50005 not ASM50006 and is the 3rd case.

The real problem here is that there is no test assertion for ASM50006 at all.  I fixed that by adding a new TA, ASM-TA-5043

12) TA-5009 - claims to be for ASM50006 but ASM50006 is about callbacks. Must be a typo here?

See the answer for 11)

13) ASM50009 - since it's fairly complex, I wonder if it's worth having an additional negative test? perhaps a 1..n -> 0..1 case?

Sure - fancy writing it ?   ;-)

14) TA-5014, TA-5015, TA-5019, TA-5020 - These look like candidates for removal and placement into the language TCs.

A good point, but first we have to get one of the language TCs to 'fess up to having "wiredByImpl" implementations

15) Test_ASM_5016 - I think you should use WSDL interfaces instead of Java for these ASM testcases. The testcases are good and should be moved to Java TC.

I do now (you need to see SCA_Assembly_TestCases_23 to see this)- I spent some time cleaning up testcases in response to some comments

from Bryan.  Only ones that can't be done in WSDL have Java interfaces at the top level now (basically the callback cases for mixed local/remote


16) TA-5017, Test_ASM-5017 and TestComposite10 highlight the fact that there are no conformance statements in section 8.5 of the spec.

Yes, something for the Assembly TC to noodle on.  Surely <binding.sca/> has some requirements on it?

TestComposite10 uses @uri with binding.sca.

Yes it does - but this is allowed.  See lines 2899 - 2901 of the Assembly spec (section 8.5)

17) I could not find where Section 4.3.1 points 3, 5 and 6 (CD03) are addressed in the TA document as part of ASM50014. TA-5023 could be clarified that it is covering point 2 as that's what the testcase does.

OK, regarding labelling, each of the bullet points in the final list in that section has a normative statement - these are dealt with one by one in the TA document.

The numbered list in 4.3.1 is NOT dealt with as it has no normative statements in it (it just deals with a list of possibilities).

The one case that is NOT dealt with at all is [ASM50016] and that is for the simple reason that it makes demands on a binding type to

have a way of specifying an endpoint other than via a URI.  Such testcases must be left to individual Bindings specs.

18) TA-5024 is covered by ASM_5018_TestCase, just wondering if we should have a negative test specific to TA-5024 instead of grouping it in the same testcase as TA-5018

Yes, we could do.  Fancy writing it?  ;-)

19) ASM_5004_TestCase says reference with multiplicity 1..1, but the testcase uses
test.ASM_0002_Clientwhich has multiplicity 0..1 (@Reference(required=false))
Good catch.  I fixed it in the simplest way by adding @multiplicity="1..1" on the reference in the top level composite.

20) TA-5031 the testcase ASM_5002_TestCase) does not use the @value attribute as required the associated ASM50027.

Thinking about this more, I think I should create a new dedicated testcase for this

21) TA-5035 is related to ASM50037, not ASM50031.

Yep, fixed.

22) ASM50030 and ASM50031 seem to be duplicates.

You're right - an issue for the Assembly spec.

23) CD03 line 1030 contains an unmarked MUST. Need a label, TA, etc for this one.

Issue for the Assembly spec.

24) TA-5039, 5040, 5041, 5042 - don't appear in the testcase document

Aha - I have a funny feeling that these assertions were added late for some reason that I dont remember

although it may be that the normative statements they relate to were stuffed into the spec late (the numbers

are suspiciously late in the sequence).

In any case - a few new testcases are required

Dave Booz
STSM, BPM and SCA Architecture
Co-Chair OASIS SCA-Policy TC and SCA-J TC
"Distributed objects first, then world hunger"
Poughkeepsie, NY (845)-435-6093 or 8-295-6093

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]