OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

sca-assembly message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [sca-assembly] Re: reviewing ASM test assertions and testcases


Inline <dab> like this </dab>

There are only a few points left that need further discussion....i.e. please see anything not marked DONE. I'm just trying to get the points in this email into actionable work items as necessary.

Dave Booz
STSM, BPM and SCA Architecture
Co-Chair OASIS SCA-Policy TC and SCA-J TC
"Distributed objects first, then world hunger"
Poughkeepsie, NY (845)-435-6093 or 8-295-6093
e-mail:booz@us.ibm.com

Inactive hide details for Bryan Aupperle---05/29/2009 09:17:17 AM---Comments below <bea> in tis pen/marking </bea> Bryan AupperBryan Aupperle---05/29/2009 09:17:17 AM---Comments below <bea> in tis pen/marking </bea> Bryan Aupperle, Ph.D.


From:

Bryan Aupperle/Raleigh/IBM@IBMUS

To:

"OASIS Assembly" <sca-assembly@lists.oasis-open.org>

Date:

05/29/2009 09:17 AM

Subject:

Re: [sca-assembly] Re: reviewing ASM test assertions and testcases






Comments below <bea> in tis pen/marking </bea>

Bryan Aupperle, Ph.D.
STSM, WebSphere Enterprise Platform Software Solution Architect

Research Triangle Park, NC
+1 919-254-7508 (T/L 444-7508)
Internet Address: aupperle@us.ibm.com

Mike Edwards <mike_edwards@uk.ibm.com>

05/29/2009 08:43 AM

To
"OASIS Assembly" <sca-assembly@lists.oasis-open.org>
cc
Subject
[sca-assembly] Re: reviewing ASM test assertions and testcases





Dave,

Many thanks for the review - keep it coming!!

Responses in my usual red

Yours, Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
Email: mike_edwards@uk.ibm.com

Subject
: Review of Test Assertions and TestCases

I've been through sections 4 and 5. I'm looking at SCA_Assembly_Test_Assertions_16.odt, SCA_Assembly_TestCases_22.odt and the CD03 assembly spec. Some of the following may have come up before, I don't know.

1) I think we really need to restructure or rename some/all of the artifacts because it's REALLY hard to connect all the pieces together when the names aren't meaningful...what does TestComposite4.composite do again? :-) For example; if I were King for a day, I'd rename ASM_0002_Client.java to something like Passthru.java.

I think I agree with you, but of course renaming everything now will be a lot of pain, so I'm reluctant, especially as Eclipse provides no

refactoring support for SCA artifacts (wow, there is an opportunity for some powerful tooling)


I am prepared to consider a one-time renaming as part of the Public Review process - but I don't want to make changes before Tuesday.


However, I'm not sure what kind of renaming you have in mind - at least a numeric labelling allows us to have a table with details such as

does exist in the appendix of the testcases document.


For the example you give here, ASM_ClientInterface_0002 might be more appropriate, but the number IS needed as there are 0003 and 0004

versions as well.

<dab> I have started a "deferred to PR" list and have put this one on it. DONE. </dab>


2) Shouldn't the composite files be in the oasis-open.org namespace? I don't think
http://oasis/tests is good enough. Same thought for the java packages of the java code.
A good point. We're blazing a trail here - no-one else has done a test suite at OASIS before so there are no guidelines.

So what names should we be using?


We need:


1) Composite artifacts namespace

2) WSDL interfaces namespace

3) Java artifacts package name(s)

4) ....when we get there, BPEL processes name space
...etc (for other impl language types


Here are some thoughts:


1)
http://docs.oasis-open.org/ns/opencsa/scatests/200903
2)
http://docs.oasis-open.org/ns/opencsa/scatests/wsdl/200903
3) org.oasisopen.sca.test

4)
http://docs.oasis-open.org/ns/opencsa/scatests/bpel/200903
<dab> I think there was agreement about this on the TC call. This update should be a coordinated one. I think you said you had some tools that will make this easy so I'll leave it to you. </dab>


3) No TA for ASM40001(ok?)

I state this explicitly in the listings at the back of the TA doc.

It is not testable by Assembly alone as we dont (yet) have a language which requires componentType files.

Perhaps Bryan can do something about this, as they are required by C, C++

<bea> It should be pretty easy to create a test case for this conformance point using C++ or C. Dave may be on the right path, there should be a TA with a prerequisite that includes a clause similar to " the implementation type of a component requires an explicit componentType file". Then in the same manner that test cases for bidirectional interfaces having mismatched remotable/local attributes are being pushed down to the Java TC, the test case for this could be pushed down to the C/C++ TC. </bea>
<dab> I agree. Mike, are you willing to make this update? <dab>


4) ASM-TA-4001 - The spec says that constrainingType files are in the Domain, but the testcase doesn't have them in the definitions file. The constraint file is just hanging around in the General contribution. The Def'ns file doesn't have an element to hold constrainingTypes. Seems like a hole in the spec.

There is a hole in the spec here. constrainingType is defined as a top level element in the SCA XSD.

There is then a statement relating to "Constraining type document" in Section 13.1, which implies that

they are defined as separate files in the Domain. However, there are no statements about location and

naming of these files anywhere, particularly in section 6 which covers constrainingType.


I dont think it was ever the intention to have constrainingType declarations within a Definitions file.

What needs doing is to raise an issue to define the document file for constrainingType. I used

"constraint" as the file extension, but others may have different views.

<dab> I raised the issue. DONE. </dab>


5) TA-4008 is not in the testcase document but it seems possible to write a test for it.

ASM_5002_Testcase covers this TA (as described in the table in section 3)

<dab> You're right. I needed to keep searching. DONE. </dab>


6) TestComposite43 and 44 could be more similar. One uses component1, the other uses a seemingly gratuitously different component name. I suspect this might be true of other composites in that these things evolved organically but some cleanup is needed so that the only differences between the composites is something that is central to it's purpose....and then as I said above, rename it to reflect it's purpose also.

Agreed on this point - again a result of changing design decisions over time.

I now know that it is best to try and uniquely label all components, using the composite name in the component name is

a good practice. Otherwise when you look at error traces you can get confused.


The problem here is the simple on of the sheer effort in redoing all the artifacts. If someone wants to help, then it might be made to happen.

<dab> I'll add it to the list of "deferred to PR". DONE </dab>

7) property/@file never appears in a TA or TC, same for property/@many=true


I suggest an issue against the TA document is appropriate for this.

<dab> I raised the issue. DONE. </dab>

8) I don't think @autowire is ever used on a component or composite, but I might have missed something in my search (searched both TA and TC documents)

I think this is because there is no normative statement relating to @autowire on a composite or component.
5.4.2 describes the inheritance but never makes a normative statement about it. It probably should.


This requires an issue against the Assembly spec.

<dab> I raised the issue. DONE. </dab>

9) ASM50007 - has a typo, it refers to service when it should say reference.


No. Time to raise an issue against the Assembly spec.

<dab> I raised the issue. DONE. </dab>

10) ASM_5010_TestCase claims to be related to ASM-TA-5005, but that's a typo, should be ASM-TA-5006


Thanks for spotting this. Fixed.

<dab> ok. DONE </dab>

11) ASM50005 has a 3rd case that I don't see covered by the TAs (I see only TA-5007 and TA-5008) - Component overrides bindings present in componentType


This is the result of a typo.

ASM-TA-5009
actually relates to ASM50005 not ASM50006 and is the 3rd case.

The real problem here is that there is no test assertion for ASM50006 at all. I fixed that by adding a new TA, ASM-TA-5043

<dab> ok. DONE </dab>

12) TA-5009 - claims to be for ASM50006 but ASM50006 is about callbacks. Must be a typo here?

See the answer for 11)

<dab> ok DONE </dab>

13) ASM50009 - since it's fairly complex, I wonder if it's worth having an additional negative test? perhaps a 1..n -> 0..1 case?

Sure - fancy writing it ? ;-)

<dab> I have added it to the "deferred to PR" list. DONE. </dab>

14) TA-5014, TA-5015, TA-5019, TA-5020 - These look like candidates for removal and placement into the language TCs.

A good point, but first we have to get one of the language TCs to 'fess up to having "wiredByImpl" implementations

<dab> Based on another discussion it sounds like we've decided to leave these TAs in place. I'm ok with that. DONE. </dab>

15) Test_ASM_5016 - I think you should use WSDL interfaces instead of Java for these ASM testcases. The testcases are good and should be moved to Java TC.

I do now (you need to see SCA_Assembly_TestCases_23 to see this)- I spent some time cleaning up testcases in response to some comments

from Bryan. Only ones that can't be done in WSDL have Java interfaces at the top level now (basically the callback cases for mixed local/remote

interfaces)

<dab> ok, DONE </dab>

16) TA-5017, Test_ASM-5017 and TestComposite10 highlight the fact that there are no conformance statements in section 8.5 of the spec.

Yes, something for the Assembly TC to noodle on. Surely <binding.sca/> has some requirements on it?

<dab> I raised an issue so the noodling can begin. DONE. <dab>

TestComposite10 uses @uri with binding.sca.

Yes it does - but this is allowed. See lines 2899 - 2901 of the Assembly spec (section 8.5)


17) I could not find where Section 4.3.1 points 3, 5 and 6 (CD03) are addressed in the TA document as part of ASM50014. TA-5023 could be clarified that it is covering point 2 as that's what the testcase does.

OK, regarding labelling, each of the bullet points in the final list in that section has a normative statement - these are dealt with one by one in the TA document.

The numbered list in 4.3.1 is NOT dealt with as it has no normative statements in it (it just deals with a list of possibilities).

<dab> I agree. However, my point is slightly different. ASM50014 implicitly brings in the numbered points because it says "...by any of the other ways listed above". I interpret those words to mean the preceding numbered list. I think we need additional TAs (and TCs) to properly cover ASM50014. </dab>

The one case that is NOT dealt with at all is [ASM50016] and that is for the simple reason that it makes demands on a binding type to

have a way of specifying an endpoint other than via a URI. Such testcases must be left to individual Bindings specs.

<dab> Yep. I was not concerned about that. </dab>


18) TA-5024 is covered by ASM_5018_TestCase, just wondering if we should have a negative test specific to TA-5024 instead of grouping it in the same testcase as TA-5018

Yes, we could do. Fancy writing it? ;-)

<dab> I have added it to the "deferred to PR" list. DONE. </dab>
19) ASM_5004_TestCase says reference with multiplicity 1..1, but the testcase uses
test.ASM_0002_Clientwhich has multiplicity 0..1 (@Reference(required=false))
Good catch. I fixed it in the simplest way by adding @multiplicity="1..1" on the reference in the top level composite.

<dab> ok DONE </dab>
20) TA-5031 the testcase ASM_5002_TestCase) does not use the @value attribute as required the associated ASM50027.

Thinking about this more, I think I should create a new dedicated testcase for this

<dab> Have you done this yet? I don't see it in the TC document. </dab>


21) TA-5035 is related to ASM50037, not ASM50031.

Yep, fixed.

<dab> ok DONE </dab>
22) ASM50030 and ASM50031 seem to be duplicates.

You're right - an issue for the Assembly spec.

<dab> I raised an issue. DONE </dab>
23) CD03 line 1030 contains an unmarked MUST. Need a label, TA, etc for this one.

Issue for the Assembly spec.

<dab> I raised an issue. DONE </dab>
24) TA-5039, 5040, 5041, 5042 - don't appear in the testcase document

Aha - I have a funny feeling that these assertions were added late for some reason that I dont remember

although it may be that the normative statements they relate to were stuffed into the spec late (the numbers

are suspiciously late in the sequence).

In any case - a few new testcases are required

<dab> You've added TCs to draft 23 of the TC document and TCs tin SVN. DONE</dab>


Dave Booz
STSM, BPM and SCA Architecture
Co-Chair OASIS SCA-Policy TC and SCA-J TC
"Distributed objects first, then world hunger"
Poughkeepsie, NY (845)-435-6093 or 8-295-6093
e-mail:booz@us.ibm.com





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU






[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]