[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: reviewing ASM test assertions and testcases
I've been through sections 4 and
5. I'm looking at SCA_Assembly_Test_Assertions_16.odt, SCA_Assembly_TestCases_22.odt
and the CD03 assembly spec. Some of the following may have come up before,
I don't know.
1) I think we really need to restructure or rename some/all of the artifacts
because it's REALLY hard to connect all the pieces together when the names
aren't meaningful...what does TestComposite4.composite do again? :-) For
example; if I were King for a day, I'd rename ASM_0002_Client.java to something
like Passthru.java.
I think I agree with you, but of course
renaming everything now will be a lot of pain, so I'm reluctant, especially
as Eclipse provides no
refactoring support for SCA artifacts (wow,
there is an opportunity for some powerful tooling)
I am prepared to consider a one-time renaming
as part of the Public Review process - but I don't want to make changes
before Tuesday.
However, I'm not sure what kind of renaming
you have in mind - at least a numeric labelling allows us to have a table
with details such as
does exist in the appendix of the testcases
document.
For the example you give here, ASM_ClientInterface_0002
might be more appropriate, but the number IS needed as there are 0003 and
0004
versions as well.
2) Shouldn't the composite files be in the oasis-open.org namespace? I
don't think http://oasis/tests is
good enough. Same thought for the java packages of the java code.
A good point. We're blazing a trail
here - no-one else has done a test suite at OASIS before so there are no
guidelines.
So what names should we be using?
We need:
1) Composite artifacts namespace
2) WSDL interfaces namespace
3) Java artifacts package name(s)
4) ....when we get there, BPEL processes
name space
...etc (for other impl language types
Here are some thoughts:
1) http://docs.oasis-open.org/ns/opencsa/scatests/200903
2) http://docs.oasis-open.org/ns/opencsa/scatests/wsdl/200903
3) org.oasisopen.sca.test
4) http://docs.oasis-open.org/ns/opencsa/scatests/bpel/200903
3) No TA for ASM40001(ok?)
I state this explicitly in the listings
at the back of the TA doc.
It is not testable by Assembly alone as we
dont (yet) have a language which requires componentType files.
Perhaps Bryan can do something about this,
as they are required by C, C++
4) ASM-TA-4001 - The spec says that constrainingType files are in the Domain,
but the testcase doesn't have them in the definitions file. The constraint
file is just hanging around in the General contribution. The Def'ns file
doesn't have an element to hold constrainingTypes. Seems like a hole in
the spec.
There is a hole in the spec here. constrainingType
is defined as a top level element in the SCA XSD.
There is then a statement relating to "Constraining
type document" in Section 13.1, which implies that
they are defined as separate files in the
Domain. However, there are no statements about location and
naming of these files anywhere, particularly
in section 6 which covers constrainingType.
I dont think it was ever the intention to
have constrainingType declarations within a Definitions file.
What needs doing is to raise an issue to
define the document file for constrainingType. I used
"constraint" as the file extension,
but others may have different views.
5) TA-4008 is not in the testcase document but it seems possible to write
a test for it.
ASM_5002_Testcase covers this TA (as described
in the table in section 3)
6) TestComposite43 and 44 could be more similar. One uses component1, the
other uses a seemingly gratuitously different component name. I suspect
this might be true of other composites in that these things evolved organically
but some cleanup is needed so that the only differences between the composites
is something that is central to it's purpose....and then as I said above,
rename it to reflect it's purpose also.
Agreed on this point - again a result
of changing design decisions over time.
I now know that it is best to try and uniquely
label all components, using the composite name in the component name is
a good practice. Otherwise when you
look at error traces you can get confused.
The problem here is the simple on of the
sheer effort in redoing all the artifacts. If someone wants to help,
then it might be made to happen.
7) property/@file never appears in a TA or TC, same for property/@many=true
I suggest an issue against the TA document
is appropriate for this.
8) I don't think @autowire is ever used on a component or composite, but
I might have missed something in my search (searched both TA and TC documents)
I think this is because there is no normative
statement relating to @autowire on a composite or component.
5.4.2 describes the inheritance but never
makes a normative statement about it. It probably should.
This requires an issue against the Assembly
spec.
9) ASM50007 - has a typo, it refers to service when it should say reference.
No. Time to raise an issue against
the Assembly spec.
10) ASM_5010_TestCase claims to be related to ASM-TA-5005, but that's a
typo, should be ASM-TA-5006
Thanks for spotting this. Fixed.
11) ASM50005 has a 3rd case that I don't see covered by the TAs (I see
only TA-5007 and TA-5008) - Component overrides bindings present in componentType
This is the result of a typo.
ASM-TA-5009
actually relates to ASM50005 not ASM50006 and is the 3rd case.
The real problem here is that there is no
test assertion for ASM50006 at all. I fixed that by adding a new
TA, ASM-TA-5043
12) TA-5009 - claims to be for ASM50006 but ASM50006 is about callbacks.
Must be a typo here?
See the answer for 11)
13) ASM50009 - since it's fairly complex, I wonder if it's worth having
an additional negative test? perhaps a 1..n -> 0..1 case?
Sure - fancy writing it ? ;-)
14) TA-5014, TA-5015, TA-5019, TA-5020 - These look like candidates for
removal and placement into the language TCs.
A good point, but first we have to get
one of the language TCs to 'fess up to having "wiredByImpl" implementations
15) Test_ASM_5016 - I think you should use WSDL interfaces instead of Java
for these ASM testcases. The testcases are good and should be moved to
Java TC.
I do now (you need to see SCA_Assembly_TestCases_23
to see this)- I spent some time cleaning up testcases in response to some
comments
from Bryan. Only ones that can't be
done in WSDL have Java interfaces at the top level now (basically the callback
cases for mixed local/remote
interfaces)
16) TA-5017, Test_ASM-5017 and TestComposite10 highlight the fact that
there are no conformance statements in section 8.5 of the spec.
Yes, something for the Assembly TC to noodle
on. Surely <binding.sca/> has some requirements on it?
TestComposite10 uses @uri
with binding.sca.
Yes it does - but this is allowed. See
lines 2899 - 2901 of the Assembly spec (section 8.5)
17) I could not find where Section 4.3.1 points 3, 5 and 6 (CD03) are addressed
in the TA document as part of ASM50014. TA-5023 could be clarified that
it is covering point 2 as that's what the testcase does.
OK, regarding labelling, each of the bullet
points in the final list in that section has a normative statement - these
are dealt with one by one in the TA document.
The numbered list in 4.3.1 is NOT dealt with
as it has no normative statements in it (it just deals with a list of possibilities).
The one case that is NOT dealt with at all
is [ASM50016] and that is for the simple reason that it makes demands on
a binding type to
have a way of specifying an endpoint other
than via a URI. Such testcases must be left to individual Bindings
specs.
18) TA-5024 is covered by ASM_5018_TestCase, just wondering if we should
have a negative test specific to TA-5024 instead of grouping it in the
same testcase as TA-5018
Yes, we could do. Fancy writing it?
;-)
19) ASM_5004_TestCase says reference with multiplicity 1..1, but the testcase
uses test.ASM_0002_Clientwhich
has multiplicity 0..1 (@Reference(required=false))
Good catch. I fixed it in the simplest
way by adding @multiplicity="1..1" on the reference in the top
level composite.
20) TA-5031 the testcase ASM_5002_TestCase) does not use the @value attribute
as required the associated ASM50027.
Thinking about this more, I think I should
create a new dedicated testcase for this
21) TA-5035 is related to ASM50037, not ASM50031.
Yep, fixed.
22) ASM50030 and ASM50031 seem to be duplicates.
You're right - an issue for the Assembly
spec.
23) CD03 line 1030 contains an unmarked MUST. Need a label, TA, etc for
this one.
Issue for the Assembly spec.
24) TA-5039, 5040, 5041, 5042 - don't appear in the testcase document
Aha - I have a funny feeling that these assertions
were added late for some reason that I dont remember
although it may be that the normative statements
they relate to were stuffed into the spec late (the numbers
are suspiciously late in the sequence).
In any case - a few new testcases are required
Dave Booz
STSM, BPM and SCA Architecture
Co-Chair OASIS SCA-Policy TC and SCA-J TC
"Distributed objects first, then world hunger"
Poughkeepsie, NY (845)-435-6093 or 8-295-6093
e-mail:booz@us.ibm.com
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]