[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [sca-bindings] Comments on JMS test case document
I've uploaded a new revision of the testcases document,
sca-jmsbinding-1.1-testcases-wd03, that includes changes from those
comments. I've not yet been able to make updates for all the comments, the
status is as follows:
BJM_30002_TestCase - do we need separate cases for each of the resources
that can appear in the @uri?
-DONE. split into 3002_1, 3002_2
BJM_30010_TestCase - I believe the assertion is untestable, but there is a
corresponding testcase; not sure that it really tests the statement
- DONE. deleted testcase
BJM_30019_TestCase - should there be separate cases for destination element
and @uri? two different failures
- how to find the activation spec destination name? See also comments at
bottom
BJM_30024_1/2/3/4_TestCase - do we need separate test cases for each
header?
- each of the tests is already testing all headers. I suppose there could
be tests for all combinations of each header set in each place but to do
that conclusively would requre a lot of tests, is that really necessary?
BJM_30024/5_TestCase - do we need separate test cases for service response
and reference request?
- still to do
BJM_30031_TestCase - would have expected to see a negative test here for a
JCA 1.5 RA when the RA element is missing
- the test is testing the "...and is ignored otherwise" part. I'm not sure
how to test the "...MUST...for a JMS provider that implements the JCA 1.5
Spec" part
BJM_40001_TestCase - what's it actually testing? That the response is as
expected for the default wire formant? What about the different cases for
the operation selector?
- tested as per comment in WD06 TA doc - "Not sure how to test this other
than that the default wire format and operation selector elements present
on a binding.jms element are not rejected."
BJM_40002_TestCase - is this testing the cases for the operation selector?
- still to do
BJM_40004_TestCase - why negative... shouldn't this test that the response
is as expected?
DONE. Added an additional positive test that checks the response is as
expected
BJM_40005T/B_TestCase - does this test receiving requests at a service and
responses at a reference?
- still to do
BJM_40006_TestCase - does this test receiving requests at a service and
responses at a reference?
- still to do
BJM_40008_TestCase - I assume this is the same as BJM_40002_TestCase, same
comment.
- still to do
BJM_40009_TestCase - same as BJM_40004_TestCase.
- this is a positive test already so what is the change to make it similar
to 40004?
BJM_50002_TestCase - is there a negative form of this when the assertion
fails?
- no. Not sure i understand, the test is for "JMS intent is always be
included in the @alwaysProvides attribute of the JMS bindingType" which is
done be checking requires="JMS" works ok, how can it test the negative
form?
BJM_60003_TestCase - do we need separate tests for the different
correlationScheme values?
-DONE. Split 6003 into three _1, _2, _3
BJM_60006_TestCase - not sure how we test each of the cases.
- currently only tests a request with a messageID correlation set will work
ok, can't see a way to test using a unique destination
BJM_60007_TestCase - does this need to include cases where there is or
isn't a response/destination?
- DONE. Added a test with a response destination on the service binding
BJM_60009_TestCase - untestable, not marked as such in the TA document
- no test
BJM_60010_TestCase - separate tests needed for different
correlationSchemes? This really seems to duplicate BJM30003/4/5/6.
- as that would duplicate BJM30003/4/5/6 does it really need to test all
options again?
BJM_60013_TestCase - separate tests needed for 60004/60005 cases?
- still to do
BJM_60014/14A - renamed for consistency to BJM-TA-60014-1 and -2
- DONE now bjm_6014_1_1,bjm_6014_1_2, bjm_6014_2
BJM_60014_TestCase - separate tests needed for destination identified by
@uri or <destination>?
-DONE. added BJM_6014_2_TestCase
BJM_60017_TestCase - separate test cases required for one-way and
request/response operations? etc...
- DONE. added another test
BJM-TA-60018 currently has no testcase but not stated as being untestable.
- but does have TA comment: I’m not too happy with this statement at this
level; the normative statements in section
6.4.x don’t limit themselves to this case, perhaps this statement should be
moved to
section 8, Conformance as its actually defining section 6.4 as optional in
some way.
What should the test doc do?
BJM_30019_TestCase/BJM30022_TestCase - need to see how the activation spec
is checked for referring to the same destination;
-? how to do it, the activation spec interface defines nothing to get the
destination? see also 30019 comments above
BJM_40011_TestCase - how is the invalid resolved operation name achieved?
Are there variants on this?
- the client is just a vanilla JMS client sending a JMS message to the
service and the message contains a non-existent operation name
...ant
Simon
Holdsworth/UK/IBM
@IBMGB To
sca-bindings@lists.oasis-open.org
30/09/2010 14:33 cc
Subject
[sca-bindings] Comments on JMS test
case document
Folks,
I've done a side-by-side comparison of the JMS TA and TC documents, and
have some comments. Unfortunately I have not had time to review the test
case artefacts against the TC document.
General comments about testing approach:
1) BJM_30001_TestCase - says that the @uri must have valid syntax according
to IETF spec.... how exhaustive to we expect that to be (asked before)
2) There are a few normative statements which include a prioritised list of
cases. Lets say the rule says A then B then C. The test cases are written
to verify that behaviour is as expected with A, notA and B, notA and notB
and C. However I don't believe there are tests for A and B and notC, A and
notB and C, A and B and C, notA and B and C. As a specific example, we
test that @uri is used, and we test that if @uri is absent then
<destination> is used, but I'm not sure that we test that if both are
present then @uri is used. Should we have tests that cover all
combinations?
Specific test case comments:
BJM_30002_TestCase - do we need separate cases for each of the resources
that can appear in the @uri?
BJM_30010_TestCase - I believe the assertion is untestable, but there is a
corresponding testcase; not sure that it really tests the statement
BJM_30019_TestCase - should there be separate cases for destination element
and @uri? two different failures
BJM_30024_1/2/3/4_TestCase - do we need separate test cases for each
header?
BJM_30024/5_TestCase - do we need separate test cases for service response
and reference request?
BJM_30031_TestCase - would have expected to see a negative test here for a
JCA 1.5 RA when the RA element is missing
BJM_40001_TestCase - what's it actually testing? That the response is as
expected for the default wire formant? What about the different cases for
the operation selector?
BJM_40002_TestCase - is this testing the cases for the operation selector?
BJM_40004_TestCase - why negative... shouldn't this test that the response
is as expected?
BJM_40005T/B_TestCase - does this test receiving requests at a service and
responses at a reference?
BJM_40006_TestCase - does this test receiving requests at a service and
responses at a reference?
BJM_40008_TestCase - I assume this is the same as BJM_40002_TestCase, same
comment.
BJM_40009_TestCase - same as BJM_40004_TestCase.
BJM_50002_TestCase - is there a negative form of this when the assertion
fails?
BJM_60003_TestCase - do we need separate tests for the different
correlationScheme values?
BJM_60006_TestCase - not sure how we test each of the cases.
BJM_60007_TestCase - does this need to include cases where there is or
isn't a response/destination?
BJM_60009_TestCase - untestable, not marked as such in the TA document
BJM_60010_TestCase - separate tests needed for different
correlationSchemes? This really seems to duplicate BJM30003/4/5/6.
BJM_60013_TestCase - separate tests needed for 60004/60005 cases?
BJM_60014/14A - renamed for consistency to BJM-TA-60014-1 and -2
BJM_60014_TestCase - separate tests needed for destination identified by
@uri or <destination>?
BJM_60017_TestCase - separate test cases required for one-way and
request/response operations? etc...
BJM-TA-60018 currently has no testcase but not stated as being untestable.
Some comments about test cases that I would want to verify against the test
case artefacts.
BJM_30019_TestCase/BJM30022_TestCase - need to see how the activation spec
is checked for referring to the same destination;
BJM_40011_TestCase - how is the invalid resolved operation name achieved?
Are there variants on this?
Regards, Simon
Simon Holdsworth
STSM, SCA Bindings Architect; Master Inventor; OASIS SCA Bindings TC Chair,
AT&T and Boeing Lab Advocate
MP 211, IBM UK Labs, Hursley Park, Winchester SO21 2JN, UK
Tel +44-1962-815059 (Internal 245059) Fax +44-1962-816898
Internet - Simon_Holdsworth@uk.ibm.com
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]