Comments on JMS test case document

[Simon Holdsworth <simon_holdsworth@uk.ibm.com>]

Folks,

I've done a side-by-side comparison
of the JMS TA and TC documents, and have some comments. Unfortunately I
have not had time to review the test case artefacts against the TC document.

General comments about testing approach:

1) BJM_30001_TestCase - says that the
@uri must have valid syntax according to IETF spec....  how exhaustive
to we expect that to be (asked before)

2) There are a few normative statements
which include a prioritised list of cases.  Lets say the rule says
A then B then C.  The test cases are written to verify that behaviour
is as expected with A,  notA and B, notA and notB and C.  However
I don't believe there are tests for A and B and notC, A and notB and C,
A and B and C, notA and B and C.  As a specific example, we test that
@uri is used, and we test that if @uri is absent then <destination>
is used, but I'm not sure that we test that if both are present then @uri
is used.   Should we have tests that cover all combinations?

Specific test case comments:

BJM_30002_TestCase - do we need separate
cases for each of the resources that can appear in the @uri?
BJM_30010_TestCase - I believe the assertion
is untestable, but there is a corresponding testcase; not sure that it
really tests the statement
BJM_30019_TestCase - should there be
separate cases for destination element and @uri? two different failures
BJM_30024_1/2/3/4_TestCase - do we need
separate test cases for each header?
BJM_30024/5_TestCase - do we need separate
test cases for service response and reference request?
BJM_30031_TestCase - would have expected
to see a negative test here for a JCA 1.5 RA when the RA element is missing
BJM_40001_TestCase - what's it actually
testing?  That the response is as expected for the default wire formant?
 What about the different cases for the operation selector?
BJM_40002_TestCase - is this testing
the cases for the operation selector?
BJM_40004_TestCase - why negative...
shouldn't this test that the response is as expected?
BJM_40005T/B_TestCase - does this test
receiving requests at a service and responses at a reference?
BJM_40006_TestCase -  does this
test receiving requests at a service and responses at a reference?
BJM_40008_TestCase - I assume this is
the same as BJM_40002_TestCase, same comment.
BJM_40009_TestCase - same as BJM_40004_TestCase.
BJM_50002_TestCase - is there a negative
form of this when the assertion fails?
BJM_60003_TestCase - do we need separate
tests for the different correlationScheme values?
BJM_60006_TestCase - not sure how we
test each of the cases.
BJM_60007_TestCase - does this need
to include cases where there is or isn't a response/destination?
BJM_60009_TestCase - untestable, not
marked as such in the TA document
BJM_60010_TestCase - separate tests
needed for different correlationSchemes?  This really seems to duplicate
BJM30003/4/5/6.
BJM_60013_TestCase - separate tests
needed for 60004/60005 cases?
BJM_60014/14A - renamed for consistency
to BJM-TA-60014-1 and -2
BJM_60014_TestCase - separate tests
needed for destination identified by @uri or <destination>?
BJM_60017_TestCase - separate test cases
required for one-way and request/response operations? etc...
BJM-TA-60018 currently has no testcase
but not stated as being untestable.

Some comments about test cases that
I would want to verify against the test case artefacts.

BJM_30019_TestCase/BJM30022_TestCase
- need to see how the activation spec is checked for referring to the same
destination;
BJM_40011_TestCase - how is the invalid
resolved operation name achieved?  Are there variants on this?

Regards, Simon

Simon Holdsworth
STSM, SCA Bindings Architect; Master Inventor; OASIS SCA Bindings TC Chair,
AT&T and Boeing Lab Advocate
MP 211, IBM UK Labs, Hursley Park, Winchester SO21 2JN, UK
Tel +44-1962-815059 (Internal 245059) Fax +44-1962-816898
Internet - Simon_Holdsworth@uk.ibm.com

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU