OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

sca-bindings message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Comments on JMS test case document


I've done a side-by-side comparison of the JMS TA and TC documents, and have some comments. Unfortunately I have not had time to review the test case artefacts against the TC document.

General comments about testing approach:

1) BJM_30001_TestCase - says that the @uri must have valid syntax according to IETF spec....  how exhaustive to we expect that to be (asked before)

2) There are a few normative statements which include a prioritised list of cases.  Lets say the rule says A then B then C.  The test cases are written to verify that behaviour is as expected with A,  notA and B, notA and notB and C.  However I don't believe there are tests for A and B and notC, A and notB and C, A and B and C, notA and B and C.  As a specific example, we test that @uri is used, and we test that if @uri is absent then <destination> is used, but I'm not sure that we test that if both are present then @uri is used.   Should we have tests that cover all combinations?

Specific test case comments:

BJM_30002_TestCase - do we need separate cases for each of the resources that can appear in the @uri?
BJM_30010_TestCase - I believe the assertion is untestable, but there is a corresponding testcase; not sure that it really tests the statement
BJM_30019_TestCase - should there be separate cases for destination element and @uri? two different failures
BJM_30024_1/2/3/4_TestCase - do we need separate test cases for each header?
BJM_30024/5_TestCase - do we need separate test cases for service response and reference request?
BJM_30031_TestCase - would have expected to see a negative test here for a JCA 1.5 RA when the RA element is missing
BJM_40001_TestCase - what's it actually testing?  That the response is as expected for the default wire formant?  What about the different cases for the operation selector?
BJM_40002_TestCase - is this testing the cases for the operation selector?
BJM_40004_TestCase - why negative... shouldn't this test that the response is as expected?
BJM_40005T/B_TestCase - does this test receiving requests at a service and responses at a reference?
BJM_40006_TestCase -  does this test receiving requests at a service and responses at a reference?
BJM_40008_TestCase - I assume this is the same as BJM_40002_TestCase, same comment.
BJM_40009_TestCase - same as BJM_40004_TestCase.
BJM_50002_TestCase - is there a negative form of this when the assertion fails?
BJM_60003_TestCase - do we need separate tests for the different correlationScheme values?
BJM_60006_TestCase - not sure how we test each of the cases.
BJM_60007_TestCase - does this need to include cases where there is or isn't a response/destination?
BJM_60009_TestCase - untestable, not marked as such in the TA document
BJM_60010_TestCase - separate tests needed for different correlationSchemes?  This really seems to duplicate BJM30003/4/5/6.
BJM_60013_TestCase - separate tests needed for 60004/60005 cases?
BJM_60014/14A - renamed for consistency to BJM-TA-60014-1 and -2
BJM_60014_TestCase - separate tests needed for destination identified by @uri or <destination>?
BJM_60017_TestCase - separate test cases required for one-way and request/response operations? etc...
BJM-TA-60018 currently has no testcase but not stated as being untestable.

Some comments about test cases that I would want to verify against the test case artefacts.

BJM_30019_TestCase/BJM30022_TestCase - need to see how the activation spec is checked for referring to the same destination;
BJM_40011_TestCase - how is the invalid resolved operation name achieved?  Are there variants on this?

Regards, Simon

Simon Holdsworth
STSM, SCA Bindings Architect; Master Inventor; OASIS SCA Bindings TC Chair, AT&T and Boeing Lab Advocate
MP 211, IBM UK Labs, Hursley Park, Winchester SO21 2JN, UK
Tel +44-1962-815059 (Internal 245059) Fax +44-1962-816898
Internet - Simon_Holdsworth@uk.ibm.com

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]