[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [ebxml-iic] minutes and more - response and use case examples
|
Mike:
that's definitely better, but there are some
places
where the test scripts don't seem to work (or do what the use case
expects)
So I
point to these problems and outline a solution.
Also
we still do not handle the "test case outcome" semantics
in a
satisfying way , I think, and that becomes obvious on one of the use
cases.
See my proposal (I try to dissociate explicitly the boolean
value
of an
Assertion, and the thread operational outcome for the test case:
fail/pass/undetermined/continue)
See
attached the original Use Case defs, and for each, my comments
labelled:
"...comments from jacques on latest solution from Mike (May 20):" Cheers,
Jacques
|
Test use cases, representative of test case control patterns. They should help focus on the minimal set of operators for control flow. -------------------------------------- Use Case #1: Basic Business Transaction, (e.g. PIP 3A4) with TimeToPerform, and TimeToAcknowledge -------------------------------------- Test Case: (test driver plays "buyer") Step 1: Buyer Send a P.O. to Supplier. Step 2: Receive a signal ack, correlated with the P.O. based on Conversation ID Step 3: Receive a P.O. Confirmation or Rejection. Step 4: Send a signal Ack. Constraints: - total time for 1-2-3 is bounded by TimeToPerform. - total time for 1-2 is bounded by TimeToAcknowledge. Success: - within TimeToPerform, receive either a P.O. Confirmation or Rejection, but not both. - TimeToAcknowledge is satisfied. Variant: The correlation is based on a PO reference # that is in the payload. --------- Use Case #1: comments from jacques on latest solution from Mike (May 20): - I am not convinced that this new "Transaction" element is necessary (one more construct, also), especially for a general purpose test framework (seems too focused on BPSS bus transactions)... As we discussed, we may want to measure time for any subset of test step sequence, and possibly overlapping ones (which prohibits a "bloc" approach) I thought we opted for some timestamping mechanism. For example: we could split a thread that starts a counter (e.g. just use "sleep" op) and at the end of the sleep, checks (Assertion) if some global var (e.g. "Transaction1Done") is true. And the thread that does the time-bound transaction (e.g. BPSS bus transaction) would at the end set Transaction1Done to "true". (so we can use SetXPathParameter) - About 'TimeToAcknowledgeReceipt': not sure how that works?(where is used/set the TimeToAcknowledge value for the test case). That also could be done using a global signal var and a "sleeper thread". - Second GetMessage op: <ebTest:VerifyContent>//*[Confirmation and Rejection] Shounldn't we use : <ebTest:VerifyContent>/FilterResult//*[Confirmation and Rejection]? Also, we want to fail teh case if this condition is satisfied ! So we need to actually wait for TimeToPerform to make sure we have not gotten "too many" messages. I suggest using the "sleeper" thread suggested above, and add one more test of failure: if count(/FilterResult//*[Confirmation or Rejection]) > 1. -------------------------------------- Use Case #2: Catching unexpected ebXML Error messages (a test case "design pattern") -------------------------------------- Test Case: (test driver plays "buyer") Step 1: buyer sends a message M1 Step 2: receive and verify message M2 Correlation M1-M2 based on COnversation ID. But in case an error is received that correlates with M1 (in addition or instead of M2), at any point in time within 300sec after sending M1, before or after receiving M2, and regardless of the outcome of verifying M2, we want the test case to fail. --------- Use Case #2: comments from jacques on latest solution from Mike (May 20): - Do we really need the "wrapper" element ThreadRef nameRef='thread_01' ? that seems redundant. - I believe the Assertion test for the error-catching thread: /FilterResult//*[not(eb:ErrorList)] will not work !! because this thread will exit on success if we catch the response message M2 BEFORE an error message arrives! So we need to actually explicitly test for catching the error message (/FilterResult//eb:ErrorList ), and explicitly fail the test case if we catch it. We discussed this kind of use case before, and frankly I don't see how we can do without a more explicit statement to tell the semantics of the outcome of teh Assertion check, with regard to the overall test case, something of the "exitCondition" type we already have in Appendix E (but don't seem to use anywhere) (fail / pass / undetermined / continue). Any "implicit" semantics, like (Assertion fails-->test Case fails), (Asssertion pass --> test case pass "except if there is something after"...) will be confusing, thread-context-dependent, and unclear to users. One way to do this is to add attributes to VerifyContent op (like we did for stepDuration for GetMessage): VerifyContent when_true="fail" when_false="continue" - A minor point: in step #2 the test on "action=mute" would rather be done in the VerifyContent part, as we want to test all messages that correlate, including those with bad Action field (for errors) (so we don't want to filter these out). -------------------------------------- Use Case #3: conditional branching -------------------------------------- Test Case: (test driver plays "buyer") - buyer sends M1 - received M2 (e.g. an approval, or rejection) ---> if M2 is "approval", we will expect a sequence of: - receive M3 (e.g. a quotation) - send M4 (e.g. approval of quote and payment info) - receive M5 (final delivery details). ---> if M2 is "rejection", we will expect a sequence of: - receive M6 (proposed alternative) We need to verify all received messages, as the test case would fail if they do not comply. Any error message received, would also fail the test case (see use case #2) --------- Use Case #3: comments from jacques on latest solution from Mike (May 20): - the error cases could be handled all separately from main thread, like suggested in use case #1, so we don't need to complicate the main thread with them (step #2 would not have to worry about receiving both 1 acceptance and a rejection, which anyway needs to be tested over a longer period of time, e.g. till TimeToPerform expires.) - another unclear pattern: we have these two sections: if (GetMessage = approval) then... endif; if (GetMessage = rejection) then... endif; The risk here is taht both conditions may succeed in case we get an Approval (first) then a Rejection. It would be much safer if we GetMessage just once, and whatever this response is (first message we catch), we then branch to the right thread. But if we do it that way, it seems that we may have to put the "if...then" inside the step, e.g. wrapping the Assertion check. One possible solution: If we use the proposed "when_true" and "when_false" attributes proposed in use case #2, then we could do without an "if" ! e.g. VerifyContent when_true="threadABC" when_false="threadXYZ" Or, if we want to still have two tests for "Approval" and "Rejection": VerifyContent when_true="threadApprove" when_false="continue" ... schemaLocation=http://http://www.eBusines.org/approval.xsd VerifyContent when_true="threadReject" when_false="fail" ... schemaLocation=http://http://www.eBusines.org/rejection.xsd - other than that, that seems to work (though steps don't match exactly those in use case)
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]