RE: [ebBP] [ebxml-iic] Scripting of Test Cases (Process Concepts)

[Jacques Durand <JDurand@us.fujitsu.com>]

Title: RE: [ebBP] [ebxml-iic] Scripting of Test Cases (Process Concepts)

To make it easier to follow-up on Monica request (my blurb she quoted is a bit
out of context), what we are looking for is:

- what are the possible failure cases for a BP engine supposed to run
a business transaction, that can be observed by the other party,
(we support so far: timeouts, not getting the right message in time, getting a message of the wrong kind, getting too many messages. Any specific instance

of these, or specific combination that we should care for?)

- a review of, or additions to the 3 use cases attached (keep in mind these cases
are intended to exercise various features of teh test framework, and do not necessarily represent typical BP transactions / collaborations.)

In these use cases, a test driver would simulate one party.

- any realistic case where there is looping involved from the testing party?
(e.g. iterating on: sending a message + receiving a message, until
the received message satisfies a condition.)

Thx,

Jacques

-----Original Message-----
From: Monica J. Martin [mailto:Monica.Martin@sun.com]
Sent: Tuesday, May 25, 2004 5:21 PM
To: ebXML BP
Cc: Jacques Durand; Michael Kass
Subject: [ebBP] [ebxml-iic] Scripting of Test Cases (Process Concepts)

ebBP team,
If anyone has any comments on the IIC work (applying relevant process
concepts to test framework), please let me know. Further details from
that team today.

Good progress. Thanks.

> Durand: We had a productive call yesterday, and also discussion after
> the call.
> I think we outlined a scripting solution that can handle every use
> cases properly, BPSS reqs,
> while keeping the scripting simple.
> Here is my summary of the consensus we reached, of course this needs
> to be validated
> after a detailed review and use case re-coding. Mike please correct if
> needed:
>
> Based on review of the three use cases and previous scripting:
>
> - When branching is required based on received message: either header
> data or payload data can be used. But only one GetMessage op execution
> should be needed (not one per condition outcome).
>
> - Branching (like needed in Use Case #3) will be done by invoking
> threads by name, within a split() op.
> - A thread, when invoked (whether within or outside a conditional
> statement), will inherit visibility of parameters from the invoking
> thread.
>
> - Because we did not want to spawn a thread from within a test step,
> and for other reasons, we came to the conclusion that it is better to
> extract the "Assertion" test outside of the GetMessage step.
>
> - The Assertion becomes a control flow operator: it may test
> FilterOutput material from several GetMessage inputs (these need be
> named), or may test value of variables (no message data).
>
> - The outcome of the assertion test will be stated explicitly (e.g.
> might be indicated via two attributes: "when_true", "when_false" that
> are just a representation for if..then...else, without the
> composability.) These statement effectively decouple the boolean
> result of an assertion, with the test case outcome.
>
> This was needed to handle "error-catching threads" that must fail the
> case when assertion is true.
> - The Assertion outcome (e.g. specified as value of above attributes)
> may be:
> . exit(...) with arg = fail/pass/undetermined. This terminates the
> test case.
> . continue (default). This let the test case continue its flow (next
> test step, thread join, etc.)
> . split(threadA [,...]). This spawn a thread concurrently to current
> thread.
> - Timeout may be simply handled by: (1) spawning a sleeping thread
> (sleep maxtime), (2) setting a flag variable when the last step is
> complete, (3) sleeping thread checks the status of the flag after
> sleeping, and makes decision (e.g. exit) if failure. This allows for
> any pattern of time checks (intertwined, etc.)
>
> - Looping ("while...do") can be handled by a thread invoking itself
> recursively, from an Assertion op.
>
>
> Jacques
>
>

 

 Test use cases, representative of test case control patterns.
They should help focus on the minimal set of operators for control flow.

-------------------------------------- 
  Use Case #1: Basic Business Transaction, 
(e.g. PIP 3A4) with TimeToPerform, and TimeToAcknowledge 
  -------------------------------------- 

Test Case: (test driver plays "buyer")
Step 1: Buyer Send a P.O. to Supplier.
Step 2: Receive a signal ack, correlated with the P.O. based on Conversation ID
Step 3: Receive a P.O. Confirmation or Rejection.
Step 4: Send a signal Ack.

Constraints: 
- total time for 1-2-3 is bounded by TimeToPerform.
- total time for 1-2 is bounded by TimeToAcknowledge.

Success:
- within TimeToPerform, receive either a P.O. Confirmation or Rejection, but not both.
- TimeToAcknowledge is satisfied.

Variant:
The correlation is based on a PO reference # that is in the payload.


  -------------------------------------- 
  Use Case #2: Catching unexpected ebXML Error messages (a test case "design pattern") 
  -------------------------------------- 

  Test Case: (test driver plays "buyer")

  Step 1: buyer sends a message M1 
  Step 2: receive and verify message M2 
  

  Correlation M1-M2 based on COnversation ID.
But in case an error is received that correlates with M1 (in addition or instead of M2), 
at any point in time within 300sec after sending M1, before or after receiving M2, 
and regardless of the outcome of verifying M2, we want the test case to fail. 


 
  -------------------------------------- 
  Use Case #3: conditional branching 
  -------------------------------------- 
  Test Case: (test driver plays "buyer")

  - buyer sends M1 
  - received M2 (e.g. an approval, or rejection) 
  ---> if M2 is "approval", we will expect a sequence of: 
- receive M3 (e.g. a quotation)  
- send M4 (e.g. approval of quote and payment info) 
- receive M5 (final delivery details). 
  ---> if M2 is "rejection", we will expect a sequence of: 
- receive M6 (proposed alternative)
  
We need to verify all received messages, as the test case would fail if they do not comply. 
Any error message received, would also fail the test case (see use case #2)