OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [ebxml-iic] Use cases for concurrent test cases - Schema and instance examples


Title: Use cases for concurrent test cases
Jacques,
 
   I've attached my response to your comments regarding scripting structure.  I am still working
on a use case for paramter/variable scoping, and the impact of concurrency and nesting. 
 
   I've attached the previously posted  test cases and examples for reference.
I will post new use case instances and schema incorporating Jacques and other's suggestions
tomorrow.
 
 
Comments from implementers are encouraged.
 
Thanks,
Mike
 
----- Original Message -----
Sent: Friday, January 23, 2004 10:05 PM
Subject: RE: [ebxml-iic] Use cases for concurrent test cases - Schema and instance examples

Mike:
 
Here is attached my review of your scripting of the 3 test cases
that you had sent before.
Some of my comments illustrate issues you mention below,
so no more inline comments added to yours.
 

One issue that still remains, and looms large, now that we've adopted asynchronous functionality, and nesting of <TestSteps>  is "scoping" and "synchronizing"  parameter/variable references between <TestSteps>.  This is a tricky subject, and we need to define how this will work.

 

can you come up with a concrete example? I believe the same rules still apply :

what counts is the sequence of execution. Scope should be entire test case I think.

 

Jacques
 
 
-----Original Message-----
From: Michael Kass [mailto:michael.kass@nist.gov]
Sent: Thursday, January 22, 2004 12:20 PM
To: Jacques Durand; ebXML IIC - main list (E-mail) (E-mail)
Subject: Re: [ebxml-iic] Use cases for concurrent test cases - Schema and instance examples

Jacques,

 

     I've modified the Test Framework scripting schema to reflect your suggestions for a

"minimal thread" approach. (in your orignal email below ).     I've attached a schema graphic, along with the 3 use cases as

an XSL transformation to HTML ( for easier viewing ).  In addition, I am attaching a ZIP file containing

all of the XML instances and modified schemas.  Here are my comments for each of the suggestions

you provided:

 

- A Test Case could be seen as a related set of Test Threads. [Mike] - Done.   Plus I added the functionality that a <TestCase> can additionally  be a set of <Thread>s, and/or <ThreadRefs>  ( for reusability of <Thread>s via name attribute)

 

 

-  A Test Thread is identified by a "thread name" and is a sequence of test steps.  [MIKE] Done.  Added <Thread> as a tag, with name as an attribute.    I expanded a <Thread> to be either a sequence of <TestSteps> and/or <TestStepRefs> ( for reuseability )  and/or <If> statements

 

- The simplest test case is just one test thread. [MIKE] - Agreed

 

- An if-then-else is forking to either a single step, or to a thread name.  [MIKE] -Why not make if-then-else a AND/OR Split?  Then we could do more creative types of evaluations .. like <If> (a AndSplit  b ) <THEN> (c), <ElseIf> (f OrSplit g) then (h) else  (i)  (I did this with the schema, by providing  the <If> element with a "splitType" attribute. Then all child <Threads> or <TestSteps> are an <And> or <Or> operation.
[Jacques Durand] I believe split has different semantics 

 

-  At any point in a test thread, a test step could be a thread invocation. [MIKE]-  I suggest keeping a <TestStep> as the lowest common denominator ( only able to do a PutMessage or GetMessage operation ).  This provides a clear delineatino between a <TestStep> ( an atomic operation ) and a <Thread> (a series of one or more <TestSteps> )

This is reflected in the schema

 

-  A thread invocation is of one of two forms: sync(threadname), async(threadname). [MIKE] - Done.  Provided a "type" attribute to <Thread> element ( asynchronous, or synchronous )

 

-  An asynchronous thread invocation will run in parallel to the caller (a kind of "split").[MIKE] - Agreed

 

-  A synchronous thread invocation will block the caller until is complete (e.g., if-then-else,

or a kind of subroutine) [MIKE] - Agreed

 

- with this model, an Exception handling (e.g. handling an error message)

could be defined as a thread (i.e. possibly several steps). [MIKE] - Agreed

 

One issue that still remains, and looms large, now that we've adopted asynchronous functionality, and nesting of <TestSteps>  is "scoping" and "synchronizing"  parameter/variable references between <TestSteps>.  This is a tricky subject, and we need to define how this will work.

 

Comments welcome,

 

Mike

----- Original Message -----
Sent: Monday, January 12, 2004 8:10 PM
Subject: [ebxml-iic] Use cases for concurrent test cases

Following discussion today, Here is some test cases that we need to
be able to script:

--------------------------------------
Concurrency Use Case #1: Exception handling
--------------------------------------

 Test Case:
- send a message M1
- receive message M2
- verify M2

But in case an error is received that correlates with M1, at any point in time
within 300sec after sending M1, before or after receiving M2, and regardless of
the outcome of verifying M2, we want the test case to fail.

--------------------------------------
Concurrency Use Case #2: workflow split
--------------------------------------
Test Case:
- send M1
(---> this will trigger two concurrent subprocesses 2 and 3 on the remote side,
which will send back two threads of messages: M2a + M2b for subprocess 2, and M3 for
subprocess 3. There is no order between subprocesses 2 and 3 )
- split {(receive M2a; verify M2a; receive M2b; verify M2b)(receive M3; verify M3)}
(---> then when M2a , M2b and M3 have been verified, one last message M4 is expected)
- receive message M4
- verify M4

--------------------------------------
Conditional Use Case #3: conditional branching
--------------------------------------
Test Case:
- send M1
- received M2 (e.g. an approval, or rejection)
(---> if M2 is "approval", we will expect a sequence of: receive M3 + send M4 + receive M5.
if M2 is "rejection", we will expect a sequence of: receive M6)
We need to verify all received messages, as the test case would fail if they do not comply.

Regards,

Jacques

ÿþ<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"><head xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"> <META http-equiv="Content-Type" content="text/html; charset=UTF-16"> <title>ebXML Test Suite Author: Michael Kass Organization: NIST Date: 05/15/2003 Version 1.0 </title><link rel="stylesheet" type="text/css" href="xmlverbatim.css"></head><center xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"><h2>POC for BPSS testing: Case 1</h2></center><table border="2" xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"><tr><td width="10%" align="center"><h4>Test Object</h4></td><td width="10%" align="center"><h4>ID</h4></td><td width="10%" align="center"><h4>Description</h4></td><td width="10%" align="center"><h4>Mode</h4></td><td width="10%" align="center"><h4>Operation</h4></td><td width="10%" align="center"><h4>Configuration</h4></td><td width="40%" align="center"><h4>Message Expression</h4></td></tr><tr><td width="10%" align="left">Test Suite</td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="left"></td><td width="40%" align="center"></td></tr><tr><td><b> Test Case </b></td><td><i><b>testcase_1</b></i></td><td><i><b>POC of exception handling</b></i></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='synchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_01_01</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Send a message m1</td><td></td><td>PutMessage</td><td></td><td width="50%"></td></tr><tr><td>TestStep</td><td>step_01_02</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='asynchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_03</td><td>Test that an Error message was NOT received, any time within 300 seconds after step_01</td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Filter any ErrorList elements in the MessageStore</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore//Message//[eb:ErrorList]</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Error is NOT present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message[count()=0]</td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_01_03</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message M2</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//[eb:MessageHeader[eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId='m2']</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr></table>ÿþ<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"><head xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"> <META http-equiv="Content-Type" content="text/html; charset=UTF-16"> <title>ebXML Test Suite Author: Michael Kass Organization: NIST Date: 05/15/2003 Version 1.0 </title><link rel="stylesheet" type="text/css" href="xmlverbatim.css"></head><center xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"><h2>POC for BPSS testing: Case 2</h2></center><table border="2" xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"><tr><td width="10%" align="center"><h4>Test Object</h4></td><td width="10%" align="center"><h4>ID</h4></td><td width="10%" align="center"><h4>Description</h4></td><td width="10%" align="center"><h4>Mode</h4></td><td width="10%" align="center"><h4>Operation</h4></td><td width="10%" align="center"><h4>Configuration</h4></td><td width="40%" align="center"><h4>Message Expression</h4></td></tr><tr><td width="10%" align="left">Test Suite</td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="left"></td><td width="40%" align="center"></td></tr><tr><td><b> Test Case </b></td><td><i><b>testcase_2</b></i></td><td><i><b>Workflow Split</b></i></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='synchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_01</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Send a message m1</td><td></td><td>PutMessage</td><td></td><td width="50%"></td></tr><tr bgcolor="00FF00"><td>If</td><td></td><td>AndSplit</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='asynchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_02_01</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message m2a</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m2a"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr><td>TestStep</td><td>step_02_02</td><td>Verify Response Message m2b</td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m2b"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='asynchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_02_03</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message m3</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m3"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="0000FF"><td>Then</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_04</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message m4</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m4"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="0000FF"><td>End Then</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="00FF00"><td>End If</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr></table>ÿþ<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"><head xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"> <META http-equiv="Content-Type" content="text/html; charset=UTF-16"> <title>ebXML Test Suite Author: Michael Kass Organization: NIST Date: 05/15/2003 Version 1.0 </title><link rel="stylesheet" type="text/css" href="xmlverbatim.css"></head><center xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"><h2>POC for BPSS testing: Case 3</h2></center><table border="2" xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:ebTest="http://www.oasis-open.org/tc/ebxml-iic/tests"><tr><td width="10%" align="center"><h4>Test Object</h4></td><td width="10%" align="center"><h4>ID</h4></td><td width="10%" align="center"><h4>Description</h4></td><td width="10%" align="center"><h4>Mode</h4></td><td width="10%" align="center"><h4>Operation</h4></td><td width="10%" align="center"><h4>Configuration</h4></td><td width="40%" align="center"><h4>Message Expression</h4></td></tr><tr><td width="10%" align="left">Test Suite</td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="center"></td><td width="10%" align="left"></td><td width="40%" align="center"></td></tr><tr><td><b> Test Case </b></td><td><i><b>testcase_3</b></i></td><td><i><b>Conditional Branching</b></i></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='synchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_01</td><td>Send a message M1</td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Send a message</td><td></td><td>PutMessage</td><td></td><td width="50%"></td></tr><tr bgcolor="00FF00"><td>If</td><td></td><td>AndSplit</td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_02</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message m2 and test if message is an 'Approval' </td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m2"))</td></tr><tr bgcolor="FFFFFF"><td>Precondition</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Get message payload</td><td></td><td>GetPayload Content-Id = 'cid:response'<br></td><td></td><td width="50%"></td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify that message is an 'approval' </td><td></td><td>VerifyContent</td><td></td><td width="50%">//Approval</td></tr><tr bgcolor="0000FF"><td>Then</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>Thread</td><td></td><td> type='synchronous' </td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_02_01</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message m3 and test if message is an 'Approval' </td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m3"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr><td>TestStep</td><td>step_02_02</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Send a message m4</td><td></td><td>PutMessage</td><td></td><td width="50%"></td></tr><tr><td>TestStep</td><td>step_02_03</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve message m5</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m5"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="0000FF"><td>End Then</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="00FFFF"><td>ElseIf</td><td></td><td>AndSplit</td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_03</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve Response message m2 and test if message is an 'Rejection' </td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m2"))</td></tr><tr bgcolor="FFFFFF"><td>Precondition</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Get message payload</td><td></td><td>GetPayload Content-Id = 'cid:response'<br></td><td></td><td width="50%"></td></tr><tr bgcolor="FFFFFF"><td>Precondition</td><td></td><td>Verify that message is an 'approval' </td><td></td><td>VerifyContent</td><td></td><td width="50%">//Rejection</td></tr><tr bgcolor="0000FF"><td>Then</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>TestStep</td><td>step_03_01</td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FFFFFF"><td></td><td></td><td>Retrieve message m6</td><td></td><td>GetMessage</td><td></td><td width="50%">/MessageStore/Message//(eb:MessageHeader(eb:ConversationId=$ConversationId and eb:Action="Mute" and eb:MessageData/eb:MessageId="m6"))</td></tr><tr bgcolor="FFFFFF"><td>Assertion</td><td></td><td>Verify Response is present</td><td></td><td>VerifyContent</td><td></td><td width="50%">//Message(count()=1)&gt;</td></tr><tr bgcolor="0000FF"><td>End Then</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="00FFFF"><td>End ElseIf</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="00FF00"><td>End If</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr bgcolor="FF0000"><td>End Thread</td><td></td><td></td><td></td><td></td><td></td><td></td></tr></table>

scripting_schema.jpg

scripting_issues_jd_mk.doc



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]