[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [ebxml-iic] Use cases for concurrent test cases - Schema and instance examples
|
Mike:
Here
is attached my review of your scripting of the 3 test cases
that
you had sent before.
Some
of my comments illustrate issues you mention below,
so no
more inline comments added to yours.
One issue that still remains, and looms large, now that we've adopted asynchronous functionality, and nesting of <TestSteps> is "scoping" and "synchronizing" parameter/variable references between <TestSteps>. This is a tricky subject, and we need to define how this will work.
can you come up with a concrete example? I believe the same rules still apply : what counts is the sequence of execution. Scope should be entire test case I think.
Jacques
|
--------------------------------------
Concurrency Use Case #1: Exception handling
--------------------------------------
Test Case:
- send a message M1
- receive message M2
- verify M2
But in case an error is received that correlates with M1, at any point in time
within 300sec after sending M1, before or after receiving M2, and regardless of
the outcome of verifying M2, we want the test case to fail.
-------------------
<JD> Overall, your scripting is working, but I find the embedding of threads clumsy:
- it is not very readable.
- in case the same thread needs be started several times in same test case,
we need to avoid embedding it each time: I assume we'll just start it by referring to its name.
- embedding the full sequence of steps of a thread, inside the calling thread, is misleading:
it seems that the step #3 in use case 1 needs to wait for the end of the embedded thread,
while it is not true.
So I propose to modify your test case for UC 1 (Exception Handling) as follows:
Instead of the following embedding pattern you have in your example:
Test Case 1:
Thread:
( Test Step 1.01 ( Put ) ;
Test Step 1.02 ( async Thread: Test Step 03 (Get, assertion )) ;
Test Step 1.03 ( Get, assertion )
We could dissociate the threads (and use their names):
Test Case 1:
Thread: "main"
{ Test Step 1.01 ( Put ) ;
Test Step 1.02 ( async thread("Exception")) ;
Test Step 1.03 ( Get, assertion )
}
Thread: "Exception"
( Test Step 2.01 (Get, assertion) }
Also, the script is not explicit about the 300sec limit:
There are several ways we could handle this:
- time the Get operation itself, by adding to it an optional "timeout" parameter.
If the Get could not select a message within the timeout, the test step proceeds further
here with the "Assertion" op, which would verify no error received.
- introduce a new type of step, a "sleep", which would block the thread some amount
of time before passing control to the next step. (note that could be useful generally).
Finally, a more general question: now that we have several threads in a tset case,
should we always require that ALL threads terminate successfully, for the test case to succeed?
- It could be more natural to state that if thread X terminates on a successsful assertion
(e.g. Error message is obtained, and matches requires condition) then this causes the
entire test case to fail. That will be generally what Exceptions threads are for.
For this we could use an explicit qualifier "fail" or "pass" for the final Assertions
(if the Assertion is verified, the qualifier decides of the entire test case: "fail" or "pass").
- without pass/fail qualifiers, the thread execution simply has boolean semantics, for
the overall flow control of the case ("if (thread A) then ...").
- Conversely, it could be that it is enough for thread XYZ to terminate successfully,
for the test case to succeed, regardless of how other threads do. Then we would qualify
its last assertion with "pass".
- note that we could acheive the same effect with a pass or fail action:
if (thread A) then "fail";
or
if (thread A) then "pass" else "fail";
So a thread in itself would not decide of the total outcome (except for the main thread,
which has a default of: if (thread main) then "pass" else "fail";)
--------------------------------------
Concurrency Use Case #2: workflow split
--------------------------------------
Test Case:
- send M1
(---> this will trigger two concurrent subprocesses 2 and 3 on the remote side,
which will send back two threads of messages: M2a + M2b for subprocess 2, and M3 for
subprocess 3. There is no order between subprocesses 2 and 3 )
- split {(receive M2a; verify M2a; receive M2b; verify M2b)(receive M3; verify M3)}
(---> then when M2a , M2b and M3 have been verified, one last message M4 is expected)
- receive message M4
- verify M4
<JD> In this script, the "if...then...endThen" seems unnecessary:
also the notion of "AndSplit" does not really match common workflow notions:
subthreads are always all started and concurrently in a split (so there is no such thing
as an or-split I believe).
It seems that here we don't need more than split, or-join, and-join which are
known workflow operators.
Split (A, B) is actually equivalent to a sequence { async thread(A); async thread(B) }
so maybe we don't even need a split.
We still need a join, here and-join to wait for all threads,
before proceeding further in the parent thread.
So it seems to me could have:
Thread: "main"
- step 1;
- split (thread A, thread B)
- and-join ( A, B)
- step 4;
--------------------------------------
Conditional Use Case #3: conditional branching
--------------------------------------
Test Case:
- send M1
- received M2 (e.g. an approval, or rejection)
(---> if M2 is "approval", we will expect a sequence of: receive M3 + send M4 + receive M5.
if M2 is "rejection", we will expect a sequence of: receive M6)
We need to verify all received messages, as the test case would fail if they do not comply.
<JD> In this case, as it is either one thread or the other, we probably need no "split"
(I think we have a misunderstanding on the split semantics)
only an "if (assertion) then (thread A) else (thread B)" would be sufficient.
Now, should we use the "if (...) " at assertion level, instead of thread level?
because we need to branch within a thread, like here, in some cases.
So both could be possible in general.
the "Then" and "Else" would still start threads (synchronous in our use case, as you pointed out).
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]