RE: [ebxml-iic] scripting of test cases: latest advances

Jacques,

My comments below:

-----Original Message-----
From: Jacques Durand [mailto:JDurand@us.fujitsu.com]
Sent: Friday, June 04, 2004 10:14 PM
To: 'Michael Kass'; Jacques Durand; ebXML IIC - main list (E-mail) (E-mail)
Cc: tsakach@certivo.net
Subject: RE: [ebxml-iic] scripting of test cases: latest advances

Mike:

inline (JD-3)

[MIKE2] - Thinking about this more, I am wondering if we should just "do away" with <TestStep> entirely, and treat everything as a <Thread> (i.e. merge the operations of a <TestStep> in to <Thread>). While this removes the meaning of a <TesStep> as a container for common message request/receive operations and test assertion verification, it would simplify the testing model even more. Plus it would permit a simpler syntax for recursion:
[Jacques Durand 2] If a Test Step is a container that can be as complex as a thread, then lets do away with it ! I thought it was jsut for one message op at a time, either GetMessage or PutMessage, but no more than one. I think it is still useful to have a notion of container even for atomic operations, because we can attach some properties to it that can then be common to any contained op, like timeout, etc. So I'd keep the notion of TestStep, but only containing one main op statement at most, plus some variable setting statements if needed?

[MIKE3] - I think that a <Thread> can still have context (like <TestStep> for a contained operation ( such as a PutMessage or GetMessage ). And since, <Thread> will perform the same operations, an "atomic" level <Thread> could simply be one that (optionally ) does one single operation instead of multiple <PutMessage> and <GetMessage> operations. However, it is an advantage to be able to cluster <PutMessage> and <GetMessage> operations together in a single <Thread>, since that is realistically more in line with what a business transaction is ( i.e. as described in BPSS ). Plus, we keep the context ( CPAId, ConversationId ) the same for all of those operations.. instead of having to "pass" that context to the next sibling <TestStep> ( which creates parameter scoping problems anyway). So I am in favor of unifying the functionality of a <Thread> and <TestStep> in to the <Thread> class, and eliminating <TestStep> entirely.

[Jacques Durand 3] I am all for getting rid of unneeded constructs. My concern with using "thread" basically instead of "TestStep" is that it blurrs the thread semantics: now threads can be nested, without being "forked" (split),

and each operation could be in its own little thread...all that could be confusing.

In addition, a thread cannot pretend to be atomic like a test step was: if we get rid of test step, the basic building block (the "op") is the atomic unit of execution.

[MIKE4] – I think that this is acceptable. All that we’ve done is removed the <TestStep> container for the op.

So far we have succeeded in keeping "thread chaining" (or composition) simple, aligned with WFlow practice: a thread only passes control to other threads via branching operators (split / join, within a conditional or not). A thread is not a bloc structure, but is a flat sequence of WFlow activities that can spawn other threads while controlling their concurrency. Can we keep it that way?

[Mike4] – If we use the example you have below ( using <Split> and <Join> ) we can. If it is necessary to use <Split> and <Join> to maintain workflow semantics, then I say let’s use it.

So I don't have issue with grouping ops in smaller threads if needed, as long as we stick to the control ops above (split / join) and avoid any nesting. I believe so far our use cases don't need more than this.

[MIKE4] – OK. Let’s use <Split><Join> forking child <Thread>s, but let’s get rid of <TestStep>. It is an unnecessary construct

So I am not sure that we really need to use "small threads" (one op, or a "clusters"), but if we do, then we can do as follows (and pass context at the time we "split" (how?)):

</thread>

<put...>

</thread>

<get...>

<put...>

</thread>

So even if that chaining above seems a bit heavy, I am not sure it will be needed: I don't see that our Use Cases require this.

[Mike4] - I’m coding the use cases now, and will see if the constructs above will be necessary.

We could still eliminate TestStep , but only if we really can do without having to use instead "atomic Threads"...

I think your concern about contexts (see my next comment later on) can be addressed without need for "clustering" Ops and Assertions)

[Jacques Durand] Again, we need to justify the logic of what is a Test Step. I'd say it is an atomic unit of execution, but it is NOT a control flow operator like split, or now Assertion (e.g. cannot branch somewhere else). Should it include sleep()?

[MIKE2] - Under our new definition, <TestStep> is no longer an "atomic" level of execution.. since we can now <Split> Threads from within a TestStep. This blurs the distinction between <Thread> and <TestStep>, and raises the question if we should not simply have <Threads> alone, capable of doing all the operations of a TestStep.
[Jacques Durand 2] so again if we take Assertion (or TestAssertion) out of a tset step, as a standalone control operator for branching or exiting, then we would not split() in any way from inside a test Step, and Test Step could remain an atomic unit of execution.

[MIKE3] - This is true. But branching only at the higher <Thread> level means you lose the "context" ( i.e. the current <FilterResult> .. the current ConversationId, CPAId.. and other context of the <TestStep>). That was the original purpose of <TestStep> .. to provide the low level context to which you refer previously in this discussion. So really, one must either keep <TestStep> for the purpose of maintaining context and branching based upon a <TestAssertion> within a <TestStep>.. or (what I favor).. eliminating <TestStep>.. and giving a <Thread> all of a <TestStep>'s properties. That way, context is always maintained when you <Split> a <Thread>. Comments?

[Jacques Durand 3] I favor the latter too... I think Thread-level is good-enough granularity for context management, our Use Cases don't need more than that I believe. Also, we have to keep in mind that now the Assertion, as a conditional operator, should be able to access the FilterResult of more than one Get operation, e.g. compare the material from two messages obtained separately within the same thread. So I think it is not so important to associate Assertion with a FilterResult (could be the latest one produced in the thread by default), and generally we'll need to explicitly identify FilterResults, and reference their ID in Assertion !

` [MIKE4] – Agreed. On most of the above. I’m not sure that we’ll need to maintain all <FilterResults>.. could just be that a <TestAssertion> refers (by default) to the latest <FilterResult>

[MIKE4] - I have some additional suggestions for simplifying scripting.. chief one is: eliminating <GetPayload> operation… and instead, combining XML payloads with the message associated envelope in the Message Store (i.e treat the whole message ( including XML payloads ) as a single message in the MessageStore. You could still refer to individual payloads via a “contented” attribute. This could greatly simplify Test Driver design, as well as further simplify test case scripting.

So you have an XPath query construct of the Message Store like:

TEST:MessageStore/Message/Payload [@contented=”cid:PurchaseOrderResponse”]/PurchaseOrderResponse….

And treat XML payloads as just another part of the message. Binary payloads would not be stored. Comments?

ebxml-iic message