[ebxml-iic] tests material comments: on test steps and conditions

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Mike:

 

some comments on yours, (we seem to converge bit by bit...)

 

cheers,

 

jacques

>  
Commetns in line <Jacques2>

--------------------------------------------------------------------

Please see my comments below


At 03:50 PM 9/23/2002 -0700, Jacques Durand wrote:

  Mike:
   
  I like the new test material upgrades definitely better...Some comments on your latest draft of MS Test Cases:
   
  - we need also to set the CPAId of messages we generate: that would set the CPA
  that should govern each test case. (the new "$CPA_xyz" parameters would allow to 
  use elements of such referenced CPAs, in "Conditions)
   
  - The new operation "Condition" should only include logical conditions and relational operators
  (<,>,==,!=), not assignments (=) like the XPAth expr used to prepare a message to be sent out (for a PutMessage).
  So these assignments could be left as arguments of the PutMessage op (in its "message expression" field)
  like they were before.
   
  - just a matter of format: The arguments for SetPayload, (Content-Type =..., COntent-Id =...) should be treated
  like any other "message expressions" (so should be in this field, like the XPath expressions).



[MIKE] - Agreed and done.  See latest attachment



  - The semantics of the Condition attached to a GetMessage op,  is one of a filter (that typically selects
  a message based on COnversationID and RefToMessageId, to actually decide whether or not to pick
  an incoming message, as further test material.)
  It is different than other COnditions, which only work on message material already in store, from past steps. 
  For this reason, I'd keep the "filter" conditions in "message Expressions" of the GetMessage op itself (i.e. they
  are "parameters" of GetMessage, as they help do the selection). 


[MIKE] - Agreed.. I'd go a step further and suggest that we assume a standard "filter of CPAId, ConversationId and
MessageId.. and not even write out the filter statement for <GetMessage>.. this more likely will fit the ultimate
implementation XML of:
<GetMessage cpaId="abc" conversationId='def' messageId='ghi'>

    <Condition>.....</Condition>  <!-- this only applies to real condition/conformance test filtering -->
    <Condition>..</Condition>      <!-- this only applies to real condition/conformance test filtering -->

</GetMessage>

<Jacques2> standard "filters" are fine: 
(would these attribute names correspond to the "parameter" names?)
 Still we need (in addition, or even sometimes instead) to make it possible to filter based 
on ad-hoc filter-conditions. I assume in such case these would go in <Condition> children elements?
For the conformance verification, my main motivation in dissociating the test conformance condition 
(verification) from GetMessage, is that normally the conformance test can use material from
any previous step - in more complex test cases, we could get more than one GetMessage op, 
and have a verification test comparing both, etc.)
So we could indeed still keep it embedded inside the latest GetMessage step, but we should be able
to reference in some way the material from previous steps.

Another reason to clearly distinguish "filter" conditions from any other condition,
is that they have a different semantics: 
the former decide whether or not to select and keep the message(s) in store, for next verifications.
while other conditions merely verify if whatever message in store satisfy some requirement.
So if we want to select only those messages that verify an ad-hoc condition, we shouldmake
sure to associate the ad-hoc condition as a filter of GetMessage: (e.g. using a "FilterCondition"
element):

Case 1:
<GetMessage cpaId="abc" conversationId='def' >
    <FilterCondition>my ad-hoc filter "C1" </FilterCondition>  
    <Condition> any other condition "C2" (some pre-condition, conformance verification...)</Condition>      
</GetMessage>

May have entire different outcome than:

Case 2:
<GetMessage cpaId="abc" conversationId='def' >
    <Condition>"C1" </Condition>  
    <Condition> "C2"</Condition>      
</GetMessage>

And also different outcome than:

Case 3:
<GetMessage cpaId="abc" conversationId='def' >
    <FilterCondition>"C1" </FilterCondition>  
    <FilterCondition> "C2"</FilterCondition>      
</GetMessage>


Indeed, in case 1, the first message that satisfies the standard filters + C1, will be selcted, 
and then checked for C2, 
while in Case 2, first message received that satisfies standard filter is stored, and then used to check C1 + C2.
in Case 3, first message that satisfies standard filter + C1 + C2 is stored. If none, this fails.
Note that, in case several message candidates may match the standard filter (general case),
then (1) (2) (3) may each select different messages !!!
(e.g. (2) has the most chance to fail, as it may miss a "good" message that would be selected by (3).
Same for (1) which is in-between.
</Jacques2> 


It will also simplify the look of the abstract tests.  Comments?


  For some other "Condition" operations,
  (like the ones on "optional" material, or the ones used for final verification) 
we could dissociate them from GetMessage 
  and make them really separate ops (or steps)... 
  (Should we "flatten" our steps, and have roughly each one of your table row correspond 
to a step - after Get and Put
  have their related expressions - message builders and filters - back in their 
"message expr" fields?) 
   


[MIKE] -  My concern with flattening has to do with associating <Condition>s with a particular <GetMessage>...
The way things are currently defined we have:

<TestCase>
   <Step>
       <PutMessage/>
   </Step>
   <Step>
      <GetMessage>
          <Condition1/>
          <Condition2/>
      </GetMessage>
</TestCase>

You are suggesting:

<TestCase>
   <Step>
      <PutMessage/>
   </Step>
   <Step>GetMessage></Step>
   <Step>
      <Condition1/>
   </Step>
   <Step>
      <Condition2/>
   </Step>
</TestCase>


By flattening. it appears that you lose the containership "parent/child" relationship 
between a <GetMessage> a particular message,
and all of the <Condition> filters associated with it.  Everything's a step, with no 
association between the
steps.  That is why XML containership is important, in my opinion. Even though we are 
describing this in an "abstract" sense, there
are some real-world consequences if we do not adequately describe the relationships 
between our test components.
When we code the test driver, this will be important.

<Jacques2> so we may not have to "flatten" anymore, if we
clearly identify "filter" conditions to associate with GetMessage as I just proposed. 
Besides this, I do not have problem with including other conditions inside GetMessage step,
but we must then make sure we can use references to material from other previous 
steps from these (default message material would be the one selected by current GetMessage).
</Jacques2> 

Tables like the one we are using for describing the Abstract Tests are not well suited 
to represent containership. I 
represent that containership through the use of the Step # column, with the only steps 
defined as either a <SetMessage>
or <GetMessage>.

Using object oriented principles, these <Steps> could return any number of exceptions, 
depending on which object generates
a which type of a fatal exception.  Looking at this now, I would probably go a step 
further and using object oriented design,
change the Condition class to be <TestCondition> or <Condition>.. one throws a 
"fatalTest" exception, the other a "fatalPrecondition" 
exception... rather than having to create an attribute called "errorStatus" that 
the test writer must assign a value.  So we would
have as potential classes:
<SetMesssage>
<GetMessage>
<TestCondition>
<Condition>
<SetPayload>
<GetPayload>

My feeling is, even though we are describing "abstract" tests, we should consider the 
implementation consequences of how we
are describing these tests.

I will respond to your other comments separately, within the associated mail message.

Thanks,
Mike