[ebxml-iic] Re: tests material comments: on test steps and conditions

[Michael Kass <michael.kass@nist.gov>]

Jacques,

I've added my responses to your questions.  Let me know what you
think,
and what we should ultimately do to move forward with finalizing
abstract
test description (mapping to underlying XML schema).

Thanks,
Mike
  



Commetns in line <Jacques2>

--------------------------------------------------------------------

Please see my comments below


At 03:50 PM 9/23/2002 -0700, Jacques Durand wrote:

  Mike:
   
  I like the new test material upgrades definitely better...Some comments on your latest draft of MS Test Cases:
   
  - we need also to set the CPAId of messages we generate: that would set the CPA
  that should govern each test case. (the new "$CPA_xyz" parameters would allow to 
  use elements of such referenced CPAs, in "Conditions)
   
  - The new operation "Condition" should only include logical conditions and relational operators
  (<,>,==,!=), not assignments (=) like the XPAth expr used to prepare a message to be sent out (for a PutMessage).
  So these assignments could be left as arguments of the PutMessage op (in its "message expression" field)
  like they were before.
   
  - just a matter of format: The arguments for SetPayload, (Content-Type =..., COntent-Id =...) should be treated
  like any other "message expressions" (so should be in this field, like the XPath expressions).



[MIKE] - Agreed and done.  See latest attachment



  - The semantics of the Condition attached to a GetMessage op,  is one of a filter (that typically selects
  a message based on COnversationID and RefToMessageId, to actually decide whether or not to pick
  an incoming message, as further test material.)
  It is different than other COnditions, which only work on message material already in store, from past steps. 
  For this reason, I'd keep the "filter" conditions in "message Expressions" of the GetMessage op itself (i.e. they
  are "parameters" of GetMessage, as they help do the selection). 


[MIKE] - Agreed.. I'd go a step further and suggest that we assume a standard "filter of CPAId, ConversationId and
MessageId.. and not even write out the filter statement for <GetMessage>.. this more likely will fit the ultimate
implementation XML of:
<GetMessage cpaId="abc" conversationId='def' messageId='ghi'>

    <Condition>.....</Condition>  <!-- this only applies to real condition/conformance test filtering -->
    <Condition>..</Condition>      <!-- this only applies to real condition/conformance test filtering -->

</GetMessage>

<Jacques2> standard "filters" are fine: 
(would these attribute names correspond to the "parameter" names?)
 Still we need (in addition, or even sometimes instead) to make it possible to filter based 
on ad-hoc filter-conditions. I assume in such case these would go in <Condition> children elements?
For the conformance verification, my main motivation in dissociating the test conformance condition 
(verification) from GetMessage, is that normally the conformance test can use material from
any previous step - in more complex test cases, we could get more than one GetMessage op, 
and have a verification test comparing both, etc.)
So we could indeed still keep it embedded inside the latest GetMessage step, but we should be able
to reference in some way the material from previous steps.

Another reason to clearly distinguish "filter" conditions from any other condition,
is that they have a different semantics: 
the former decide whether or not to select and keep the message(s) in store, for next verifications.
while other conditions merely verify if whatever message in store satisfy some requirement.
So if we want to select only those messages that verify an ad-hoc condition, we shouldmake
sure to associate the ad-hoc condition as a filter of GetMessage: (e.g. using a "FilterCondition"
element):

[MIKE2] - Well, perhaps we should express <GetMessage> the same way we now express <PutMessage> in the
abstract ( and ultimately "real" tests )... by putting the <FilterCondition> content directly into the
<GetMessage> content.  I believe in XML it would be expressed as: 

<GetMessage>
<Condition> any other condition "c2" </Condition>
/SOAP:Envelope/SOAP:Header/eb:MessageHeader/eb:CPAId==$CPA_Id 
and /SOAP:Envelope/SOAP:Header/eb:MessageHeader/eb:ConversationId==’$ConversationId’ 
and /SOAP:Envelope/SOAP:Header/eb:MessageHeader/eb:MessageData/eb:RefToMessageId==’$MessageId’
</GetMessage>

We could express it the same way in the abstract tests ( no longer having a <Condition> to describe this.. but actually
including it on the same line as the <GetMessage>

That way, you end up with the <FilterCondition> expression automatically embeded into the <GetMessage> element.
Any child <Condition> elements would either be "precondition" or "conformance test" filters.
While we are on the subject, wouldn't it be better to "extend" the <Condition> element into <Precondition> and
<ConformanceTest> filters?  That way, the user would not have to differentiate the <Condition> with the
"errorStatus" attribute value ( fatalPrecondtion or fatalTest ).  This seems more logical, and more
object oriented.  Each of those test objects would return a different errorStatus result based upon success
or failure... So what we could have is: ( this is how we express embedded content mixed with child elements )


<GetMessage>
<Precondition> my pre-condition filter expression "C1" </Precondition>
<ConformanceTest> my conformance test filter exprfession "C2" </ConformanceTest>
/SOAP:Envelope/SOAP:Header/eb:MessageHeader/eb:CPAId==$CPA_Id 
and /SOAP:Envelope/SOAP:Header/eb:MessageHeader/eb:ConversationId==’$ConversationId’ 
and /SOAP:Envelope/SOAP:Header/eb:MessageHeader/eb:MessageData/eb:RefToMessageId==’$MessageId’
</GetMessage>


Case 1:
<GetMessage cpaId="abc" conversationId='def' >
    <FilterCondition>my ad-hoc filter "C1" </FilterCondition>  
    <Condition> any other condition "C2" (some pre-condition, conformance verification...)</Condition>      
</GetMessage>

May have entire different outcome than:

Case 2:
<GetMessage cpaId="abc" conversationId='def' >
    <Condition>"C1" </Condition>  
    <Condition> "C2"</Condition>      
</GetMessage>

And also different outcome than:

Case 3:
<GetMessage cpaId="abc" conversationId='def' >
    <FilterCondition>"C1" </FilterCondition>  
    <FilterCondition> "C2"</FilterCondition>      
</GetMessage>




Indeed, in case 1, the first message that satisfies the standard filters + C1, will be selcted, 
and then checked for C2, 
while in Case 2, first message received that satisfies standard filter is stored, and then used to check C1 + C2.
in Case 3, first message that satisfies standard filter + C1 + C2 is stored. If none, this fails.
Note that, in case several message candidates may match the standard filter (general case),
then (1) (2) (3) may each select different messages !!!
(e.g. (2) has the most chance to fail, as it may miss a "good" message that would be selected by (3).
Same for (1) which is in-between.
</Jacques2> 


It will also simplify the look of the abstract tests.  Comments?


[MIKE2] - I agree with you you say above. We need to be clear/unambiguous about the meaning of the abstract test
design.. because ultimately it must be uniform.. and map to our XML schema design.
I think that we agree with the general approach here.  We just need to agree on the final syntax.

[MIKE2] - Also, I believe that <PutMessage> and <GetMessage> should look as much "alike" as possible, to make it easy for
test developers to interpret our meaning, and map to XML elements and attributes.
I believe that the XML expression I described above ( embedding the main "filter" expression, defining a 
<Precondition> and <ConformanceTest> element/class as children, will satisfy both of our requirements, and will
create a "uniform" look to our abstracts tests, while providing an easily understandable and codeable test case.

[MIKE2] - The one thing that we still need to "hash out" is where the MIME attributes should be described and filtered.
I believe that they do not belong in our SOAP/ebXML message expressions.. because they cannot and will not be
interpreted by an XPath processor and processed.  They cannot be because they are not XML entities and cannot be 
parsed by an XML parser.  They are at another higher level in the message structure, and I believe should be treated 
"outside" of the message expression ( as attributes in the <GetMessage> and <PutMessage> elements.  
I believe that <GetMessage Content-ID="abc"> or <PutMessage Content-ID="abc">is where these MIME attributes should 
be manipulated or evaluated and described.


  For some other "Condition" operations,
  (like the ones on "optional" material, or the ones used for final verification) 
we could dissociate them from GetMessage 
  and make them really separate ops (or steps)... 
  (Should we "flatten" our steps, and have roughly each one of your table row correspond 
to a step - after Get and Put
  have their related expressions - message builders and filters - back in their 
"message expr" fields?) 
   


[MIKE] -  My concern with flattening has to do with associating <Condition>s with a particular <GetMessage>...
The way things are currently defined we have:

<TestCase>
   <Step>
       <PutMessage/>
   </Step>
   <Step>
      <GetMessage>
          <Condition1/>
          <Condition2/>
      </GetMessage>
</TestCase>

You are suggesting:

<TestCase>
   <Step>
      <PutMessage/>
   </Step>
   <Step>GetMessage></Step>
   <Step>
      <Condition1/>
   </Step>
   <Step>
      <Condition2/>
   </Step>
</TestCase>


By flattening. it appears that you lose the containership "parent/child" relationship 
between a <GetMessage> a particular message,
and all of the <Condition> filters associated with it.  Everything's a step, with no 
association between the
steps.  That is why XML containership is important, in my opinion. Even though we are 
describing this in an "abstract" sense, there
are some real-world consequences if we do not adequately describe the relationships 
between our test components.
When we code the test driver, this will be important.

<Jacques2> so we may not have to "flatten" anymore, if we
clearly identify "filter" conditions to associate with GetMessage as I just proposed. 
Besides this, I do not have problem with including other conditions inside GetMessage step,
but we must then make sure we can use references to material from other previous 
steps from these (default message material would be the one selected by current GetMessage).
</Jacques2> 

[MIKE2] - I believe that we can handle "across message" filtering by introducing new operations.
We already have <GetMessage> and <PutMessage>.. why not add <CompareMessage>?  It can use the
same syntax/format we use for <GetMessage>. ( it will have to "merge" the 2 messages into one
document object, then XPath expressions could evaluate the merged object contents ) 

Tables like the one we are using for describing the Abstract Tests are not well suited 
to represent containership. I 
represent that containership through the use of the Step # column, with the only steps 
defined as either a <SetMessage>
or <GetMessage>.

Using object oriented principles, these <Steps> could return any number of exceptions, 
depending on which object generates
a which type of a fatal exception.  Looking at this now, I would probably go a step 
further and using object oriented design,
change the Condition class to be <TestCondition> or <Condition>.. one throws a 
"fatalTest" exception, the other a "fatalPrecondition" 
exception... rather than having to create an attribute called "errorStatus" that 
the test writer must assign a value.  So we would
have as potential classes:
<SetMesssage>
<GetMessage>
<TestCondition>
<Condition>
<SetPayload>
<GetPayload>

My feeling is, even though we are describing "abstract" tests, we should consider the 
implementation consequences of how we
are describing these tests.

I will respond to your other comments separately, within the associated mail message.

Thanks,
Mike