OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: RE: [ebxml-iic] more on message correlation


Title: some late feedback...(on MS conf)
Mike:
 
not much to add, as it seems we both agree.
Let me summarize just to make sure:
 
(a)- we'll be more specific, in the test suite tables of the conformance suite, about when / how frequently special "parameters" such as
COnversationID, MessageId, are expected to be generated during execution of the test suite.
 
(b)- To make more explicit the import of such parameter values in new messages material, we make a systematic use
of such expressions as: eb:MessageData/RefToMessageId=$MessageId ,  where the second operand is a parameter.
 
(c)- We remove RefToMessageId from the parameter lists, as it is not really automatically generated by Test Driver
(and Test Framework spec does n't say anything about how, which would be a tricky default behavior).
 
(d)- Unless explicitly specified in the "abstact message content" column, an optional message header element
will not be included by default. This is the case for RefToMessageId.
 
(e)- There are two cases where RefToMessageId needs be set by the test framework:
(1) either by the Test Service when responding to a message generated by Test Driver.
This is automatically taken care of by the service actions (no need be specified in test case description).
(2) or by the Test Driver, when responding to a message received from Test Service, within same conversation.
(this case is not frequent in MS conformance suite.) In that case, we realise the best way to handle this
is to have some means to "pluck" values from previous received message, hold them in some temporary parameters
(or variables), and reuse it in expressions like in (b), e.g. here:
eb:MessageData/RefToMessageId=$myParameter
 
--------
 
So I'd propose we go with (e) in the few test cases where the Test Driver needs to answer to a received message.
(e.g. Test Cases 90, 107, 114, 141, 142, 147 ... seem to apply)
(BTW, it seems test case(s) 90 is duplicated? or misnumbered?)
That means its OK in your abstract test suite to introduce working variables such as "myID" in
Step 1 (receiving): we add "...and $myID=eb:MessageData/MessageId and...)
Step 2 (sending response): we add : "... eb:MessageData/RefToMessageId=$myID..."
 
Then, by the time we are done with ebMS conformance test suite, I propose we vote at same time the minor
upgrade in Test Fraemwork spec that would require support for such var mechanism in Test Driver...
 
Regards,
 
jacques

[Jacques Durand]  -----Original Message-----
From: Michael Kass [mailto:michael.kass@nist.gov]
Sent: Wednesday, July 23, 2003 8:53 PM
To: Jacques Durand; ebxml-iic@lists.oasis-open.org
Subject: Re: [ebxml-iic] more on message correlation

Jacques,
 
 
   Below are answers to your comments.  I'm also attaching a revised ebXML Conformance Test Suite Specification, based upon your comments, as well as comments from
Pete, Monica and Mike Dillon.
 
   NOTE- Based on our conference call discussion, and comments from you and others, I have removed the section entitled "Other Conformance Testing Activities".  I also agree
that referring to other programs ( and companies ) that may or may not be part of a conformance activity in the future is not something that belongs in a conformance testing
specification.   Perhaps in a non-normative appendix....
 
Comments welcome,
 
Mike
 
----- Original Message -----
Sent: Wednesday, July 23, 2003 8:42 PM
Subject: RE: [ebxml-iic] more on message correlation

Mike:
-----Original Message-----
From: Michael Kass [mailto:michael.kass@nist.gov]
Sent: Monday, July 21, 2003 5:55 PM
To: Jacques Durand; ebxml-iic@lists.oasis-open.org
Subject: Re: [ebxml-iic] more on message correlation

 
----- Original Message -----
Sent: Monday, July 21, 2003 7:15 PM
Subject: [ebxml-iic] more on message correlation

Mike and all:
 
a follow-up on my issue #5 below, that I 'd like to reword:
I still believe we have an issue with message correlation:
 
- unlike in interop testing where it is behaving like an application, the Test Driver crafts all elements of the message
when used in a conformance test harness (including elements that are usually determined by the MSH,
and not decided by the application, like Messagge IDs.)
So now the MessageID has to be put in by the Test Driver:
(a)- either explictly (in the message content field of the test case,like done for CPAid)
(b)- or generated automatically, as suggested in parameters of table in 3.5.2.
 
[MIKE] - That is how the Test Framework describes CPAId, MessageId, ConversationId and RefToMessageId...
either explicitly declared or automatically generated by the Test Driver
 
If it were (a), the Message ID would be defined in the test case data, like CPAid value.
But it is more like (b) in our suite.
 
[MIKE]  - Yes.  I explicitly defined CPAId in the Abstract Test Suite.. although I could have left it out,
and whatever CPAId is in the ConfigurationGroup content would be used by default. I felt that having CPAId defined
explicitly added to the understanding of the test.
[Jacques Durand] it has to be the way you did anyway: CPAid cannot be generated  by Test Driver...
( its part of test configuration group doc)
 
[MIKE2] - You're right.. my mistake. It's not dynamically generated by the Test Driver.. it is "inserted" by the Test Driver after
reading the <ConfigurationGroup> content.
 
So, questions/comments arise:
 
1. As mentioned in TestFramework, (4.2.1.2) COnversationID and MessageID are automatically generated by
Test Driver. It would be good to remind the way this is done (somewhere after table in MS conf doc 3.5.2):
[MIKE] - I describe that in section 3.5.5.. maybe I should move it back to section 3.5.2 ( it is a more logical place for it ).
[Jacques Durand] I think we could just be more explicit in the table 3.5.2: e.g. Add the "life span" of the parameter setting:
- COnversationId: generated for each test case.
- MessageId: generated for each sending step.
- RefToMessageId:  generated for each sending step (optionally)
 
[MIKE2] - I would suggest that RefToMessageId be generated for the <GetMessage> (i.e. "receiving") operation only.. because 
a Test Driver "autogenerated" RefToMessageId would be totally meaningless for sending, unless that was the intention of the
Test Writer.  Currently, there are no Test Cases where RefToMessageId is randomly generated by the Test Driver.
 In the case where the test writer does want to set the RefToMessageId in a SENT message, we will have to extend the scripting language
to support insertion of $RefToMessageId values into the <MessageDeclaration> content.  We cannot assume that $RefToMessageId value will
be equal to the MessageId of the last received message, because messages will be received asynchronously, and may not be in order.
 Currently, $RefToMessageId values can only be dereferenced in <GetMessage><Filter> XPath statements, and are set to the value of the
MessageId of the last message SENT by the Test Driver.  I discuss this further (the 4th comment) below.
 
NOTE: I see something confusing in 3.5.1 and 3.5.2 tables: Table 3.5.1 claims to contain "Test Driver Configuration Parameters",
which is fine, but we see then column 3 in 3.5.2 that is named "set in Test Driver Configuration ", for parameters that are
entirely different. I would rename this column "set in Test COnfigurationGroup", which is more consistent with
Test Framework section 3.3.3 (which says "Test COnfiguration"), and also with params listed in Test Framework section 4.2.1 .
for ConfigurationGroup.
 
[MIKE2] - Done
 
Also, it would be good to say if the config group parameters we refer to, belong to a Config Group for the entire test suite
 like SenderParty, ReceiverParty, Service, or if we use some lower level Config Group (test case level) whihc I think we don't,
 
[MIKE2] - I added this in the "Lifecycle" column in tables #3 and #4 ( sections 3.5.1 and 3.5.2 ) .. which you suggested above 
 
as anything test case specific is override in the test case script itself (message content field), and that is always the
case in fact for parameters such as CPAid, Action (we have to specify their default value) .
So in table 3.5.2, under column " set in Test COnfigurationGroup", we could say:
- CPAid: default value generated for entire test suite
- SenderParty: default value generated for entire test suite
etc.
 
[MIKE2] - I tried to describe this in the LIfeCycle column..
 
 
 
This semantics seems to be described in TestFramework spec  (7.1.4), which states that the conv ID changes for each test case,
and the MessageId for each step.
So this parameter ($conversationId) is generated by the Test Driver for each TCase, but we still need to "assign" it to
the header of the message to be sent, it seems:  (eb:ConversationId=$ConversationId ). 
 
[MIKE] - If you're talking about the executable test suite.. the Test Driver automatically assigns the ConversationId when it constructs the
message from the <MessageDeclaration>....</MessageDeclaration> .  There is no need to "assign" it in the MessageDeclaration.
If you are talking about the Abstract Test Cases.. we can do whatever we want if it makes the tests clearer to someone who is going
to write it.  As you see in the Abstract Test Suite.. I already have eb:ConversationId=$ConversationId. 
[Jacques Durand] That is fine. 
 
 
I don't have a problem with that, but then why don't we do the same for MessageID? (e.g. eb:MessageData/MessageId=$MessageId),
which we know will change for each sending step).
 
[MIKE] - I just left MessageId out because I felt that it added more complexity than someone who was examining the test needed. 
It would be easy to add them into the Abstract Test Suite.  But they are not in the Executable Test Suite.. because they are not needed.
[Jacques Durand] it is just that it is hard to explain why we mention ConversationId and not MessageId in the scripts - whether abstract or concrete. Keeping the MessageId "assignment" or condition, in the abstract suite clarifies the way we do message correlation, if we "surface" this
correlation in the script, see below.
 
[MIKE2] - Agreed.  We should be clear/non-ambiguous.  I will add MessageId=$MessageId to the <MessageDeclaration>
 
 
2. The RefToMessageId element typically should be absent, for the first message of a conversation (which is,
for each request messge of each test case.)
".... If there is no earlier related message, the element MUST NOT be present."
Although we rightly ignore RefToMessageId in message content for sent messages, it is not clear that
the way we specify this in table in 3.5.2, the Test Driver will make sure to NOT insert a RefToMessageId element.
 
[MIKE] - It would be good to add a caveat in table 4 ( section 3.5.2 ) stating that RefToMessage, while automatically generated
by the test driver, is not used for message declaration... HOWEVER, it is used for message filtering ( my next comment below ).
If I look at the table in 3.5.2, I have the feeling that RefToMessageId is treated the same as MessageID: the
Test Driver will generate one, and even give it a value... I would remove RefToMessageId from this table
(this element should always be set by the app, or Test Service, in responses. And if it needs be set by the Test Driver, that
should be after the MessageId of a previous response. So cannot be automatically generated).
[Jacques Durand] I actually withdraw my comment:  the Test Driver must be able to "respond" to the
test service, and therefore to inssert a RefToMessageId.
 
[MIKE2] - Yes, but RefToMessageId is an "optional" MessageHeader value... not required.  We cannot have the Test Driver
"assume" that it is responding to the last received Message.. That would be a dangerous assumption.. especially if messages are not
received in the order expected ( network latency... MessageOrdering...etc... )  I believe that RefToMessageId should not be
"autogenerated" by a Test Driver and placed in a <MessageDeclaration>.. only in a <GetMessage><Filter>....</Filter></GetMessage>
operation.
 
But how can we specify that sometimes an element should NOT be added in the header, and sometimes should be?
I feel we can't rely on the Test Driver to hardcode this knowledge, nor on the default configuration.
It has to be specified somehow at test step level, in the script, I think.
 
[MIKE2] - The way to do this would be to modify/relax the ebXML TestSuite schema to accept explicit $RefToMessageId  or $ConversationId
or $CPAId or $MyParameter as an argument for element or attribute values.  A Test Drive would then reade the <MessageDeclaration> and
parse $MyParameter and insert its parameter value into the Message.  Currently, the Test Framework does not allow this, but we could modify
it to do so.   The disadvantage is that we will lose strong data typing ( i.e. dateTime, duration, URI ) and instead just accept a $MyParameter string.
 
 
[MIKE] - Why not just state ( in a caveat in table 4 )  that RefToMessageId is used by the Test Driver for message filtering, not message generation?
If I am "filtering" returned messages, and I want only those messages that refer to my previous sent message, then I MUST know the
"RefToMessageId" of the message I receive.  I know what it is.. because it is the MessageId of the message I sent with the last Test Step.
RefToMessageId is critical for <GetMessage>  .   RefToMessageId is irrelevant for <PutMessage>.
[Jacques Durand] right for  <GetMessage>, but RefToMessageId is not irrelevant to sent messages, as we both thought,
per my comment above... (I know most test cases don't need the test driver to "respond", but some will).
 
[MIKE2] - I agree that it is not irrelevant, and I think that we have to introduce "paramaterization" of sent message content in order to solve
this problem.  I believe that there are a few Test Cases in this Test Suite where this would help.
 
 
Furthermore, could we use something like: eb:MessageData/RefToMessageId=nil
To make it clear the element should be absent?
 
[MIKE] - I would just leave that declaration out entirely.. since RefToMessageId is an optional element in a message to begin with.
[Jacques Durand] but this optionality is critical for sent messages: we need to instruct teh Test Driver
when to insert this element and when not to...  
 
[MIKE2] - By default, as defined in [ebTestFramework#7.1.4.4] The Test Driver will not generate/insert this element.  So we only need to tell it when we WANT it to,
therefore this declaration is not needed.
 
 
3. Now, we still need to correlate a response with the message sent.
We will first assume that the Test Service (behaving like an app) will be in charge of setting
the RefToMessageId. So the test case script / Test Driver does not have to do this (at least for regular test cases.)
But in the test cases, the correlation statement is confusing. We correlate first on ConvID, which is OK. But in addition, we do:
eb:MessageData/RefToMessageId=$RefToMessageId]]
 
[MIKE] - As I stated above, $RefToMessageId  is equal to $MessageId from the previous <PutMessage>.  That is how we
can correlate and filter messages and look at what we want to look at.  So the Test Driver "knows" what the RefToMessageId will be, and
if the Test Service is functioning like it should.. it should send back the correct RefToMessageId
[Jacques Durand] So we have to be more explicit about this"default" peculiar behavior of the Test Driver: as it was not
specified in Test Framework... (that is why I think it would avoid confusion if we explicitly use the "overriding" mechanism
of the script material instead...).
 
[MIKE2] - I'll be happy to add the $MessageId to the <MessageDeclaration> , however eb:MessageData/RefToMessageId=nil
I believe is confusing and incorrect, since the RefToMessageId element does not exist in the message sent by the Test Driver, and that syntax does not conform to the
XPath syntax that defines exactly what a sent Message MUST contain.  I would prefer not to alter XPath meaning.   Comments?
 
(1) Either we have to define explicitly this"default" behavior such as " when sending a message,
RefToMessageId  is generated by Test Driver as follows: if the message is first of its conversation, no RefToMessageId  
inserted, if the message is a response to another message received in same conversation, then it is set to the
MessageId of the previous received message."
(2) Or we use explicit correlation expression in tet cases, like we have now (except that I would use eb:MessageData/RefToMessageId=$MessageId ,
as the set-up of RefToMessageId parameter is again not clearly specified in Test Framework.)
Given that (1) is actually an update to the Test Framework spec, I'd go with (2), which requres minor updates in our
conformance spec...
 
 
 
Which is not correct, as again RefToMessageId should be set by the receiving party (unless the
Test Driver sets it up to previous sent MessageId..
 
[MIKE] - That is what the Test Driver does
 
.but that built-in behavior is unflexible, and error prone)
 
 
Normally, the RefToMessageId of the received message should be same as the MessageId of a previously sent message.
Assuming we always need to compare only with the last sent message:
 
[MIKE] - And normally, 99 out of 100 times.. that is what we do with this Test Suite
 
How do we remember a parameter (MessageId ) generated by the Test Driver in a previous step?
 
[MIKE] - We have a <SetParameter> function that can pull a value out of a message using an XPath expression, and
assign it to some arbitrary Parameter name.. whatever we want to call it.
 
We could assume that $MessageId will only change for each "sending" step, so we can still use it
to compare with, in the following "receive" step.
So one way to fix this, is to use:
eb:MessageData/RefToMessageId=$MessageId
 
[MIKE] - Yes we can do that, and eliminate the RefToMessageId element.. I just had it in the Test Suite so that it was a little
clearer what we were doing.
[Jacques Durand] I'd go with that, so we don't need to make any assumption on the ability of teh Test Driver to "generate" RefToMessageId parameter, from data of a received message .
 
 
[MIKE2] - This is true.  However, you are right that we need to be able to explicitly add a $Paramater into a  <MessageDeclaration> when we need to create a message
based on some value that we've "plucked" from a previous message.  Right now, $Parameters can only be used in <GetMessage><Filter>{XPath Expression}</Filter></GetMessage> operations, not in <PutMessage><MessageDeclaration></MessageDeclaration></PutMessage> operations.  This will make the scripting much more powerful, and would only
require a minimal modification of the Test Framework document.
 
Now, for sent messages, when applicable, we still need to find a way to tell:
"insert a RefToMessageId  element, and use for it the value $MessageId of the previous receiving step"
This is the hard part: how to specify this? we can't use $MessageID to point to the previous received value,
as $MessageId is automatically updated to a new generated ID for this current sending step...
Possible solution: we could introduce parameters that are variables, in test steps:
e.g.
Step 1 (receiving): we add "...and $myID=eb:MessageData/MessageId and...)
Step 2 (sending response): we add : "... eb:MessageData/RefToMessageId=$myID..."
And if we dont specify any expression on ,MessageData/RefToMessageId  then that means the element will not be inserted at all.
We might need an upgrade of the Test Framework spec to support this, I believe,
but that seems pretty flexible (more than a hardcoded set up of RefToMessageId by Test Driver  I think)
Opinion?
We may also plan for a minor upgrade of TestFramework at the time we release conformance suite.
 
[MIKE2] - Agreed 
 
In which case again, the semantics of when/how a parameter like $MessageId is changed by TestDriver, should be reminded in the spec.
In a next verison of TesFramework, we should be able to refer to parameters set in arbitrary steps, by refering to step#,
e.g. like ($1)MessageId  for $MessageId  set in Step #1.
 
[MIKE] - That is something to consider for the next version.
 
 
Regards,
 
Jacques
 
-----Original Message-----
From: Jacques Durand [mailto:JDurand@fsw.fujitsu.com]
Sent: Monday, July 21, 2003 11:00 AM
To: 'ebxml-iic@lists.oasis-open.org'
Subject: [ebxml-iic] some late feedback...(on MS conf)

Mike and all:

Quick feedback on MS conf spec (will send more later):

1. need more names in "reviewers" section

2. Profiles section need more exposure (a full section, not header 3), and we'll need a
conforming profile document pointing to the right reqs. We may need to re-discuss a bit the
profile defs (ping/pong/status svc place, syncReply options)

3. Test cases for multi-hop still use the "concrete" syntax (GetMessage...)

4. The wording of some test reqs is not concrete enough, in terms of testable assertion
(see reqs 6, 7: "the MSH accepts the message", we need to say something more concrete,
like: passes to application without generating an error.)

5. In Test Cases: would correlation based on Conv ID be sufficient, for associating req-resp?
(instead of MesgID / RefToMesgID)
(not sure we can always assume the Test Driver can know, as an app, the MesgID that are generated by its MSH)

Regards,

Jacques





[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]