----- Original Message -----
Sent: Monday, July 21, 2003 7:15
PM
Subject: [ebxml-iic] more on message
correlation
Mike and all:
a follow-up on my issue #5 below, that I 'd like to
reword:
I still believe we have an issue with message
correlation:
- unlike in interop testing where it is behaving like an
application, the Test Driver crafts all elements of the
message
when used in a conformance test harness (including elements that
are usually determined by the MSH,
and not decided by the application, like Messagge
IDs.)
So now the MessageID has to be put in by the Test
Driver:
(a)- either explictly (in the message content field of the
test case,like done for CPAid)
(b)- or generated automatically, as suggested in parameters of
table in 3.5.2.
[MIKE] - That is how the Test Framework describes CPAId, MessageId,
ConversationId and RefToMessageId...
either explicitly declared or automatically generated by the Test
Driver
If it were (a), the Message ID would be defined in the test case
data, like CPAid value.
But it is more like (b) in our suite.
[MIKE] - Yes. I explicitly defined CPAId in the
Abstract Test Suite.. although I could have left it
out,
and whatever CPAId is in the ConfigurationGroup content would be
used by default. I felt that having CPAId defined
explicitly added to the understanding of the
test.
[Jacques
Durand] it has to be the way you did anyway: CPAid cannot be
generated by Test Driver...
( its part of test configuration group
doc)
[MIKE2] - You're right.. my mistake.
It's not dynamically generated by the Test Driver.. it is "inserted" by
the Test Driver after
reading the
<ConfigurationGroup> content.
So, questions/comments arise:
1. As mentioned in TestFramework, (4.2.1.2) COnversationID and
MessageID are automatically generated by
Test Driver. It would be good to remind the way this is done
(somewhere after table in MS conf doc 3.5.2):
[MIKE] - I describe that in section 3.5.5..
maybe I should move it back to section 3.5.2 ( it is a more logical place
for it ).
[Jacques
Durand] I think we could just be more explicit in the table 3.5.2:
e.g. Add the "life span" of the parameter
setting:
- COnversationId:
generated for each test case.
- MessageId: generated
for each sending step.
-
RefToMessageId: generated for each sending step
(optionally)
[MIKE2] - I would suggest that
RefToMessageId be generated for the <GetMessage> (i.e. "receiving")
operation only.. because
a Test Driver "autogenerated"
RefToMessageId would be totally meaningless for sending, unless that was
the intention of the
Test Writer. Currently,
there are no Test Cases where RefToMessageId is randomly generated by
the Test Driver.
In the case where the test
writer does want to set the RefToMessageId in a SENT message, we will have
to extend the scripting language
to support insertion of
$RefToMessageId values into the <MessageDeclaration> content.
We cannot assume that $RefToMessageId value
will
be equal to the MessageId of the
last received message, because messages will be received
asynchronously, and may not be in order.
Currently, $RefToMessageId values can
only be dereferenced in <GetMessage><Filter>
XPath statements, and are set to the value of
the
MessageId of the last message SENT
by the Test Driver. I discuss this further (the 4th comment)
below.
NOTE: I see something
confusing in 3.5.1 and 3.5.2 tables: Table 3.5.1 claims to contain "Test
Driver Configuration Parameters",
which is fine, but we
see then column 3 in 3.5.2 that is named "set in Test Driver Configuration
", for parameters that are
entirely different. I
would rename this column "set in Test COnfigurationGroup", which is more
consistent with
Test Framework section
3.3.3 (which says "Test COnfiguration"), and also with params listed in
Test Framework section
4.2.1 .
for
ConfigurationGroup.
[MIKE2] -
Done
Also, it would be good
to say if the config group parameters we refer to, belong to a
Config Group for the entire test
suite
like
SenderParty, ReceiverParty, Service, or if we use
some lower level Config Group (test case level) whihc I think we
don't,
[MIKE2] - I
added this in the "Lifecycle" column in tables #3 and #4 ( sections
3.5.1 and 3.5.2 ) .. which you suggested
above
as anything test case
specific is override in the test case script
itself (message content field), and that is always
the
case in fact for
parameters such as CPAid, Action (we have to specify their default
value) .
So in table 3.5.2,
under column " set in Test COnfigurationGroup", we could
say:
- CPAid: default value
generated for entire test suite
- SenderParty: default
value generated for entire test suite
etc.
[MIKE2] - I
tried to describe this in the LIfeCycle column..
This semantics seems to be described in TestFramework spec
(7.1.4), which states that the conv ID changes for each test
case,
and the MessageId for each step.
So this parameter ($conversationId) is generated by the Test Driver
for each TCase, but we still need to "assign" it
to
the header of the message to be sent, it
seems: (eb:ConversationId=$ConversationId
).
[MIKE] - If you're talking about the executable test suite.. the
Test Driver automatically assigns the ConversationId when it constructs
the
message from the
<MessageDeclaration>....</MessageDeclaration>
. There is no need to "assign" it in the
MessageDeclaration.
If you are talking about the Abstract Test Cases.. we can do
whatever we want if it makes the tests clearer to someone who is
going
to write it. As you see in the Abstract Test
Suite.. I already have eb:ConversationId=$ConversationId.
[Jacques
Durand] That is fine.
I don't have a problem with that, but then why don't we do the same
for MessageID? (e.g.
eb:MessageData/MessageId=$MessageId),
which we know will change for each sending
step).
[MIKE] - I just left MessageId out because I felt that it added
more complexity than someone who was examining the test needed.
It would be easy to add them into the Abstract
Test Suite. But they are not in the Executable Test Suite.. because
they are not needed.
[Jacques Durand] it is just that it is hard to explain
why we mention ConversationId and not MessageId in the scripts - whether
abstract or concrete. Keeping the MessageId "assignment" or
condition, in the abstract suite clarifies the way we do message
correlation, if we "surface" this
correlation in
the script, see below.
[MIKE2] - Agreed. We should be
clear/non-ambiguous. I will add MessageId=$MessageId to the
<MessageDeclaration>
2. The RefToMessageId element typically should be absent, for the
first message of a conversation (which is,
for each request messge of each test case.)
".... If there is no earlier related message, the element MUST
NOT be present."
Although
we rightly ignore RefToMessageId in message content for sent messages, it
is not clear that
the way we specify this in table in 3.5.2, the Test
Driver will make sure to NOT insert a RefToMessageId
element.
[MIKE] - It would be good to add a caveat in table 4 ( section
3.5.2 ) stating that RefToMessage, while automatically
generated
by the test driver, is not used for message
declaration... HOWEVER, it is used for message filtering ( my next comment
below ).
If I look at the table in 3.5.2, I have the feeling that
RefToMessageId is treated the same as MessageID: the
Test Driver will generate one, and even give it a value... I would
remove RefToMessageId from this table
(this element should always be set by the app, or Test Service, in
responses. And if it needs be set by the Test Driver,
that
should be after the MessageId of a previous
response. So cannot be automatically generated).
[Jacques Durand] I actually withdraw my comment:
the Test Driver must be able to "respond" to
the
test service, and therefore
to inssert a
RefToMessageId.
[MIKE2] - Yes, but RefToMessageId is an
"optional" MessageHeader value... not required. We cannot have the
Test Driver
"assume" that it is responding to the last
received Message.. That would be a dangerous assumption.. especially if
messages are not
received in the order expected ( network
latency... MessageOrdering...etc... ) I believe that RefToMessageId should not be
"autogenerated" by a Test Driver and placed in a
<MessageDeclaration>.. only in a
<GetMessage><Filter>....</Filter></GetMessage>
operation.
But how can we specify that sometimes an element should NOT
be added in the header, and sometimes should be?
I feel we can't rely
on the Test Driver to hardcode this knowledge, nor on the default
configuration.
It has to be specified
somehow at test step level, in the script, I
think.
[MIKE2] - The way to do this would be to
modify/relax the ebXML TestSuite schema to accept explicit
$RefToMessageId or $ConversationId
or $CPAId or $MyParameter as an argument for
element or attribute values. A Test Drive would then reade the
<MessageDeclaration> and
parse $MyParameter and insert its parameter value
into the Message. Currently, the Test Framework does not allow
this, but we could modify
it to do so. The disadvantage is that
we will lose strong data typing ( i.e. dateTime, duration, URI ) and
instead just accept a $MyParameter string.
[MIKE] - Why not just state ( in a caveat in table 4 )
that RefToMessageId is used by the Test Driver for message
filtering, not message generation?
If I am "filtering" returned messages, and I want only those
messages that refer to my previous sent message, then I MUST know
the
"RefToMessageId" of the message I receive. I know what it
is.. because it is the MessageId of the message I sent with the last Test
Step.
RefToMessageId is critical for
<GetMessage> . RefToMessageId is irrelevant
for <PutMessage>.
[Jacques Durand] right
for <GetMessage>, but RefToMessageId is not irrelevant to
sent messages, as we both
thought,
per my comment above... (I know most test cases don't need
the test driver to "respond", but some
will).
[MIKE2] - I agree that it is not irrelevant, and I think that we
have to introduce "paramaterization" of sent message content in order to
solve
this problem. I believe that there are a few Test Cases in
this Test Suite where this would help.
Furthermore, could we use something like: eb:MessageData/RefToMessageId=nil
To make it clear the
element should be absent?
[MIKE] - I would just
leave that declaration out entirely.. since RefToMessageId is an optional
element in a message to begin with.
[Jacques Durand] but
this optionality is critical for sent messages: we need to instruct teh
Test Driver
when to insert this element
and when not to...
[MIKE2]
- By default, as defined in [ebTestFramework#7.1.4.4] The Test
Driver will not generate/insert this element. So we only need
to tell it when we WANT it to,
therefore this declaration is not
needed.
3. Now, we still need to
correlate a response with the message sent.
We will
first assume that the Test Service (behaving like an app) will be in
charge of setting
the
RefToMessageId. So the test case script / Test Driver does not have to do
this (at least for regular test cases.)
But in
the test cases, the correlation statement is confusing. We
correlate first on ConvID, which is OK. But in addition, we
do:
eb:MessageData/RefToMessageId=$RefToMessageId]]
[MIKE] - As I stated above, $RefToMessageId is equal to
$MessageId from the previous <PutMessage>. That is how we
can correlate and filter messages and look at what we want to look
at. So the Test Driver "knows" what the RefToMessageId will be,
and
if the Test Service is functioning like it
should.. it should send back the correct RefToMessageId
[Jacques Durand] So we
have to be more explicit about this"default" peculiar behavior of the Test
Driver: as it was not
specified in Test
Framework... (that is why I think it would avoid confusion if we
explicitly use the "overriding"
mechanism
of the script material
instead...).
[MIKE2] - I'll be happy to add the $MessageId to the
<MessageDeclaration> , however eb:MessageData/RefToMessageId=nil
I believe is
confusing and incorrect, since the RefToMessageId element does not exist
in the message sent by the Test Driver, and that syntax does not conform
to the
XPath syntax that
defines exactly what a sent Message MUST contain. I would prefer not
to alter XPath meaning.
Comments?
(1) Either we have to
define explicitly this"default" behavior such as " when sending a
message,
RefToMessageId is generated by Test Driver as
follows: if the message is first of its conversation, no RefToMessageId
inserted, if the message is a response to another
message received in same conversation, then it is set to
the
MessageId of the previous received
message."
(2) Or we use explicit correlation expression in tet cases,
like we have now (except that I would use eb:MessageData/RefToMessageId=$MessageId
,
as
the set-up of RefToMessageId parameter is again not clearly specified in
Test
Framework.)
Given
that (1) is actually an update to the Test Framework spec, I'd go with
(2), which requres minor updates in
our
conformance
spec...
Which is not correct, as again RefToMessageId should be set by the
receiving party (unless the
Test Driver sets it up to previous sent
MessageId..
[MIKE] - That is what the Test Driver does
.but that built-in behavior is unflexible, and error
prone)
Normally, the RefToMessageId of the received message should be same
as the MessageId of a previously sent message.
Assuming we always need to compare only with the last sent
message:
[MIKE] - And normally, 99 out of 100 times.. that is what we do
with this Test Suite
How do we remember a parameter (MessageId ) generated by the Test
Driver in a previous step?
[MIKE] - We have a <SetParameter> function that can pull a
value out of a message using an XPath expression, and
assign it to some arbitrary Parameter name.. whatever we want to
call it.
We could assume that $MessageId will only change for each "sending"
step, so we can still use it
to compare with, in the following "receive"
step.
So one way to fix this, is to use:
eb:MessageData/RefToMessageId=$MessageId
[MIKE] - Yes we can do that, and eliminate the RefToMessageId
element.. I just had it in the Test Suite so that it was a
little
clearer what we were doing.
[Jacques Durand] I'd go
with that, so we don't need to make any assumption on the ability of teh
Test Driver to "generate" RefToMessageId parameter, from data of a
received message .
[MIKE2] - This is true. However, you are
right that we need to be able to explicitly add a $Paramater into a
<MessageDeclaration> when we need to create a
message
based on some value that we've "plucked" from a
previous message. Right now, $Parameters can only be used in
<GetMessage><Filter>{XPath
Expression}</Filter></GetMessage> operations, not in
<PutMessage><MessageDeclaration></MessageDeclaration></PutMessage>
operations. This will make the scripting much more powerful, and
would only
require a minimal modification of the Test
Framework document.
Now, for sent messages, when applicable, we still need to
find a way to tell:
"insert a RefToMessageId element, and use for it the
value $MessageId of the previous receiving
step"
This is the hard part: how to specify this? we can't use
$MessageID to point to the previous received
value,
as $MessageId is automatically updated to a new
generated ID for this current sending
step...
Possible solution: we could introduce parameters
that are variables, in test steps:
e.g.
Step 1 (receiving): we add "...and $myID=eb:MessageData/MessageId and...)
Step 2 (sending
response): we add : "... eb:MessageData/RefToMessageId=$myID..."
And if we dont
specify any expression on ,MessageData/RefToMessageId
then that means the element will not be inserted at
all.
We might need an upgrade of the Test Framework
spec to support this, I believe,
but that seems pretty flexible (more than a
hardcoded set up of RefToMessageId by Test Driver I
think)
Opinion?
We may also plan for a minor upgrade of
TestFramework at the time we release conformance
suite.
[MIKE2] - Agreed
In which case again, the semantics of when/how a parameter like
$MessageId is changed by TestDriver, should be reminded in the
spec.
In a next verison of TesFramework, we should be able to refer to
parameters set in arbitrary steps, by refering to
step#,
e.g. like ($1)MessageId for $MessageId set in Step
#1.
[MIKE] - That is something to consider for the next
version.
Regards,
Jacques
Mike and all:
Quick feedback on MS conf spec (will send
more later):
1. need more names in "reviewers"
section
2. Profiles section need more exposure (a
full section, not header 3), and we'll need a
conforming profile document pointing to the right
reqs. We may need to re-discuss a bit the
profile defs (ping/pong/status svc place, syncReply
options)
3. Test cases for multi-hop still use the
"concrete" syntax (GetMessage...)
4. The wording of some test reqs is not
concrete enough, in terms of testable assertion
(see reqs 6, 7: "the MSH accepts the message", we need
to say something more concrete,
like:
passes to application without generating an error.)
5. In Test Cases: would correlation based on
Conv ID be sufficient, for associating req-resp?
(instead of MesgID / RefToMesgID)
(not sure we can always assume the Test Driver can
know, as an app, the MesgID that are generated by its MSH)
Regards,
Jacques