----- Original Message -----
Sent: Monday, July 21, 2003 7:15
PM
Subject: [ebxml-iic] more on
message correlation
Mike and all:
a follow-up on my issue #5 below, that I 'd like to
reword:
I still believe we have an issue with message
correlation:
- unlike in interop testing where it is behaving like an
application, the Test Driver crafts all elements of the
message
when used in a conformance test harness (including elements that
are usually determined by the MSH,
and not decided by the application, like Messagge
IDs.)
So now the MessageID has to be put in by the Test
Driver:
(a)- either explictly (in the message content field of the
test case,like done for CPAid)
(b)- or generated automatically, as suggested in parameters of
table in 3.5.2.
[MIKE] - That is how the Test Framework describes CPAId,
MessageId, ConversationId and RefToMessageId...
either explicitly declared or automatically generated by the Test
Driver
If it were (a), the Message ID would be defined in the test case
data, like CPAid value.
But it is more like (b) in our suite.
[MIKE] - Yes. I explicitly defined CPAId in the
Abstract Test Suite.. although I could have left it
out,
and whatever CPAId is in the ConfigurationGroup content would be
used by default. I felt that having CPAId defined
explicitly added to the understanding of the
test.
[Jacques
Durand] it has to be the way you did anyway: CPAid cannot be
generated by Test Driver...
( its part of test configuration group
doc)
[MIKE2] - You're right.. my
mistake. It's not dynamically generated by the Test Driver.. it is
"inserted" by the Test Driver after
reading the
<ConfigurationGroup> content.
So, questions/comments arise:
1. As mentioned in TestFramework, (4.2.1.2) COnversationID and
MessageID are automatically generated by
Test Driver. It would be good to remind the way this is done
(somewhere after table in MS conf doc 3.5.2):
[MIKE] - I describe that in section 3.5.5..
maybe I should move it back to section 3.5.2 ( it is a more logical
place for it ).
[Jacques Durand] I think we could just be more
explicit in the table 3.5.2: e.g. Add the "life span" of the parameter
setting:
- COnversationId:
generated for each test case.
- MessageId:
generated for each sending
step.
-
RefToMessageId: generated for each sending step
(optionally)
[MIKE2] - I would suggest that
RefToMessageId be generated for the <GetMessage> (i.e.
"receiving") operation only.. because
a Test Driver "autogenerated"
RefToMessageId would be totally meaningless for sending, unless that was
the intention of the
Test Writer. Currently,
there are no Test Cases where RefToMessageId is randomly generated
by the Test Driver.
In the case where the test
writer does want to set the RefToMessageId in a SENT message, we will
have to extend the scripting language
to support insertion of
$RefToMessageId values into the <MessageDeclaration>
content. We cannot assume that $RefToMessageId value
will
be equal to the MessageId of the
last received message, because messages will be received
asynchronously, and may not be in order.
Currently, $RefToMessageId values
can only be dereferenced in
<GetMessage><Filter> XPath statements, and are set to the
value of the
MessageId of the last message SENT
by the Test Driver. I discuss this further (the 4th comment)
below.
NOTE: I see
something confusing in 3.5.1 and 3.5.2 tables: Table 3.5.1 claims to
contain "Test Driver Configuration
Parameters",
which is fine, but
we see then column 3 in 3.5.2 that is named "set in Test Driver
Configuration ", for parameters that
are
entirely different.
I would rename this column "set in Test COnfigurationGroup", which is
more consistent with
Test Framework
section 3.3.3 (which says "Test COnfiguration"), and also with params
listed in Test
Framework section
4.2.1 .
for
ConfigurationGroup.
[MIKE2] -
Done
Also, it would be
good to say if the config group parameters we refer to, belong
to a Config Group for the entire test
suite
like
SenderParty, ReceiverParty, Service, or if we
use some lower level Config Group (test case level) whihc I think we
don't,
[MIKE2] - I
added this in the "Lifecycle" column in tables #3 and #4 ( sections
3.5.1 and 3.5.2 ) .. which you suggested
above
as anything test
case specific is override in the
test case script itself (message content
field), and that is always the
case in fact for
parameters such as CPAid, Action (we have to specify their default
value) .
So in table 3.5.2,
under column " set in Test COnfigurationGroup", we could
say:
- CPAid: default
value generated for entire test suite
- SenderParty:
default value generated for entire test suite
etc.
[MIKE2] - I
tried to describe this in the LIfeCycle column..
This semantics seems to be described in TestFramework spec
(7.1.4), which states that the conv ID changes for each test
case,
and the MessageId for each step.
So this parameter ($conversationId) is generated by the Test
Driver for each TCase, but we still need to "assign" it
to
the header of the message to be sent, it
seems: (eb:ConversationId=$ConversationId
).
[MIKE] - If you're talking about the executable test suite.. the
Test Driver automatically assigns the ConversationId when it constructs
the
message from the
<MessageDeclaration>....</MessageDeclaration>
. There is no need to "assign" it in the
MessageDeclaration.
If you are talking about the Abstract Test Cases.. we can do
whatever we want if it makes the tests clearer to someone who is
going
to write it. As you see in the Abstract
Test Suite.. I already have eb:ConversationId=$ConversationId.
[Jacques
Durand] That is fine.
I don't have a problem with that, but then why don't we do the
same for MessageID? (e.g.
eb:MessageData/MessageId=$MessageId),
which we know will change for each sending
step).
[MIKE] - I just left MessageId out because I felt that it added
more complexity than someone who was examining the test needed.
It would be easy to add them into the
Abstract Test Suite. But they are not in the Executable Test
Suite.. because they are not needed.
[Jacques Durand] it is
just that it is hard to explain why we mention ConversationId and not
MessageId in the scripts - whether abstract or concrete. Keeping
the MessageId "assignment" or condition, in the abstract suite clarifies
the way we do message correlation, if we "surface" this
correlation in
the script, see below.
[MIKE2] - Agreed. We should
be clear/non-ambiguous. I will add MessageId=$MessageId to the
<MessageDeclaration>
2. The RefToMessageId element typically should be absent, for the
first message of a conversation (which is,
for each request messge of each test case.)
".... If there is no earlier related message, the element MUST
NOT be present."
Although
we rightly ignore RefToMessageId in message content for sent messages,
it is not clear that
the way we specify this in table in 3.5.2, the Test
Driver will make sure to NOT insert a RefToMessageId
element.
[MIKE] - It would be good to add a caveat in table 4 ( section
3.5.2 ) stating that RefToMessage, while automatically
generated
by the test driver, is not used for message
declaration... HOWEVER, it is used for message filtering ( my next
comment below ).
If I look at the table in 3.5.2, I have the feeling that
RefToMessageId is treated the same as MessageID: the
Test Driver will generate one, and even give it a value... I
would remove RefToMessageId from this table
(this element should always be set by the app, or Test Service,
in responses. And if it needs be set by the Test Driver,
that
should be after the MessageId of a previous
response. So cannot be automatically generated).
[Jacques Durand] I actually withdraw my
comment: the Test Driver must be able to "respond" to
the
test service, and therefore
to inssert a
RefToMessageId.
[MIKE2] - Yes, but RefToMessageId is an
"optional" MessageHeader value... not required. We cannot have the
Test Driver
"assume" that it is responding to the last
received Message.. That would be a dangerous assumption.. especially if
messages are not
received in the order expected ( network
latency... MessageOrdering...etc... ) I believe that RefToMessageId should not be
"autogenerated" by a Test Driver and placed in
a <MessageDeclaration>.. only in a
<GetMessage><Filter>....</Filter></GetMessage>
operation.
But how can we
specify that sometimes an element should NOT be added in the header, and
sometimes should be?
I feel we can't rely
on the Test Driver to hardcode this knowledge, nor on the default
configuration.
It has to be
specified somehow at test step level, in the script, I
think.
[MIKE2] - The way to do this would be to
modify/relax the ebXML TestSuite schema to accept explicit
$RefToMessageId or $ConversationId
or $CPAId or $MyParameter as an argument for
element or attribute values. A Test Drive would then reade the
<MessageDeclaration> and
parse $MyParameter and insert its parameter
value into the Message. Currently, the Test Framework does
not allow this, but we could modify
it to do so. The disadvantage is
that we will lose strong data typing ( i.e. dateTime, duration, URI
) and instead just accept a $MyParameter
string.
[MIKE] - Why not just state ( in a caveat in table 4 )
that RefToMessageId is used by the Test Driver
for message filtering, not message generation?
If I am "filtering" returned messages, and I want only those
messages that refer to my previous sent message, then I MUST know
the
"RefToMessageId" of the message I receive. I know what it
is.. because it is the MessageId of the message I sent with the last
Test Step.
RefToMessageId is critical for
<GetMessage> . RefToMessageId is irrelevant
for <PutMessage>.
[Jacques Durand] right
for <GetMessage>, but RefToMessageId is not irrelevant
to sent messages, as we both
thought,
per my comment above... (I know most test cases don't need
the test driver to "respond", but some
will).
[MIKE2] - I agree that it is not irrelevant, and I think that we
have to introduce "paramaterization" of sent message content in order to
solve
this problem. I believe that there are a few Test Cases in
this Test Suite where this would help.
Furthermore, could we use something like: eb:MessageData/RefToMessageId=nil
To make it clear
the element should be absent?
[MIKE] - I would
just leave that declaration out entirely.. since RefToMessageId is an
optional element in a message to begin with.
[Jacques Durand] but
this optionality is critical for sent messages: we need to instruct teh
Test Driver
when to insert this element
and when not to...
[MIKE2] - By default, as defined in
[ebTestFramework#7.1.4.4] The Test Driver will not generate/insert
this element. So we only need to tell it when we WANT it
to,
therefore this declaration is not
needed.
3. Now, we still need to
correlate a response with the message sent.
We
will first assume that the Test Service (behaving like an app) will be
in charge of setting
the
RefToMessageId. So the test case script / Test Driver does not have to
do this (at least for regular test cases.)
But
in the test cases, the correlation statement is
confusing. We correlate first on ConvID, which
is OK. But in addition, we do:
eb:MessageData/RefToMessageId=$RefToMessageId]]
[MIKE] - As I stated above, $RefToMessageId is equal to
$MessageId from the previous <PutMessage>. That is how we
can correlate and filter messages and look at what we want to
look at. So the Test Driver "knows" what the RefToMessageId will
be, and
if the Test Service is functioning like it
should.. it should send back the correct RefToMessageId
[Jacques Durand] So we
have to be more explicit about this"default" peculiar behavior of the
Test Driver: as it was
not
specified in Test
Framework... (that is why I think it would avoid confusion if we
explicitly use the "overriding"
mechanism
of the script
material instead...).
[MIKE2] - I'll be happy to add the $MessageId to the
<MessageDeclaration> , however eb:MessageData/RefToMessageId=nil
I believe is
confusing and incorrect, since the RefToMessageId element does not exist
in the message sent by the Test Driver, and that syntax does not conform
to the
XPath syntax that
defines exactly what a sent Message MUST contain. I would prefer
not to alter XPath meaning.
Comments?
(1) Either we have
to define explicitly this"default" behavior such as " when sending
a message,
RefToMessageId is generated by Test Driver as
follows: if the message is first of its conversation, no RefToMessageId
inserted, if the message is a
response to another message received in same conversation, then it is
set to
the
MessageId of the previous received
message."
(2) Or we use explicit
correlation expression in tet cases, like we have now (except that I
would use eb:MessageData/RefToMessageId=$MessageId
,
as the set-up of RefToMessageId
parameter is again not clearly specified in Test
Framework.)
Given that (1) is actually an
update to the Test Framework spec, I'd go with (2), which requres minor
updates in
our
conformance
spec...
Which is not correct, as again RefToMessageId should be set by the
receiving party (unless the
Test Driver sets it up to previous sent
MessageId..
[MIKE] - That is what the Test Driver does
.but that built-in behavior is unflexible, and error
prone)
Normally, the RefToMessageId of the received message should be
same as the MessageId of a previously sent message.
Assuming we always need to compare only with the last sent
message:
[MIKE] - And normally, 99 out of 100 times.. that is what we do
with this Test Suite
How do we remember a parameter (MessageId ) generated by the Test
Driver in a previous step?
[MIKE] - We have a <SetParameter> function that can pull a
value out of a message using an XPath expression, and
assign it to some arbitrary Parameter name.. whatever we want to
call it.
We could assume that $MessageId will only change for each
"sending" step, so we can still use it
to compare with, in the following "receive"
step.
So one way to fix this, is to use:
eb:MessageData/RefToMessageId=$MessageId
[MIKE] - Yes we can do that, and eliminate the RefToMessageId
element.. I just had it in the Test Suite so that it was a
little
clearer what we were
doing.
[Jacques
Durand] I'd go with that, so we don't need to make any assumption
on the ability of teh Test Driver to "generate" RefToMessageId
parameter, from data of a received message .
[MIKE2] - This is true. However, you are
right that we need to be able to explicitly add a $Paramater into
a <MessageDeclaration> when we need to create a
message
based on some value that we've "plucked" from a
previous message. Right now, $Parameters can only be used in
<GetMessage><Filter>{XPath
Expression}</Filter></GetMessage> operations, not in
<PutMessage><MessageDeclaration></MessageDeclaration></PutMessage>
operations. This will make the scripting much more powerful, and
would only
require a minimal modification of the Test
Framework document.
Now, for sent messages,
when applicable, we still need to find a way to
tell:
"insert a
RefToMessageId element, and use for it the value $MessageId of the
previous receiving step"
This is the hard part: how
to specify this? we can't use $MessageID to point to the previous
received value,
as $MessageId is automatically updated to a new
generated ID for this current sending
step...
Possible solution: we could introduce
parameters that are variables, in test
steps:
e.g.
Step 1 (receiving): we add "...and $myID=eb:MessageData/MessageId and...)
Step 2 (sending
response): we add : "... eb:MessageData/RefToMessageId=$myID..."
And if we dont
specify any expression on ,MessageData/RefToMessageId then that means
the element will not be inserted at
all.
We might need an upgrade of the Test Framework
spec to support this, I believe,
but that seems pretty flexible (more than a
hardcoded set up of RefToMessageId by Test Driver I
think)
Opinion?
We may also plan for a minor upgrade of
TestFramework at the time we release conformance
suite.
[MIKE2] - Agreed
In which case again, the semantics of when/how a parameter like
$MessageId is changed by TestDriver, should be reminded in the
spec.
In a next verison of TesFramework, we should be able to refer to
parameters set in arbitrary steps, by refering to
step#,
e.g. like ($1)MessageId for $MessageId set in Step
#1.
[MIKE] - That is something to consider for the next
version.
Regards,
Jacques
Mike and all:
Quick feedback on MS conf spec (will send
more later):
1. need more names in "reviewers"
section
2. Profiles section need more exposure (a
full section, not header 3), and we'll need a
conforming profile document pointing to the right
reqs. We may need to re-discuss a bit the
profile defs (ping/pong/status svc place, syncReply
options)
3. Test cases for multi-hop still use the
"concrete" syntax (GetMessage...)
4. The wording of some test reqs is not
concrete enough, in terms of testable assertion
(see reqs 6, 7: "the MSH accepts the message", we
need to say something more concrete,
like: passes to application without generating an
error.)
5. In Test Cases: would correlation based
on Conv ID be sufficient, for associating req-resp?
(instead of MesgID / RefToMesgID)
(not sure we can always assume the Test Driver can
know, as an app, the MesgID that are generated by its MSH)
Regards,
Jacques