----- Original Message -----
Sent: Monday, July 21, 2003 7:15
PM
Subject: [ebxml-iic] more on message
correlation
Mike and all:
a
follow-up on my issue #5 below, that I 'd like to
reword:
I
still believe we have an issue with message correlation:
-
unlike in interop testing where it is behaving like an application, the Test
Driver crafts all elements of the message
when used in a conformance test harness (including elements that are
usually determined by the MSH,
and not decided by the application, like Messagge
IDs.)
So
now the MessageID has to be put in by the Test Driver:
(a)- either explictly (in the message content field of the test
case,like done for CPAid)
(b)- or generated automatically, as suggested in parameters of table
in 3.5.2.
[MIKE] - That is how the Test Framework describes CPAId, MessageId,
ConversationId and RefToMessageId...
either explicitly declared or automatically generated by the Test
Driver
If
it were (a), the Message ID would be defined in the test case data, like
CPAid value.
But it is more like (b) in our suite.
[MIKE] - Yes. I explicitly defined CPAId in the Abstract
Test Suite.. although I could have left it out,
and whatever CPAId is in the ConfigurationGroup content would be used
by default. I felt that having CPAId defined
explicitly added to the understanding of the
test.
[Jacques
Durand] it has to be the way you did anyway: CPAid cannot be
generated by Test Driver...
( its part of test configuration group
doc)
So, questions/comments arise:
1.
As mentioned in TestFramework, (4.2.1.2) COnversationID and MessageID are
automatically generated by
Test Driver. It would be good to remind the way this is done
(somewhere after table in MS conf doc 3.5.2):
[MIKE] - I describe that in section 3.5.5.. maybe
I should move it back to section 3.5.2 ( it is a more logical place for it
).
[Jacques
Durand] I think we could just be more explicit in the table 3.5.2: e.g.
Add the "life span" of the parameter
setting:
- COnversationId:
generated for each test case.
- MessageId: generated
for each sending step.
-
RefToMessageId: generated for each sending step
(optionally)
NOTE: I see something
confusing in 3.5.1 and 3.5.2 tables: Table 3.5.1 claims to contain "Test
Driver Configuration Parameters",
which is fine, but we
see then column 3 in 3.5.2 that is named "set in Test Driver Configuration
", for parameters that are
entirely different. I
would rename this column "set in Test COnfigurationGroup", which is more
consistent with
Test Framework section
3.3.3 (which says "Test COnfiguration"), and also with params listed in
Test Framework section
4.2.1 .
for
ConfigurationGroup.
Also, it would be good
to say if the config group parameters we refer to, belong to a
Config Group for the entire test
suite
like SenderParty,
ReceiverParty, Service, or if we use some lower level
Config Group (test case level) whihc I think we
don't,
as anything test case
specific is override in the test case script itself (message
content field), and that is always
the
case in fact for
parameters such as CPAid, Action (we have to specify their default
value) .
So in table 3.5.2, under
column " set in Test COnfigurationGroup", we could
say:
- CPAid: default value
generated for entire test suite
- SenderParty: default
value generated for entire test suite
etc.
This semantics seems to be described in TestFramework spec
(7.1.4), which states that the conv ID changes for each test
case,
and the MessageId for each step.
So
this parameter ($conversationId) is generated by the Test Driver for each
TCase, but we still need to "assign" it to
the header of the message to be sent, it
seems: (eb:ConversationId=$ConversationId ).
[MIKE] - If you're talking about the executable test suite.. the Test
Driver automatically assigns the ConversationId when it constructs
the
message from the
<MessageDeclaration>....</MessageDeclaration> . There
is no need to "assign" it in the MessageDeclaration.
If
you are talking about the Abstract Test Cases.. we can do whatever we want
if it makes the tests clearer to someone who is going
to write it. As you see in the Abstract Test
Suite.. I already have eb:ConversationId=$ConversationId.
[Jacques Durand] That is
fine.
I
don't have a problem with that, but then why don't we do the same for
MessageID? (e.g. eb:MessageData/MessageId=$MessageId),
which we know will change for each sending step).
[MIKE] - I just left MessageId out because I felt that it added more
complexity than someone who was examining the test needed.
It would be easy to add them into the Abstract
Test Suite. But they are not in the Executable Test Suite.. because
they are not needed.
[Jacques Durand] it is just that it is hard to explain
why we mention ConversationId and not MessageId in the scripts - whether
abstract or concrete. Keeping the MessageId "assignment" or condition,
in the abstract suite clarifies the way we do message correlation, if we
"surface" this correlation in the
script, see below.
2.
The RefToMessageId element typically should be absent, for the first message
of a conversation (which is,
for each request messge of each test case.)
".... If there is no earlier related message, the element MUST NOT
be present."
Although
we rightly ignore RefToMessageId in message content for sent messages, it is
not clear that
the way we specify this in table in 3.5.2, the Test Driver will
make sure to NOT insert a RefToMessageId element.
[MIKE] - It would be good to add a caveat in table 4 ( section 3.5.2
) stating that RefToMessage, while automatically
generated
by the test driver, is not used for message
declaration... HOWEVER, it is used for message filtering ( my next comment
below ).
If
I look at the table in 3.5.2, I have the feeling that RefToMessageId is
treated the same as MessageID: the
Test Driver will generate one, and even give it a value... I would
remove RefToMessageId from this table
(this element should always be set by the app, or Test Service, in
responses. And if it needs be set by the Test Driver,
that
should be after the MessageId of a previous
response. So cannot be automatically generated).
[Jacques Durand] I actually withdraw my comment:
the Test Driver must be able to "respond" to
the
test service, and therefore to
inssert a
RefToMessageId.
But how can we specify that
sometimes an element should NOT be added in the header, and sometimes should
be?
I feel we can't rely on
the Test Driver to hardcode this knowledge, nor on the default
configuration.
It has to be specified
somehow at test step level, in the script, I
think.
[MIKE] - Why not just state ( in a caveat in table 4 )
that RefToMessageId is used by the Test Driver for message
filtering, not message generation?
If
I am "filtering" returned messages, and I want only those messages that
refer to my previous sent message, then I MUST know the
"RefToMessageId" of the message I receive. I know what it is..
because it is the MessageId of the message I sent with the last Test
Step.
RefToMessageId is critical for
<GetMessage> . RefToMessageId is irrelevant for
<PutMessage>.
[Jacques Durand] right for <GetMessage>,
but RefToMessageId is not irrelevant to sent messages, as we both
thought,
per my comment above... (I know most test cases don't need the
test driver to "respond", but some
will).
Furthermore, could we use something like: eb:MessageData/RefToMessageId=nil
To make it clear the
element should be absent?
[MIKE] - I would just
leave that declaration out entirely.. since RefToMessageId is an optional
element in a message to begin with.
[Jacques Durand] but this optionality is critical for
sent messages: we need to instruct teh Test
Driver
when to insert this element and
when not to...
3. Now, we still need to correlate
a response with the message sent.
We will
first assume that the Test Service (behaving like an app) will be in charge
of setting
the
RefToMessageId. So the test case script / Test Driver does not have to do
this (at least for regular test cases.)
But in
the test cases, the correlation statement is confusing. We
correlate first on ConvID, which is OK. But in addition, we
do:
eb:MessageData/RefToMessageId=$RefToMessageId]]
[MIKE] - As I stated above, $RefToMessageId is equal to
$MessageId from the previous <PutMessage>. That is how we
can correlate and filter messages and look at what we want to look
at. So the Test Driver "knows" what the RefToMessageId will be,
and
if the Test Service is functioning like it
should.. it should send back the correct RefToMessageId
[Jacques Durand] So we
have to be more explicit about this"default" peculiar behavior of the Test
Driver: as it was not
specified in Test
Framework... (that is why I think it would avoid confusion if we explicitly
use the "overriding" mechanism
of the script material
instead...).
(1) Either we have to
define explicitly this"default" behavior such as " when sending a
message,
RefToMessageId is generated by Test Driver as
follows: if the message is first of its conversation, no RefToMessageId
inserted, if the message is a response to another
message received in same conversation, then it is set to
the
MessageId of the previous received
message."
(2) Or we use explicit correlation expression in tet cases,
like we have now (except that I would use eb:MessageData/RefToMessageId=$MessageId
,
as the
set-up of RefToMessageId parameter is again not clearly specified in Test
Framework.)
Given
that (1) is actually an update to the Test Framework spec, I'd go with (2),
which requres minor updates in
our
conformance
spec...
Which is not correct, as again RefToMessageId should be set by the
receiving party (unless the
Test Driver sets it up to previous sent
MessageId..
[MIKE] - That is what the Test Driver does
.but that built-in behavior is unflexible, and error
prone)
Normally, the RefToMessageId of the received message should be same
as the MessageId of a previously sent message.
Assuming we always need to compare only with the last sent
message:
[MIKE] - And normally, 99 out of 100 times.. that is what we do with
this Test Suite
How do we remember a parameter (MessageId ) generated by the Test
Driver in a previous step?
[MIKE] - We have a <SetParameter> function that can pull a
value out of a message using an XPath expression, and
assign it to some arbitrary Parameter name.. whatever we want to call
it.
We
could assume that $MessageId will only change for each "sending" step, so we
can still use it
to
compare with, in the following "receive" step.
So
one way to fix this, is to use:
eb:MessageData/RefToMessageId=$MessageId
[MIKE] - Yes we can do that, and eliminate the RefToMessageId
element.. I just had it in the Test Suite so that it was a
little
clearer what we were doing.
[Jacques Durand] I'd go
with that, so we don't need to make any assumption on the ability of teh
Test Driver to "generate" RefToMessageId parameter, from data of a received
message .
Now, for sent messages, when applicable, we still need to find
a way to tell:
"insert a RefToMessageId element, and use for it the
value $MessageId of the previous receiving
step"
This is the hard part: how to specify this? we can't use
$MessageID to point to the previous received
value,
as $MessageId is automatically updated to a new
generated ID for this current sending
step...
Possible solution: we could introduce parameters
that are variables, in test steps:
e.g.
Step 1 (receiving): we add "...and $myID=eb:MessageData/MessageId and...)
Step 2 (sending
response): we add : "... eb:MessageData/RefToMessageId=$myID..."
And if we dont specify
any expression on ,MessageData/RefToMessageId
then that means the element will not be inserted at
all.
We might need an upgrade of the Test Framework spec
to support this, I believe,
but that seems pretty flexible (more than a
hardcoded set up of RefToMessageId by Test Driver I
think)
Opinion?
We may also plan for a minor upgrade of
TestFramework at the time we release conformance
suite.
.
In
which case again, the semantics of when/how a parameter like $MessageId is
changed by TestDriver, should be reminded in the spec.
In
a next verison of TesFramework, we should be able to refer to parameters set
in arbitrary steps, by refering to step#,
e.g. like ($1)MessageId for $MessageId set in Step
#1.
[MIKE] - That is something to consider for the next
version.
Regards,
Jacques
Mike and all:
Quick feedback on MS conf spec (will send more
later):
1. need more names in "reviewers"
section
2. Profiles section need more exposure (a full
section, not header 3), and we'll need a
conforming profile document pointing to the right reqs. We may need
to re-discuss a bit the
profile defs
(ping/pong/status svc place, syncReply options)
3. Test cases for multi-hop still use the
"concrete" syntax (GetMessage...)
4. The wording of some test reqs is not
concrete enough, in terms of testable assertion
(see reqs 6, 7: "the MSH accepts the message", we need
to say something more concrete,
like:
passes to application without generating an error.)
5. In Test Cases: would correlation based on
Conv ID be sufficient, for associating req-resp?
(instead of MesgID / RefToMesgID)
(not sure we can always assume the Test Driver can know,
as an app, the MesgID that are generated by its MSH)
Regards,
Jacques