[ebxml-iic-msg] Fwd: follow-up on discussions

[Matthew MacKenzie <matt@xmlglobal.com>]

Forwarding this so that we have an archive of it...

Begin forwarded message:

> From: Jacques Durand <JDurand@fsw.fujitsu.com>
> Date: Wed May 22, 2002  12:03:36  AM US/Pacific
> To: "'Michael Kass'" <michael.kass@nist.gov>, "'Matthew MacKenzie'" <matt@xmlglobal.com>
> Cc: Jacques Durand <JDurand@fsw.fujitsu.com>
> Subject: follow-up on discussions
>
> Guys:
>  
> Here is my recap on recent discussions. Please review.
> I tried to describe (informally) some test cases using these framework assumtpions.
> Seems to work.
>  
> regards,
>  
> Jacques
>
>  


----- Testing framework Assumptions, based on recent discussions -----------

1. We assume the test harness for MS conformance is made of :
(a) a Test Driver, which will process and drive all test cases.
(b) a Test Service, which will react to invocations from candidate MSH.

2. Part of a test case execution requires CPA-like configuration of the MSH.
Such configuration sets should be provided prior to running the tests, along
with CPAId values. The candidate MSH team will provide code to interpret
such configuration sets, and configure their MSH with it.
Each test case will reference/specify the CPA-like data that should be used.

3. The conformance Test Suite(s) can be run entirely by controlling the
MSH from the Test Driver side (no test driving needed on MSH application side).

4. on the "application" side of MSH, the Testing Service will have
a limited number of Actions. We identify four Actions that will/should be sufficient
to run MS conformance test cases: 
(a) a Responder action: on invocation, generate a response to a received message, by using same
message material, with a minimal header changes: swap to/from parties, set RefToMsgId,
(keep same conversation Id, same CPAid). The payload is same as in received message 
(same attachment(s)).
(b) an Initiator action: on invocation, generate a new message, by using 
message material (payload and header) that is contained in the payload of received message.
So the header of the new message can be anything that is specified. For example, used
to generate a "first" message in a conversation. Note: MSH-controlled header attributes
will not be determined by the invoking message (messageID, timestamps...)
(c) an Error action: used for MSH notification of (some) errors to the application.
These actions may generate a trace.
(d) a Mute action: on invocation, does not generate any message back. Only acknowledge
reception by generating (optionally) local trace.

5. The test verification (or validation) can be done at run-time, using 
test case input/output in-memory data (as opposed to analyzing a trace or log a-posteriori).

6. Also, the test verification can be done by using test case input/output observable
on Test Driver side only. (we should not need to execute test verifications inside
the test Service).

7. Correlation between message(s) sent and message(s) received is (by default) based on
MessageID / RefToMesgID. Might correlate on conv ID on demand (in case manipulation of
RefToMesgID required.)
 
-------- 10 test cases described using above framework (test steps description) ---------

r1.1.1: (SOAP 1.1 conformance)
step 1: test driver sends sample mesg to Responder action.
step 2: Responder action sends response mesg (with same attachment(s)).
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies test condition on response mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.1.3: (Message package content: "text/xml" type attribute in COntent-Type MIME)
step 1: test driver sends sample mesg to Responder action.
step 2: Responder action sends response mesg.
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies test condition on response mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.2.5: (Error in PartyId content)
step 1: test driver sends "bad" mesg to Mute action.
step 2: MSH sends back error mesg.
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies content error mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.2.6: (CPA resolution error)
step 1: test driver sends "bad" mesg (CPAid NOT within the initial config sets) 
to Mute action.
step 2: MSH sends back error mesg.
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies content error mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.2.9: (COnversationID consistency)
step 1: test driver sends sample mesg to Responder action.
step 2: Responder action sends response mesg (with same attachment(s), same conv ID).
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies test condition on response mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.2.11: (Unrecognized Service / Action)
step 1: test driver sends sample mesg to XYZ service or action.
step 2: MSH sends back error mesg.
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies test condition on response mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

----- NOTE: questionable test:
r1.2.14: (RefToMesgId should be set to previous)
step 1: test driver sends sample mesg to Initiator action, with new-mesg data
using same conv ID, but unrelated RefToMesgId.
step 2: Initiator action sends back new mesg (same conv ID, bad RefToMesgId).
step 3: test driver receives response mesg (that correlates with step 1 with convID)
step 4: test driver verifies test condition on response mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.2.15: (RefToMesgId should NOT be present for first message)
step 1: test driver sends "bad" mesg (RefToMesgId present, for a new convID) 
to Mute action.
step 2: MSH sends back error mesg.
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies content error mesg.
failure reported: if (step 3 not executed within timeout) or (step 4 condition fails)

r1.2.16: (RefToMesgId should be there in error mesg)
step 1: test driver sends "bad" mesg to Mute action.
step 2: MSH sends back error mesg.
step 3: test driver receives response mesg (that correlates with step 1)
step 4: test driver verifies content error mesg (just for confirming correlation)
failure reported: if (step 3 not executed within timeout, i.e. either not received, or
not correlating on RefToMesgId) or (step 4 condition fails)

r?.?.?: (Ack requested)
step 1: test driver sends sample mesg to Mute action, with Ack requested elt.
step 2: MSH sends back ack mesg.
step 3: test driver receives and correlates Ack mesg.
failure reported: if (step 3 not executed within timeout) 

r?.2.19: (Duplicate Elimination)
step 1: test driver sends sample mesg to Responder action.
step 2: MSH sends back ack mesg.
step 3: test driver receives and correlates Ack mesg.
step 4: Responder action sends response mesg (with same attachment(s), RefToMesgId set).
step 5: test driver receives response mesg (that correlates with step 1)
step 6: test driver sends exact same sample mesg to Responder action (same messageId).
step 7: MSH sends back ack mesg.
step 8: test driver receives and correlates Ack mesg.
step 9: test driver receives response mesg (that correlates with step 1)
failure reported: if (step 3 or 5 or 8 not executed within timeout) or (step 9 occurs)




>
--
Matthew MacKenzie
XML Global R&D
PGP Key available upon request.