Re: [ebxml-iic] Section 7.1 of Test Framework revision.. using Jacques workflow approach (option B)

["Michael Kass" <michael.kass@nist.gov>]

Title: call tomorrow Wed 8th, 3pm PT

Jacques,

 

   I made changes based on your suggestions, also added a few comments/questions ( see attached .. my comments/changes in italics )  that we need to resolve.. The 3 big items are:

 

    1) Making "mask" the default behavior for GetMessage ( I believe this is not necessary, difficult to implement, and should not be the default behavior for a Test Driver because it assumes too much about the intentions of the test writer ..i.e. that they "don't need" any previous messages for their testing)

    2) stepDuration (a Test Driver parameter) is set at configuration time, and is used to "timeout" a GetMessage (or PutMessage) operation after some reasonable amount of time.  It is not meant to be a "timing" mechanism for capturing all received messages withing a designated time period, although it could be used that way. Its purpose is to give the Test Driver a reasonable amount of time to query the Message Store and retrieve messages based upon the XPath Filter.  The use of the <Sleep> operator is better suited for synchronizing waits between request and response checking.  Actually, a time period like "30seconds" would be reasonable for StepDuration ( for our ebMS Test Suite ).  Our Test Driver "polls" the MessageStore, and immediately exits from GetMessage() upon satisfying its XPath Filter, so if it gets its message in 5 seconds, GetMessage() is complete, and the Test Driver executes the next operation.   Obviously though, if one does not use <Sleep> between a <PutMessage> and a <GetMessage>, then one needs to "tune" the StepDuration parameter to allow enough time for the Test Driver to receive a response message. 

    3) Do we need to define a "stepDurationFail" Test Driver parameter to be used by subsequent TestAssertion operations?  We could.  I would argue however that any real "time" testing should involve comparison of  message Timestamps, not Test Driver clock time ( since network latency, application processing time and test driver efficiency are quantities that should not play a role in determining the pass/fail of a business process implementation).  Secondly, if we are doing to do time measurement using the Test Driver clock alone, then we should "highly recommend" the use of <Sleep> to synchronized verification of reception of messages, since it does not add unknowns such as effiency of MessageStore storage and XPath processer performance ( which could ultimately impact the test case result).  StepDuration should not be a toole to accurately measure whether a message was received "in time". 

StepDuration is a "tuning" parameter for a Test Driver to reasonably set a "timeout" value for GetMessage and PutMessage operations.  Nothing more.

  

 

    I agree with the rest of your changes.  If we can resolve these last 3 issues, I believe I can finalize this section.. and move it to section 4.

 

Cheers,

Mike

 

----- Original Message -----

From: Jacques Durand

To: 'Michael Kass' ; Jacques Durand ; ebXML IIC - main list (E-mail) (E-mail)

Sent: Friday, September 10, 2004 7:33 PM

Subject: RE: [ebxml-iic] Section 7.1 of Test Framework revision.. using Jacques workflow approach (option B)

Mke:

 

agree that a Section #4 in Part 1 should be the place to put the sample scripts (4.1).

I'd put "Executing Test Cases" even after the scripts (in 4.2).

 

More comments on the use cases attached (tagged <JD5>)

 

Jacques

 

 

-----Original Message-----
From: Michael Kass [mailto:michael.kass@nist.gov]
Sent: Friday, September 10, 2004 9:50 AM
To: Jacques Durand; ebXML IIC - main list (E-mail) (E-mail)
Subject: Re: [ebxml-iic] Section 7.1 of Test Framework revision.. using Jacques workflow approach (option B)

Jacques,

 

  Please see my comments below regarding placement of section 7.1 into Part 1 ( I have no problems with it.. but believe it would be better suited in

section 4 instread of trying to put it inside section 3 ( Test Framework Components.. where we describe Test Driver and Test Service).  I think it would

be better to have a "section 4 - Test Case execution model".

 

  Also, please see 2 attached files ( your modifications to section 7.1 and my response (as MK4) , and the resultant "diffs" file. ( I forgot to turn on tracking... so I highlighted the

diffs in red).   I agree/chaged most everything based upon your suggestion.

 

Cheers,

Mike

 

-----

section_7.1_diffs-JD_MK.doc