[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [ebxml-iic] Section 7.1 of Test Framework revision.. using Jacques workflow approach (option B)
I made changes based on your suggestions, also added a few comments/questions ( see attached .. my comments/changes in italics ) that we need to resolve.. The 3 big items are:
1) Making "mask" the default behavior for GetMessage ( I believe this is not necessary, difficult to implement, and should not be the default behavior for a Test Driver because it assumes too much about the intentions of the test writer ..i.e. that they "don't need" any previous messages for their testing)
2) stepDuration (a Test Driver parameter) is set at configuration time, and is used to "timeout" a GetMessage (or PutMessage) operation after some reasonable amount of time. It is not meant to be a "timing" mechanism for capturing all received messages withing a designated time period, although it could be used that way. Its purpose is to give the Test Driver a reasonable amount of time to query the Message Store and retrieve messages based upon the XPath Filter. The use of the <Sleep> operator is better suited for synchronizing waits between request and response checking. Actually, a time period like "30seconds" would be reasonable for StepDuration ( for our ebMS Test Suite ). Our Test Driver "polls" the MessageStore, and immediately exits from GetMessage() upon satisfying its XPath Filter, so if it gets its message in 5 seconds, GetMessage() is complete, and the Test Driver executes the next operation. Obviously though, if one does not use <Sleep> between a <PutMessage> and a <GetMessage>, then one needs to "tune" the StepDuration parameter to allow enough time for the Test Driver to receive a response message.
3) Do we need to define a "stepDurationFail" Test Driver parameter to be used by subsequent TestAssertion operations? We could. I would argue however that any real "time" testing should involve comparison of message Timestamps, not Test Driver clock time ( since network latency, application processing time and test driver efficiency are quantities that should not play a role in determining the pass/fail of a business process implementation). Secondly, if we are doing to do time measurement using the Test Driver clock alone, then we should "highly recommend" the use of <Sleep> to synchronized verification of reception of messages, since it does not add unknowns such as effiency of MessageStore storage and XPath processer performance ( which could ultimately impact the test case result). StepDuration should not be a toole to accurately measure whether a message was received "in time".
StepDuration is a "tuning" parameter for a Test Driver to reasonably set a "timeout" value for GetMessage and PutMessage operations. Nothing more.
I agree with the rest of your changes. If we can resolve these last 3 issues, I believe I can finalize this section.. and move it to section 4.