----- Original Message -----
Sent: Monday, October 04, 2004 10:31
AM
Subject: Re: [ebxml-iic] Test Framework
Question
Thanks Mike.
I was reading Section 5-7 (TR, TP, and TC) of
version 1 dated May 03. I think here is why I was confused.
Section 5 says that TR is test harness
independent. Section 6 says "A Test Case is the translation of a TR (or a part
of a TR), in an executable form, for a particular Test Harness".
[MIKE] - This is true. However, the word
"executable" has a slightly different meaning in Conformance Testing
semantics. "Executable" means just that, something that can be executed
by a Test Harness. It does not imply that it is "compiled", using
different languages. What form a Test Case is in (binary or XML.. or other) is
not relevant in conformance testing jargon.
Due to my problem context, I might have too
concrete interpretation to the "Test Harness". I interpreted it as
perhaps different languages, for example, I may be writing the TC in Java or
in C++. That's why I was asking how the TD knows which TC to use b/c it does
say which Test Harness it is associated with (maybe that's the
configurationGroup element).
[MIKE] - The "native" Test Case format in the IIC
Test Framework is XML. It is assumed that an underlying message
enveloping and transport mechanism is in place in the Test Harness to
construct, send, receive and store and evaluate messages by interpreting
the XML instructions using the appropriate underlying messaging API
calls.
It seems like TC in IIC is more abstract b/c it
is descriptively defined using XML and then Test Harness implemented in
whatever languages reads this XML to execute the test. In my problem context,
a TC is a rule written in either schematron or another rule script language. I
think I understand better now.
[MIKE] - Yes, your interpretation (above)
of a TC is different from the IIC Test Framework definition. It
sound like the schematron engine is the Test Harness, and the rule is the
TC.
I think I understand better now.
One more question though, why do you have Test
Suite reference TCs instead of TRs. In my mind, TS is a functional grouping of
tests, hence I should point to TRs.
[MIKE] - The Test Suite is a collection (or
grouping) of TCs. Each TC points to a particular TR through its
"requirementReferenceId" attribute, so I think that we agree
there.
On the other hand, I see TP as arbitrary grouping
of tests, so it could point to TCs. I think I am missing a lot things b/c
IIC's TRs and TCs are much more complicated.
[MIKE] - TPs are (in the Test Framework
view), an arbitrary grouping of TRs that satisfy a particular profile.
The Test Harness would read that TR list, then search through the TS,
and execute all TC's that satisfy that list of TRs, and therefore satisfy the
testing requirements of the TP.
Thank you,
Serm
----- Original Message -----
Sent: Monday, September 27, 2004 12:44
PM
Subject: Re: [ebxml-iic] Test Framework
Question
Serm,
What particular sectio of
the spec are you reading? The answer is: yes, one Test
Requirement may have multip;e Test Cases to verify that an implemenatation
satisfies that requirement. For instance,
a Test Requirement that states that all ebXML
extension elements MUST have a "soap:mustUnderstand" attribute
present. This may require multip;e Test Cases to verify that an
implementation does this.
(one Test Case for SyncReply, one Test Case for
Acknowledgment testing, one Test Case for MessageStatus
testing...etc...) So, each of those Test Cases would point to
that single Test Requirement, and ALL of them must
pass successfully for an implementation to say
that it satisfies that requirement.
A Test Driver identifies which TCs
satisfy a requirement by examining the TC's "requirementReferenceID".. which
is a pointer (via ID) to that particular TR. A Test Driver then
executes all TC's that "point" to a particular TR.
Regards,
Mike
----- Original Message -----
Sent: Monday, September 27, 2004
11:24 AM
Subject: [ebxml-iic] Test Framework
Question
Jacques and Mike et.al,
I looked over the TF doc and have a question.
In the test material, Test Profile is an aggregation of Test Requirements.
Test Driver takes a Test Profile and searches for associated Test
Cases and execute them.
However, in one section the it is describing
that a TR may have multiple TCs associated as one may have multiple ways
of validating the TR. The question is how the Test Driver can identify a
specific TC for a TR that has multiple TCs associated or I am missing
something?
Thank you,
Serm