RE: [tag] revised "Anatomy of a TA"

["Durand, Jacques R." <JDurand@us.fujitsu.com>]

 Dave:

Thanks for quick feedback on what is still a rough draft.
See inline <JD>, on first part of your comments.

I think we need to nail down a solid proposal for the general structure
of a TA. Agree the one in the draft is not specific/helpful enough,
misses some parts.

Cheers,
Jacques 

-----Original Message-----
From: Dave Pawson [mailto:dave.pawson@gmail.com] 
Sent: Wednesday, August 29, 2007 12:08 AM
To: tag@lists.oasis-open.org
Subject: Re: [tag] revised "Anatomy of a TA"

On 28/08/2007, Durand, Jacques R. <JDurand@us.fujitsu.com> wrote:

General comments.

The acronyms and abbreviations need removing or clarifying.

antecedent and consequence seem to be borrowed from a long way away?
Pre-test and post-test conditions is more usual?

<JD> possibly. We shied away from "condition" that seems to focus more
on describing a state and seems inappropriate for events or behaviors.
I like pre-test conditions - more intuitive. But post-test seems to
suggest that the "test"  is over when these effects occur. True for a
state change which persists, but not for produced behavior/event, the
observation of which is still part of the test.
So maybe we have to be more precise and get to something closer to what
is proposed in 
http://www.oasis-open.org/committees/document.php?document_id=20661&wg_a
bbrev=ebxml-iic
Test assertion =
(a) ref to spec requirement.
(b) object under test
(c) pre-condition (of object under test, possibly includes test env.)
(d) test trigger (an operation or event)
(e) test effect (an operation or event)
(f) post-condition (of object under test, possibly includes test env.)
(g) final outcome.
Semantics: when (c)+(d) , then if (e)+(f) the test outcome = (g)
(="pass" by default)
Note: not sure if we need be more specific about the actual test:
It could consist of bringing the system under state described by (c),
then causing the trigger to occur (d), then observing (e) and (f). In
the case of schema validation, (d)="validate doc against schema xyz",
(e)="the validation is successful, warnings are OK". Note: could it be
in some cases  that trigger and effect are events that do not need to be
caused for the sake of testing, but naturally occur in the system under
test. 
</JD>


TA must identify....
The actual test and expected test result are omitted.

Need to differentiate between the test outcome and the altered state of
the test object (post-conditions)

<JD> good point. We alluded to the "nature of the test"  which should be
made clear, but no room for this in the TA anatomy.</JD>


----------------------- not yet commented:


TA-101 is time bound. ('related to a previously sent PO).
That's a pre-condition. Unless the association is part of this test,
there should be no mention of it.
The post-condition mentioned, 'validates' is a test result, not a
post-condition. I.e. it seems to be the outcome of the test?
A post-condition normally relates to the abnormal state in which a test
leaves the test object, e.g. The car window is left open after test,
which is not the neutral state.

No mention of version of the schema?
The pre-condition shouldn't contain questions. They should be resolved
prior to specification of the test.

Re 101b

Very unclear.
Rather than having an object to test, it is the sender of the message
that is being tested?  No test is stated? Again confusion between the
post-condition and the test result.

re 102.
  Makes for a very unclear example. The item under test is the purchase
order receipt. The test is that it has been received within 24 hours
from the transmission of the order.
Again no test stated. The time of transmission of the order is a
pre-condition.





--
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk