Re: [tag] Thread: TA modeling - RE: [tag] TA Model still weak ontests of structure?

[stephen.green@systml.co.uk]

An example of how I would see this working:

The example is based on a simplification of the fairly complex
TA structure used in the WS-I TAs here:
http://www.ws-i.org/Testing/Tools/2005/01/BP11_TAD_1-1.htm#BP1309

The WS-I structure can be represented as follows:


Test Assertion:
BP1309

Entry Type:
anyEnvelope

Test Type:
required

Enabled:
true

Additional Entry Types:
-Message Input:
[Not specified]
-WSDL Input:
[Not specified]

Prerequisites:
BP1701

Profile Requirements:
-Target:
R1011
-Partial-Target:
[Not specified]
-Collateral:
[Not specified]

Context:
For a candidate envelope containing a soap:Body element

Assertion Description:
The soap:Envelope does not have direct children after the soap:Body element

Failure Message:
The soap:Envelope has a direct child after the soap:Body element.

Failure Detail Description:
[Not specified]

Comments:
[Not specified]



Simplification along the lines Jacques suggested with my on it mods becomes:-

-------------------------------------

Test Assertion:
BP1309

Prerequisites:
BP1701

Trigger:
[Not specified]

Test Expression:
For a candidate envelope containing a soap:Body element the  
soap:Envelope does not have direct children after the soap:Body element

Outcome:
The Test Expression MUST be true

--------------------------------------

Here the whole TA is expressed formally under one heading which I've
called 'Test Expression'

The MUST aspect (the fact that the TA is required and the failure message
is sent when the Test Expression is false) 1. is notably part of the
WS-I structure (though partly implicit in the fact that the message they
mention is for a failure - from which one can logically deduce what
constitutes failure in relation to the test assertion) 2. is separated
from the 'Test Expression' but uses it as a reference point to describe
what constitutes success or failure 3. is summed up all in one place in
the proposed TA structure under the heading 'Outcome', 4. comprises both
the concept of failure (implicit still) and the property applied to the
TA as a whole of 'required' in this example.

I tend to accept that there is reason enough in this example to keep the
prerequisite of a previous test having completed as a separate heading.
This is because that previous test has itself a prerequisite and so on
and listing them all for this particular TA back to the starting point
would be unrealistic in this example. In don't call it a precondition
here though because that term is overloaded and because the concept has
been put forward in our group that there can be shared prerequisites which
can be placed either in individual TAs (originally this would have been
as preconditions but I would now tend away from that term) or at the head
of a group of TAs (put forward in the Korean SC's F2F slides).

So now I think my proposition in the call of a structure below is bourne out
by this and other such examples which can readily be expressed this way with
little or no loss of information compared with more complex structuring.

TA: (ID, prose, references)

...

---------------------------

Prerequisites: (reference, say, to another the ID of a TA to have run before)

Trigger: (optional - could this be a choice with 'Entry Type'?)

Test Expression: (the main expression - a predicate)

Outcome: (relates to predicate boolean to MUST, SHOULD, etc - typically
           "The Test Expression MUST be true" - could be coded / enumerated)

---------------------------


-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting "Durand, Jacques R." <JDurand@us.fujitsu.com>:

> A couple of comments on the notion of "Structural vs. behavioral" tests:
>
> 1. If we can put a general definition on these, and if it happens that
> each one of these two categories presents different challenges for TA
> writing, it may be worth some mention in our TA guide.  It is an
> intuitive classification even for those not UML-literate.
>
> - challenge: not always clear which category a TA should fall into:
> consider a requirement R1 on a message header structure (either a field
> must be conform to a format, or the header itself be schema-valid).
> Another spec requirement R2 says this message header is expected to be
> produced in response to some operation on the message handler.
> Two options here:
> (a) TA for R1: the IUT is the header document --> a structural test.
> TA for R2: the IUT is the message handler --> a behavior test, but that
> uses TA-for-R1 as pre-requisite (must be satisfied for a "pass" outcome)
> (b) TA for R1+R2: the IUT is the message handler --> a behavior test
> (header structure test is just part of describing the expected behavior:
> to send out a well-formed message header)
>
>
> 2. So far we have a candidate TA anatomy (for non-prose Tas) like:
>
> - Pre-condition
> - Test trigger
> - Test effect
> - Post-condition (maybe)
>
> It seems to me that the notion of test trigger vs test effect is more
> appropriate for "behavioral"  tests where both "trigger"  and "effect"
> refer to specific events that are clearly serialized over time (in form
> of action on test harness, or of behavior of the IUT).
>
> But for "structural"  tests like validating a doc, I feel the
> distinction is somewhat contrived. There might be a single event (e.g.
> do the doc validation against rule/schema/...) and no "effect" expected
> in response, besides the result of this validation.
>
> How about using instead (for both "structural"  and "behavioral" Tas):
>
> - [Pre-condition]
> - Test Operation: e.g. " validate the doc against rule/schema/..."
> (covers the entire test operation and possible steps/events if any)
> - Test Outcome: "fail" if..., "pass" if... (only focus on what are the
> conditions for "pass", "fail", refering to observed events/output of
> Test Operation)
> - [Post-condition]
>
> Other benefit:
> - even in case of a "prose" TA where the prose is just a pointer to the
> spec requirement, we could require the "Test Outcome" to be there (or at
> least a generic one shared by many Tas of the same kind - see option (b)
> in my email 9/21: "the test outcome is PASS if the required behavior is
> observed, FAIL otherwise.")
>
> -Jacques
>
>
>
>
> -----Original Message-----
> From: Dave Pawson [mailto:dave.pawson@gmail.com]
> Sent: Monday, September 24, 2007 11:55 PM
> To: tag@lists.oasis-open.org
> Subject: Re: [tag] TA Model still weak on tests of structure?
>
> On 24/09/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk>
> wrote:
>
>> what if something about the item's structure is being  tested? For
>> example, what if the IUT is a serial number or a  universally unique
>> identifier or the like and the test is that  it's format conforms to a
>> particular pattern?
>
> Data values internal to a test?
>
>> I found that there is a need to equally support both structures and
>> behaviours. In UML structures come under 'Foundations' and dynamic
>> aspects under 'Behaviours'. I propose we add something for a
>> structural aspect to be tested. It could be anything from schema to
>> class, property to datatype.
>
>
> I.e. it is too general to be clearly defined?
>
> Out of scope IMHO.
>
> regards
>
>
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
>