Re: [tag] Thread: TA modeling - RE: [tag] TA Model still weak ontests of structure?

[stephen.green@systml.co.uk]

Just to correct some typos, sorry, and adding some editing for clarity:


> An example of how I would see this working:
>
> The example is based on a simplification of the fairly complex
> TA structure used in the WS-I TAs here:
> http://www.ws-i.org/Testing/Tools/2005/01/BP11_TAD_1-1.htm#BP1309
>
> The WS-I structure can be represented as follows:
>
>
> Test Assertion:
> BP1309
>
> Entry Type:
> anyEnvelope
>
> Test Type:
> required
>
> Enabled:
> true
>
> Additional Entry Types:
> -Message Input:
> [Not specified]
> -WSDL Input:
> [Not specified]
>
> Prerequisites:
> BP1701
>
> Profile Requirements:
> -Target:
> R1011
> -Partial-Target:
> [Not specified]
> -Collateral:
> [Not specified]
>
> Context:
> For a candidate envelope containing a soap:Body element
>
> Assertion Description:
> The soap:Envelope does not have direct children after the soap:Body element
>
> Failure Message:
> The soap:Envelope has a direct child after the soap:Body element.
>
> Failure Detail Description:
> [Not specified]
>
> Comments:
> [Not specified]
>
>
>
  Simplification along the lines Jacques suggested with my mods becomes:-
>
> -------------------------------------
>
> Test Assertion:
> BP1309
>
> Prerequisites:
> BP1701
>
> Trigger:
> [Not specified]
>
> Test Expression:
> For a candidate envelope containing a soap:Body element the
> soap:Envelope does not have direct children after the soap:Body element
>
> Outcome:
> The Test Expression MUST be true
>
> --------------------------------------
>
> Here the whole TA is expressed formally under one heading which I've
> called 'Test Expression'
>
> The MUST aspect (the fact that the TA is required and the failure message
> is sent when the Test Expression is false) 1. is notably part of the
> WS-I structure (though partly implicit in the fact that the message they
> mention is for a failure - from which one can logically deduce what
> constitutes failure in relation to the test assertion) 2. is separated
> from the 'Test Expression' but uses it as a reference point to describe
> what constitutes success or failure 3. is summed up all in one place in
> the proposed TA structure under the heading 'Outcome', 4. comprises both
> the concept of failure (implicit still) and the property applied to the
> TA as a whole of 'required' in this example.
>
> I tend to accept that there is reason enough in this example to keep the
> prerequisite of a previous test having completed as a separate heading.
> This is because that previous test has itself a prerequisite and so on
> and listing them all for this particular TA back to the starting point
> would be unrealistic in this example. In don't call it a precondition
> here though because that term is overloaded and because the concept has
> been put forward in our group that there can be shared prerequisites which
> can be placed either in individual TAs (originally this would have been
> as preconditions but I would now tend away from that term) or at the head
> of a group of TAs (put forward in the Korean SC's F2F slides).
>
> So now I think my proposition in the call of a structure below is bourne out
> by this and other such examples which can readily be expressed this way with
> little or no loss of information compared with more complex structuring.

  TA: (ID, prose, references)

  ...

  ---------------------------

  Prerequisites: (Reference, say, to the ID of a TA to have run before)

  Trigger: (Optional. Could this be a choice between 'Trigger' and  
something else
            like 'Entry Type'? Perhaps coded / enumerated.)

  Test Expression: (The main expression - a predicate)

  Outcome: (Relates the predicate boolean to MUST, SHOULD, etc. Typically like
            "The Test Expression MUST be true". It could be coded / enumerated)

  ---------------------------

>
> -- 
> Stephen Green
>
> Partner
> SystML, http://www.systml.co.uk
> Tel: +44 (0) 117 9541606
>
> http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice
>
>
>
> Quoting "Durand, Jacques R." <JDurand@us.fujitsu.com>:
>
>> A couple of comments on the notion of "Structural vs. behavioral" tests:
>>
>> 1. If we can put a general definition on these, and if it happens that
>> each one of these two categories presents different challenges for TA
>> writing, it may be worth some mention in our TA guide.  It is an
>> intuitive classification even for those not UML-literate.
>>
>> - challenge: not always clear which category a TA should fall into:
>> consider a requirement R1 on a message header structure (either a field
>> must be conform to a format, or the header itself be schema-valid).
>> Another spec requirement R2 says this message header is expected to be
>> produced in response to some operation on the message handler.
>> Two options here:
>> (a) TA for R1: the IUT is the header document --> a structural test.
>> TA for R2: the IUT is the message handler --> a behavior test, but that
>> uses TA-for-R1 as pre-requisite (must be satisfied for a "pass" outcome)
>> (b) TA for R1+R2: the IUT is the message handler --> a behavior test
>> (header structure test is just part of describing the expected behavior:
>> to send out a well-formed message header)
>>
>>
>> 2. So far we have a candidate TA anatomy (for non-prose Tas) like:
>>
>> - Pre-condition
>> - Test trigger
>> - Test effect
>> - Post-condition (maybe)
>>
>> It seems to me that the notion of test trigger vs test effect is more
>> appropriate for "behavioral"  tests where both "trigger"  and "effect"
>> refer to specific events that are clearly serialized over time (in form
>> of action on test harness, or of behavior of the IUT).
>>
>> But for "structural"  tests like validating a doc, I feel the
>> distinction is somewhat contrived. There might be a single event (e.g.
>> do the doc validation against rule/schema/...) and no "effect" expected
>> in response, besides the result of this validation.
>>
>> How about using instead (for both "structural"  and "behavioral" Tas):
>>
>> - [Pre-condition]
>> - Test Operation: e.g. " validate the doc against rule/schema/..."
>> (covers the entire test operation and possible steps/events if any)
>> - Test Outcome: "fail" if..., "pass" if... (only focus on what are the
>> conditions for "pass", "fail", refering to observed events/output of
>> Test Operation)
>> - [Post-condition]
>>
>> Other benefit:
>> - even in case of a "prose" TA where the prose is just a pointer to the
>> spec requirement, we could require the "Test Outcome" to be there (or at
>> least a generic one shared by many Tas of the same kind - see option (b)
>> in my email 9/21: "the test outcome is PASS if the required behavior is
>> observed, FAIL otherwise.")
>>
>> -Jacques
>>
>>
>>
>>
>> -----Original Message-----
>> From: Dave Pawson [mailto:dave.pawson@gmail.com]
>> Sent: Monday, September 24, 2007 11:55 PM
>> To: tag@lists.oasis-open.org
>> Subject: Re: [tag] TA Model still weak on tests of structure?
>>
>> On 24/09/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk>
>> wrote:
>>
>>> what if something about the item's structure is being  tested? For
>>> example, what if the IUT is a serial number or a  universally unique
>>> identifier or the like and the test is that  it's format conforms to a
>>> particular pattern?
>>
>> Data values internal to a test?
>>
>>> I found that there is a need to equally support both structures and
>>> behaviours. In UML structures come under 'Foundations' and dynamic
>>> aspects under 'Behaviours'. I propose we add something for a
>>> structural aspect to be tested. It could be anything from schema to
>>> class, property to datatype.
>>
>>
>> I.e. it is too general to be clearly defined?
>>
>> Out of scope IMHO.
>>
>> regards
>>
>>
>> --
>> Dave Pawson
>> XSLT XSL-FO FAQ.
>> http://www.dpawson.co.uk
>>