Re: [tag] Groups - TA Anatomy V0.4 (AnatomyTA-v04.doc) uploaded

[stephen.green@systml.co.uk]

Further to this I did a bit more reading, especially looking at the
UML 2 Testing Profile (link to this on the wiki?)
http://www.omg.org/docs/formal/05-07-07.pdf

and it seems we may need to be even more agnostic about the possible
evaluations of the tests based on the TA since there are differences
in implementations:
we so far have suggested values for the test 'verdict' of
pass
fail
something like either 'not applicable' or 'test is not implemented'
(http://lists.oasis-open.org/archives/tag/200709/msg00016.html)

the UML Testing Profile has
pass
fail
inconclusive (maybe equates to our 'not applicable' or 'not implemented')
error

JUnit has
pass
fail
error

So I guess we can only state something in prose without necessarily
suggesting values (and without suggesting the number of possible verdict
values) since we are agnostic about what technology is used to turn the
assertions into tests.

We could perhaps just mandate that the test not take place or that the
equivalent of a pass verdict is not recorder if the pre and post conditions
are not met but stretching this technology agnosticity yet further I do
question whether we can say that expressions used to define parts of the
TA such as the conditions, trigger and result are necessarily predicates
and that they cannot contain MUST, etc since that may make too much of
an assumption about the test technology.

-Steve






> So there is no pass or fail of the TA I gather
> but isn't it written as if there is? That confuses me a bit
> (it needs careful wording which I haven't attempted to figure out).
>
> The TA isn't the test case so it doesn't evaluate to anything.
> It just gives information to the test case about how it ought
> to be evaluated to fulfil spec requirements, say (or on the
> test based on the test case based on the TA).
>
> Yet we have determined the beginning of rules to say something like
> that a postcondition or precondition which evaluates to false means the test
> should evaluate to 'not applicable'. Maybe we just need to extend that rule
> explicitly to say
>
> test (or test based on test case) based on TA evaluates to pass if (and
> only if)
> either
> test based on TA... flow result (if exists) evaluates to true
> AND TA postconditions and preconditions and test based on TA.. triggers
> (if any)
> all evaluate to true
> or
> test... based on prose evaluates to pass or true (should we make
> provision to say which is expected or is that test-case-specific)
> or
> test... based on pointer evaluates to pass or true (should we make
> provision to say which is expected?)
>
> (maybe there needs to be some rule about priority of the above)
>
> then similar rules for how the test would evaluate to fail or 'not   
> applicable'
>
>
>
>
>
> Quoting Dave Pawson <dave.pawson@gmail.com>:
>
>> On 20/09/2007, stephen.green@systml.co.uk   
>> <stephen.green@systml.co.uk> wrote:
>>> Quoting Dave Pawson <dave.pawson@gmail.com>:
>>>
>>>> On 20/09/2007, stephen.green@systml.co.uk
>>> <stephen.green@systml.co.uk> wrote:
>>>>
>>>>> Isn't the main factor here wanting to make the test assertion into an
>>>>> expression with a boolean evaluation (a predicate - I think we need to
>>>>> explain that term by the way)?
>>>>
>>>> Why complicate it Stephen? We want a pass/fail result is simpler?
>>>>
>>>
>>> So are we making it then a convention that true = pass ?
>>> i.e. TRUE for a predicate TA result equates to PASS for prose TA narrative
>>>
>>> If so, maybe we should state that explicitly
>>>
>>> Don't we need to be clear which parts of the model evaluate to true/false
>>> and which parts to pass/fail?
>>
>>
>> Which model?
>>
>> An assertion == a test. A test must be designed to pass or fail.
>> Any use of English beyond that would appear to complicate matters?
>> The definition of pass must be explicit within the test document.
>> Anything else is a fail.
>>
>> regards
>>
>> --
>> Dave Pawson
>> XSLT XSL-FO FAQ.
>> http://www.dpawson.co.uk
>>