Re: [tag] some thoughts and comments from reading a month worthofmessages

[stephen.green@systml.co.uk]

I agree that a test error is out of scope. If the test has an error
that is hardly the concern of this generalized TA. It is something
for the test itself to cover.

I'm not sure that applies to a test being inapplicable though - the
outcome of a condition not being met. So if conditions are not met
in a flow the flow might stop or take a particular branch. If the
conditions are not met in an overall TA the TA is inconclusive. We
then can say that 'inconclusive' and 'inappropriate' amount to the
same thing for the TA as a whole but are perhaps not the right terms
for a flow step.

So maybe we can define how a whole TA reaches results of pass, fail
or inconclusive. Then we can say for a step in a flow the step value
can be pass, fail or something else (not sure we have a term for the
value yet as we haven't said much about chaining steps in flows).

Can a TA overall evaluate to true/false or is it always going to be
pass/fail/inconclusive (inconclusive being an outcome of conditions
not being met on certain steps)?

What about a flow step though?

Quoting Dave Pawson <dave.pawson@gmail.com>:

> On 22/09/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote:
>
>> Yes, this is like how I was thinking with the metadata attribute like
>> 'isTrueFalse'. There would be a problem with 'isPassFail' since it
>> might be more than just pass and fail - it might be pass fail or error
>> or it might be pass fail error or inconclusive or even pass fail error
>> inconclusive or inapplicable. It is in some cases determined by the
>> logic of the TA (e.g. depending on whether there are clear conditions
>> to mean a test can be inapplicable/inappropriate) and in most cases
>> determined by the test methodology, to which the TA should be agnostic.
>
> Another scope issue?
> What happens when a test passes / fails (boolean) is an agreement
> with the spec author. If the test has an error (program logic, say)
> then it must fail.
>
> If it is inconclusive, it is badly designed.
> What happens after a pass/fail is an issue for program flow.
>
>
>
>>
>> Maybe the guideline could be that if the value is not boolean (isTrueFalse
>> = false) then it is to be assumed that the outcomes will include pass
>> and fail but not exclusively (other values may be dependent on factors
>> out of scope).
>
> I'd propose that the test is invalid. Define the test pass conditions,
> all else is a fail.
>
>
>
>
>
>
>
>
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
>