OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

tag message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [tag] Re: TAG Proposal on weak predicates


Jacques:

I like this better when opinion is replaced with fact
- when the constraints are fact and so 'testability'
is fact (though I'm not sure that 'testability' *can*
ever be fact... philosophically in any case).

However, though I accept that your case (b) is an
important one to have covered somewhere, I'm still
not sure that the TA is the right home for it.

I remember that a key consideration right from the
start for the TA Guidelines and the TC was that there
was an existing concept of 'test metadata' and that
we would not let the scope go into that area. My
little understanding of test metadata is that is differs
from a TA by its being tied to a test suite whereas the
TA is independant of any single test suite. Test suites
change and come and go but the TAs are kind of
expected to live as long as the spec. So even if it
isn't test metadata we need for case (b), it may still
be a verging on being out of scope in the same way
that test metadata is out of scope.

Maybe there is scope to suggest the 'outcome' (when
dependant on an existing test suite or known test
suite requirements) be a possible extension of the
TA but really it is still data which comes between the
TA and a given test suite. Maybe it isn't a property of
the test suite as such but a property of the association
between a TA and a given test suite or test suite type.
So perhaps it isn't necessarily strictly test metadata
either, so I sympathize with the dilemma and the
expediency of including it in the TA. The trouble is that
that specialises the TA to always being associated with
that test suite or test suite type. As such we are, in
my mind, talking of a specialisation of TAs which we
*might* want to consider as simple enough for our
initial TA Guidelines but maybe rather it belongs to our
proposed advance guidelines yet to come. Maybe
any TA markup would best leave an extension point
for this and anything else like it. I really don't think we
would want to encourage *everyone* to tie their TAs
to a particular test suite or test suite technology as
might happen if we include this 'TA outcome' / 'TA
interpretation' in the basic level TA Guidelines (unless
we call it out somehow as an extension suggestion
for a specialized case of test assertion usage.


Best regards

Steve


-- 
Stephen D. Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606
Associate Director
Document Engineering Services
http://www.documentengineeringservices.com

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



2008/7/15 Durand, Jacques R. <JDurand@us.fujitsu.com>:
> Stephen:
>
> That reinforces my opinion we need to define better what "testability"
> means, as this term seems central to all TA definitions I know of.
>
>>The 'other side of the coin' of this opinion is that the TA is *not* a
> proper place
>> for such opinions because 1) they are opinions (whereas a TA is
> otherwise to be a statement
>> of fact following on from the spec)
>
> Don't these opinions become facts when testability limits are known? (I
> am not talking of
> That's why I suspect there are two major contexts to consider in writing
> TAs:
>
> (a)- situations where no testing constraints are known nor need be
> assumed. Here it is expected that every TA will have a predicate that
> always matches the normative statement.
>
> (b)- situations where "testability" is defined by a set of testing
> constraints that are known up front. In such cases it is expected that
> some normative statements will not be testable, some will be partially
> testable.
>
> The ideal case is (a) but my experience is that (b) is quite common. And
> in (b) case many people see value in writing TAs that do not ignore
> these testing constraints. Can we convince them they should write their
> TAs like in (a)? I am afraid not. I'd think twice before telling them
> that our guideline is not for them...
>
>
> Jacques
>
>
> -----Original Message-----
> From: stephengreenubl@gmail.com [mailto:stephengreenubl@gmail.com] On
> Behalf Of Stephen Green
> Sent: Saturday, July 12, 2008 2:42 PM
> To: Durand, Jacques R.
> Cc: Patrick.Curran@sun.com; OASIS TAG TC
> Subject: Re: [tag] Re: TAG Proposal on weak predicates
>
> Comment/question below
>
> 2008/7/12 Durand, Jacques R.
>> I suggest we add a
>> "Testability" section (~1 page) in our guideline.
>> This section would deal with all the fuzziness behind the notion of
>> "testability" , which is precisely what a guideline is about, i.e.
>> best practices rather than exact science). Because testability is a
>> relative concept dependent on what test constraints are assumed, I
>> suspect that our guideline might have to consider a few cases:
>> (a) the TA writer assumes that it is always possible to derive test
>> case(s) from the TA, given the right test environment. In other words,
>
>> it is assumed there is no restriction on the execution of test cases.
>> (b) the TA writer works under "testability" constraint, and writes TA
>> with these testing constraints in mind.
>> I suspect that neither (a) nor (b) is wrong...
>>
>
>
> A TA writer will doubtless have in mind (to a greater or lesser extent)
> some expectation of a set of test constraints (else how will they
> consider what it means that their TAs are 'testable'). They will
> doubtless be able and likely to form opinions, as they write the
> assertions, about how testable the assertions are and how testable the
> corresponding spec statements are.
> I think there is no disagreement that such thoughts could be captured
> and should, when useful, be captured. What we seem to have a split of
> opinion on
> (interestingly) is *how* those thoughts may or should be used and
> associated at all with the corresponding TA.
>
> A few of us are thinking that the proper place for them is ultimately in
> metadata to be associated with test suites/cases. The 'other side of the
> coin' of this opinion is that the TA is *not* a proper place for such
> opinions because 1) they are opinions (whereas a TA is otherwise to be a
> statement of fact following on from the spec) an 2) including them with
> the TA risks diluting the logic of the spec (even if that logic is
> already considered by the TA writer to be flawed or weak in some way).
>
> Against this opinion, on the other side as it were, what reason would
> there be for including opinions about expectations of test constraints
> and/or opinions about corresponding testability of a TA in the TA
> itself?
>
> --
> Stephen D. Green
>
> Partner
> SystML, http://www.systml.co.uk
> Tel: +44 (0) 117 9541606
> Associate Director
> Document Engineering Services
> http://www.documentengineeringservices.com
>
> http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice
>


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]