[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [tag] Re: TAG Proposal on weak predicates
Jacques: I like this better when opinion is replaced with fact - when the constraints are fact and so 'testability' is fact (though I'm not sure that 'testability' *can* ever be fact... philosophically in any case). However, though I accept that your case (b) is an important one to have covered somewhere, I'm still not sure that the TA is the right home for it. I remember that a key consideration right from the start for the TA Guidelines and the TC was that there was an existing concept of 'test metadata' and that we would not let the scope go into that area. My little understanding of test metadata is that is differs from a TA by its being tied to a test suite whereas the TA is independant of any single test suite. Test suites change and come and go but the TAs are kind of expected to live as long as the spec. So even if it isn't test metadata we need for case (b), it may still be a verging on being out of scope in the same way that test metadata is out of scope. Maybe there is scope to suggest the 'outcome' (when dependant on an existing test suite or known test suite requirements) be a possible extension of the TA but really it is still data which comes between the TA and a given test suite. Maybe it isn't a property of the test suite as such but a property of the association between a TA and a given test suite or test suite type. So perhaps it isn't necessarily strictly test metadata either, so I sympathize with the dilemma and the expediency of including it in the TA. The trouble is that that specialises the TA to always being associated with that test suite or test suite type. As such we are, in my mind, talking of a specialisation of TAs which we *might* want to consider as simple enough for our initial TA Guidelines but maybe rather it belongs to our proposed advance guidelines yet to come. Maybe any TA markup would best leave an extension point for this and anything else like it. I really don't think we would want to encourage *everyone* to tie their TAs to a particular test suite or test suite technology as might happen if we include this 'TA outcome' / 'TA interpretation' in the basic level TA Guidelines (unless we call it out somehow as an extension suggestion for a specialized case of test assertion usage. Best regards Steve -- Stephen D. Green Partner SystML, http://www.systml.co.uk Tel: +44 (0) 117 9541606 Associate Director Document Engineering Services http://www.documentengineeringservices.com http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice 2008/7/15 Durand, Jacques R. <JDurand@us.fujitsu.com>: > Stephen: > > That reinforces my opinion we need to define better what "testability" > means, as this term seems central to all TA definitions I know of. > >>The 'other side of the coin' of this opinion is that the TA is *not* a > proper place >> for such opinions because 1) they are opinions (whereas a TA is > otherwise to be a statement >> of fact following on from the spec) > > Don't these opinions become facts when testability limits are known? (I > am not talking of > That's why I suspect there are two major contexts to consider in writing > TAs: > > (a)- situations where no testing constraints are known nor need be > assumed. Here it is expected that every TA will have a predicate that > always matches the normative statement. > > (b)- situations where "testability" is defined by a set of testing > constraints that are known up front. In such cases it is expected that > some normative statements will not be testable, some will be partially > testable. > > The ideal case is (a) but my experience is that (b) is quite common. And > in (b) case many people see value in writing TAs that do not ignore > these testing constraints. Can we convince them they should write their > TAs like in (a)? I am afraid not. I'd think twice before telling them > that our guideline is not for them... > > > Jacques > > > -----Original Message----- > From: stephengreenubl@gmail.com [mailto:stephengreenubl@gmail.com] On > Behalf Of Stephen Green > Sent: Saturday, July 12, 2008 2:42 PM > To: Durand, Jacques R. > Cc: Patrick.Curran@sun.com; OASIS TAG TC > Subject: Re: [tag] Re: TAG Proposal on weak predicates > > Comment/question below > > 2008/7/12 Durand, Jacques R. >> I suggest we add a >> "Testability" section (~1 page) in our guideline. >> This section would deal with all the fuzziness behind the notion of >> "testability" , which is precisely what a guideline is about, i.e. >> best practices rather than exact science). Because testability is a >> relative concept dependent on what test constraints are assumed, I >> suspect that our guideline might have to consider a few cases: >> (a) the TA writer assumes that it is always possible to derive test >> case(s) from the TA, given the right test environment. In other words, > >> it is assumed there is no restriction on the execution of test cases. >> (b) the TA writer works under "testability" constraint, and writes TA >> with these testing constraints in mind. >> I suspect that neither (a) nor (b) is wrong... >> > > > A TA writer will doubtless have in mind (to a greater or lesser extent) > some expectation of a set of test constraints (else how will they > consider what it means that their TAs are 'testable'). They will > doubtless be able and likely to form opinions, as they write the > assertions, about how testable the assertions are and how testable the > corresponding spec statements are. > I think there is no disagreement that such thoughts could be captured > and should, when useful, be captured. What we seem to have a split of > opinion on > (interestingly) is *how* those thoughts may or should be used and > associated at all with the corresponding TA. > > A few of us are thinking that the proper place for them is ultimately in > metadata to be associated with test suites/cases. The 'other side of the > coin' of this opinion is that the TA is *not* a proper place for such > opinions because 1) they are opinions (whereas a TA is otherwise to be a > statement of fact following on from the spec) an 2) including them with > the TA risks diluting the logic of the spec (even if that logic is > already considered by the TA writer to be flawed or weak in some way). > > Against this opinion, on the other side as it were, what reason would > there be for including opinions about expectations of test constraints > and/or opinions about corresponding testability of a TA in the TA > itself? > > -- > Stephen D. Green > > Partner > SystML, http://www.systml.co.uk > Tel: +44 (0) 117 9541606 > Associate Director > Document Engineering Services > http://www.documentengineeringservices.com > > http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]