[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [tag] Early users of TAG guidelines... for review
Jacques, The comment rewording looks fine to me. Minor response inline below... Best regards Steve 2008/12/1 Durand, Jacques R. <JDurand@us.fujitsu.com>: > Stephen: > > Overall I find your comments fine. > I am reformatting them more tersely as a list of bullet items below, for > ease of read and referencing, adding my 2 cents, let me know your > comments / rewords: > > --------------------------------- > - The Test Assertions Guidelines are well implemented in this early set > of test assertions. The complexity is mimized without reducing the > expressiveness of the test assertions. > > - It appears that the authors so far only needed a subset of the > recommended Prescription Level values (here: Mandatory, Preferred, > Permitted), as they make appropriate use of negative expressions in > Predicate instead. Will you follow this approach for the entire set of > SCA Assembly TAs? Let us know how you think of the notion of > prescription level and of their suggested values. Yes, good > > - The TA parts Definitions are reworded a bit from the original ones in > the TAG document. Be careful of subtle differences: > (1) prerequisite should be more precisely introduced as a boolean > expression that qualifies the target for this TA (meaning the TA does > NOT apply to this target if it fails the prereq) Yes, good > (2) Predicate reads "The meat of the assertion - something that should > evaluate to true or false for the given target.", seems to relate to the > fact that the predicate 'should' be a boolean expression. It does > however allow a little confusion with the fact that the predicate, it > could be said, should evaluate to 'true' when the requirement to which > the test assertion relates is fulfilled. A short sentence relating the > way the test assertion and particularly its predicate relate to > requirement fulfilment (taking into account the value of the > prescription level and the way the predicate can itself include the > 'NOT' > operator) might help to clarify this. > --------------------------------- Yes, my wording here was a bit obtuse, but OK and I'm the additions are good > > I am a bit concerned adding: > [In the Test Assertion Guidelines this is not always the case since the > prescription level may be negative and so the predicate can then > evaluate to false when the requirement is fulfilled.] > Because I am not sure that's what the TC agreed on... Can we really have > Predicate="false" meaning requirement is fulfilled? Well, it makes sense to me that if we allow negative Prescription Levels we have to accept that the most straightforward way to interpret how they apply to a TA is that they make the TA fulfillment such that it is true if the predicate is false. Otherwise the TA evaluation is not so intuitive and there is a danger of confusion in the interpretation of the test result and ambiguity if the normative source is overlooked. So it seems to me anyway. I agree it would be nicer if 'the TA is always fulfilled when the predicate evaluates to true in relation to the test(s)'. The clarity this would add to TA logic is the best reason, to me, to exclude negative Prescription Levels (PLs). Logically, I can't see any advantage in using a negative PL if the predicate is such that it evaluates to 'true' when the requirement is fulfilled: Why not just use a positive PL with such a predicate? A double negative seems to me to produce a positive if both negatives are part of the same TA: one the predicate and one the PL. This seems unsatisfactory when you can just use two postives instead: positive PL and positive predicate. To make it the rule that a negative PL has to be accompanied by a negative predicate just so the predicate can still evaluate to true: The confusion this might cause would to me be worse than only including the positive Prescription Levels. Anyway, maybe my logic is flawed here so I'd better not press my point further. > > -Jacques > > -----Original Message----- > From: stephengreenubl@gmail.com [mailto:stephengreenubl@gmail.com] On > Behalf Of Stephen Green > Sent: Wednesday, November 26, 2008 1:09 PM > To: Durand, Jacques R. > Cc: tag@lists.oasis-open.org > Subject: Re: [tag] Early users of TAG guidelines... for review > > Comment on SCA Assembly Testing initial use of Test Assertions > Guidelines: > > The Test Assertions Guidelines are well implemented in this early set of > test assertions. The complexity is mimized without reducing the > expressiveness of the test assertions. > > It is noted that there was appropriate exercise of discretion in > subsetting the values of the Prescription Level. It seems that, at least > with this set of test assertions, the 'positive' prescription level > values (Mandatory, Preferred, > Permitted) are adequate and negative values are redundant when it is a > straigh- forward matter to use a negated predicate instead. It would be > interesting to see if this subset is adequate for all SCA Assembly > Testing TAs. > > It is noted that abbreviated and adapted definitions of test assertion > parts are used at the start of the test assertion document to explain > the terms used. > There is some degree of ambiguity in the definition of the Predicate. > The meaning of the definition, which reads "The meat of the assertion - > something that should evaluate to true or false for the given target.", > seems to relate to the fact that the predicate 'should' be a boolean > expression. It does however allow a little confusion with the fact that > the predicate, it could be said, should evaluate to 'true' when the > requirement to which the test assertion relates is fulfilled. A short > sentence relating the way the test assertion and particularly its > predicate relate to requirement fulfilment (taking into account the > value of the prescription level and the way the predicate can itself > include the 'NOT' > operator) might help to clarify this. [In the Test Assertion Guidelines > this is not always the case since the prescription level may be negative > and so the predicate can then evaluate to false when the requirement is > fulfilled.] > > > >>> A link which allows non-member access is >>> >>> http://www.oasis-open.org/committees/download.php/30177/SCA_Assembly_ >>> Test_Assertions_01.doc >>> >>> 2008/11/25 Durand, Jacques R. <JDurand@us.fujitsu.com> >>>> >>>> FYI >>>> ________________________________ >>>> From: Mike Edwards [mailto:mike_edwards@uk.ibm.com] >>>> Sent: Tuesday, November 25, 2008 3:45 AM >>>> To: OASIS Assembly Test >>>> Cc: OASIS Java; Durand, Jacques R. >>>> Subject: Re: [Fwd: RE: [sca-j] testing architecture] >>>> >>>> >>>> Folks, >>>> >>>> Well, I did use the TAG spec in the creation of the SCA Assembly > Test Assertions document: >>>> >>>> http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/do >>>> wnload.php/30177/SCA_Assembly_Test_Assertions_01.doc >>>> >>>> Comments welcome, especially from TAG experts !! >>>> >>>> >>>> Yours, Mike. >>>> >>>> Strategist - Emerging Technologies, SCA & SDO. >>>> Co Chair OASIS SCA Assembly TC. >>>> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great > Britain. >>>> Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431 >>>> Email: mike_edwards@uk.ibm.com >>>> >>>> >>>> From: Anish Karmarkar <Anish.Karmarkar@oracle.com> >>>> To: OASIS Java <sca-j@lists.oasis-open.org> >>>> Date: 25/11/2008 06:02 >>>> Subject: [Fwd: RE: [sca-j] testing architecture] >>>> ________________________________ >>>> >>>> >>>> Forwarding per Jacques' request. He is not a member of SCA-J and >>>> cannot post to this list. >>>> >>>> -Anish >>>> -- >>>> >>>> -------- Original Message -------- >>>> Subject: RE: [sca-j] testing architecture >>>> Date: Mon, 24 Nov 2008 18:13:18 -0800 >>>> From: Durand, Jacques R. <JDurand@us.fujitsu.com> >>>> To: Anish Karmarkar <Anish.Karmarkar@oracle.com>, OASIS Java >>>> <sca-j@lists.oasis-open.org> >>>> References: >>>> <OF4390713D.430E4D13-ON85257501.00463C7F-85257501.0046D95C@us.ibm.co >>>> m> >>>> <49276ABD.5040906@oracle.com> >>>> <OFC0A22A74.BBA99BD8-ON8525750B.004A895F-8525750B.004C052C@us.ibm.co >>>> m> >>>> <492B345E.3000003@oracle.com> >>>> >>>> Anish: >>>> >>>> Yes I'm listening :-) >>>> And indeed some legal issue still keeps me from full SCA membership, > >>>> but I am willing to help on the testing. The XSLT-based analyzer >>>> technology we developed for WS-I can certainly be a good starting >>>> point, as far as I can tell, to derive something suitable for SCA. >>>> >>>> On the TA methodology side, worth noting: the Test Assertions >>>> Guidelines >>>> (TAG) OASIS TC that I am chairing is almost done with its guideline > doc: >>>> first CD can be seen at: >>>> http://www.oasis-open.org/apps/org/workgroup/tag/document.php?docume >>>> nt_i >>>> d=30070 >>>> Not surprisingly, the TA methodology and model of this guideline is >>>> already applied by WS-I successfully ;-). I drafted a 2-page digest >>>> of this guideline, which is visible on the Tech Advisory Board (TAB) > >>>> wiki (proposal in progress - no OASIS / TAB commitment!) >>>> http://wiki.oasis-open.org/tab/TestabilityGuide >>>> >>>> I am saying this because the ambition of some of us is to provide an > >>>> XSLT-based analyzer technology for this more general TA mark-up. >>>> >>>> So my advice is to indeed (1) to reuse the TA model mentioned above, > >>>> and >>>> (2) adopt Xpath2 (preferably not XPath1) for scripting some parts of > >>>> the TA (target, co-target if any, prerequisite if any, predicate). >>>> Of course, prior to scripting Tas you will have to define an xml >>>> markup for the material under test to be used as Analyzer input >>>> (combining metadata like bindings, and message capture). Ther >>>> Analyzer technology itself is independent from your specific > mark-ups. >>>> >>>> I could explain more over an informal conf call after ThxGiving. >>>> >>>> Jacques >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Anish Karmarkar [mailto:Anish.Karmarkar@oracle.com] >>>> Sent: Monday, November 24, 2008 3:10 PM >>>> To: OASIS Java >>>> Cc: Durand, Jacques R. >>>> Subject: Re: [sca-j] testing architecture >>>> >>>> Thanks for the clarification. >>>> >>>> +1 to creating architecture to cover all the TCs. >>>> >>>> Thinking more on this, I'm not sure in which situations we would >>>> necessarily need two runtimes. Initially, I was thinking of testing >>>> references which have absolute URIs of services, but I think this >>>> can still be managed without two runtimes (as long as the URLs where > >>>> the services are deployed can be determined apriori). I'm sure you >>>> have thought of cases where we would necessarily require two >>>> runtimes for testing. Would appreciate if you can elaborate on that. >>>> >>>> One more thing we might want to think about is binding testing. This > >>>> would require capturing messages on the wire and analyzing them. >>>> This is very similar to what WS-I has done. They defined a log >>>> format, where messages get logged and wrote an analyzer to correlate > >>>> and validate the messages with the metadata (in WS-I's case WSDL). >>>> We might want to do something similar. Not sure if Jacques Durand >>>> from Fujitsu (cc'ed) is a member of sca-j or is listening, but he >>>> was/is heavily involved in writing a XSLT based analyzer to validate > >>>> messages on the wire in WS-I and may have some opinions on this. >>>> >>>> -Anish >>>> -- >>>> >>>> David Booz wrote: >>>> > I'll try to clarify the "two runtime" question. In my charts, the >>>> > two runtimes are meant to be from the same vendor. The last bullet > >>>> > on the first slide was supposed to make this clear. >>>> > >>>> > From the whiteboard pic, the two runtimes are indeed from >>>> > different vendors but runtime 2 was added to the board as part of >>>> > a different discussion not addressed by the slides. That larger >>>> > discussion was about what constitutes a compliant runtime, which >>>> > bindings needed to be supported, etc. We didn't come to any >>>> > conclusions on this discussion, thus I did not include it in the > slides. >>>> > >>>> > We certainly don't need two runtimes (even from the same vendor) >>>> > for all the tests, but we need it for some of the tests. The >>>> > architecture was intended to cover all the TCs. See the 3rd bullet > >>>> > on the 1st slide. I really don't want 6 testing architectures. I >>>> > want the TCs to collaborate and share as much as possible. We need > >>>> > to position ourselves to take advantage of economies of scale. >>>> > >>>> > I hope that clarifies. >>>> > >>>> > Dave Booz >>>> > STSM, BPM and SCA Architecture >>>> > Co-Chair OASIS SCA-Policy TC and SCA-J TC "Distributed objects >>>> > first, then world hunger" >>>> > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 >>>> > e-mail:booz@us.ibm.com >>>> > >>>> > Inactive hide details for Anish Karmarkar ---11/21/2008 09:20:10 >>>> > PM---Dave, Thanks for sending this out.Anish Karmarkar >>>> > ---11/21/2008 09:20:10 PM---Dave, Thanks for sending this out. >>>> > >>>> > >>>> > From: >>>> > Anish Karmarkar <Anish.Karmarkar@oracle.com> >>>> > >>>> > To: >>>> > sca-j@lists.oasis-open.org >>>> > >>>> > Date: >>>> > 11/21/2008 09:20 PM >>>> > >>>> > Subject: >>>> > Re: [sca-j] testing architecture >>>> > >>>> > ------------------------------------------------------------------ >>>> > ---- >>>> > -- >>>> > >>>> > >>>> > >>>> > Dave, >>>> > >>>> > Thanks for sending this out. >>>> > I certainly agree with the 1st slide. >>>> > On the 2nd slide (as well as in the whiteboard picture), it shows >>>> > two runtimes (in the whiteboard picture, it shows two runtimes >>>> > from two different vendors). Since I wasn't present for the >>>> > testing discussion, >>>> >>>> > I may not be interpreting this correctly, but why do we >>>> > necessarily need two runtimes for testing? >>>> > >>>> > To pass our exit criteria [1], we need to have at least two >>>> > independent implementations. But I don't see that we necessarily >>>> > need to have two runtimes for all of our tests. >>>> > >>>> > For example, a very simple test would be an echo/ping consisting >>>> > of two Java components (with PBV and deployed locally) A and B >>>> > with B's ref connected to A's service. The logic of B could >>>> > consist of invoking >>>> >>>> > A's service and println-ing the return value into some stream that > >>>> > could be directed to stdio/file. >>>> > >>>> > For binding test, it can get tricky because we have to capture >>>> > messages on the wire and ensure that it did the Right Thing, and >>>> > every >>>> >>>> > implementation may not provide access to the messages, possibly >>>> > requiring two runtimes. But for non-binding tests we don't need > that. >>>> > Furthermore, even for binding tests, we *may* be able to get away >>>> > with >>>> >>>> > a single implementation by using utilities like tcpmon and setting > >>>> > the >>>> >>>> > URLs to facilitate redirecting/forwarding. >>>> > >>>> > I hope I haven't completely misinterpreted the slides. >>>> > >>>> > -Anish >>>> > -- >>>> > >>>> > [1] http://www.oasis-open.org/committees/sca-j/charter.php >>>> > >>>> > >>>> > David Booz wrote: >>>> > > Ashok asked me to recap my testing thoughts and the picture, >>>> > and put > them onto some slides. Some of these thoughts come from > >>>> > Mike E also, so > I don't want mis-represent where they are >>>> > coming from. I'm >>>> >>>> > happy to add > more material (there's only two slides here) as we > >>>> > progress with the > discussion and clarify the ideas. The last >>>> > slide in this deck is meant > to be equivalent to the white board > >>>> > picture that Mark C captured in a > photo and sent to the list. >>>> > > >>>> > > /(See attached file: testing-design.ppt)/ > > > Dave Booz > > >>>> > STSM, BPM and SCA Architecture > Co-Chair OASIS SCA-Policy TC and > >>>> > SCA-J TC > "Distributed objects first, then world hunger" >>>> > > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 > >>>> > e-mail:booz@us.ibm.com > > > >>>> > ------------------------------------------------------------------ >>>> > ---- >>>> > -- >>>> > > >>>> > > >>>> > ------------------------------------------------------------------ >>>> > --- > To unsubscribe from this mail list, you must leave the >>>> > OASIS TC that > generates this mail. Follow this link to all >>>> > your TCs in >>>> OASIS at: >>>> > > >>>> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups >>>> > .php >>>> > >>>> > ------------------------------------------------------------------ >>>> > --- To unsubscribe from this mail list, you must leave the OASIS >>>> > TC that generates this mail. Follow this link to all your TCs in >>>> > OASIS at: >>>> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups >>>> > .php >>>> > >>>> > >>>> > >>>> >>>> -------------------------------------------------------------------- >>>> - To unsubscribe from this mail list, you must leave the OASIS TC >>>> that generates this mail. Follow this link to all your TCs in OASIS > >>>> at: >>>> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.p >>>> hp >>>> >>>> >>>> >>>> >>>> >>>> >>>> ________________________________ >>>> >>>> Unless stated otherwise above: >>>> IBM United Kingdom Limited - Registered in England and Wales with > number 741598. >>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire >>>> PO6 3AU >>>> >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Stephen D. Green >>> >>> Document Engineering Services Ltd >>> >>> >>> >>> http://www.biblegateway.com/passage/?search=matthew+22:37 .. and >>> voice >>> >>> >> > > > > -- > Stephen D. Green > > Document Engineering Services Ltd > > > > http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice > -- Stephen D. Green Document Engineering Services Ltd http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]