[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [tag] Early users of TAG guidelines... for review
Review comment: There are a couple of points with the effect of making the test assertions somewhat ambiguous: 1. the 'predicate' in the SCA TAs is defined like this: "... something that should evaluate to true or false for the given target." This could be strictly speaking correct, except that the second difference is 2. only the positive values (out of the both positive and negative TAG values) are given in the SCA TAs for the 'prescription level': "Mandatory (for MUST requirements) or Preferred (for SHOULD requirements) or Permitted (for MAY requirements)" Although these two differences could be said to cancel out, strictly speaking what happens if the predicate 'should evaluate to false' as described? Does it ever evaluate to false? And if so how would we be able to tell that this was the case without the negative prescription levels 'Prohibited' and 'Not Recommended'? Suggested fix: either A) add the negative prescription levels and keep the predicate definition as it is - which would be in line with TAG or B) if there is a desire to limit the prescription levels, and diverge a little from TAG (which are, after all, guidelines rather than normative requirements), then just make the definition of the predicate "... something that should evaluate to true for the given target." B) would differ from TAG, then, in two respects but prevent a degree of ambiguity of the TAs. As it is I believe the small change of either A) or B) would fix the logic. I favor A) so that there can be freedom as to whether the predicate is stated positively or negatively in case there is a need to reduce ambiguity by stating the expression one way or the other (perhaps contrary to the way the spec has been worded). (Should I send this to the SCA comments list or can it just be relayed?) Best regards Steve -- Stephen D. Green Document Engineering Services Ltd http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice 2008/11/25 Stephen Green <stephen.green@documentengineeringservices.com> > > A link which allows non-member access is > > http://www.oasis-open.org/committees/download.php/30177/SCA_Assembly_Test_Assertions_01.doc > > 2008/11/25 Durand, Jacques R. <JDurand@us.fujitsu.com> >> >> FYI >> ________________________________ >> From: Mike Edwards [mailto:mike_edwards@uk.ibm.com] >> Sent: Tuesday, November 25, 2008 3:45 AM >> To: OASIS Assembly Test >> Cc: OASIS Java; Durand, Jacques R. >> Subject: Re: [Fwd: RE: [sca-j] testing architecture] >> >> >> Folks, >> >> Well, I did use the TAG spec in the creation of the SCA Assembly Test Assertions document: >> >> http://www.oasis-open.org/apps/org/workgroup/sca-assembly-testing/download.php/30177/SCA_Assembly_Test_Assertions_01.doc >> >> Comments welcome, especially from TAG experts !! >> >> >> Yours, Mike. >> >> Strategist - Emerging Technologies, SCA & SDO. >> Co Chair OASIS SCA Assembly TC. >> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain. >> Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431 >> Email: mike_edwards@uk.ibm.com >> >> >> From: Anish Karmarkar <Anish.Karmarkar@oracle.com> >> To: OASIS Java <sca-j@lists.oasis-open.org> >> Date: 25/11/2008 06:02 >> Subject: [Fwd: RE: [sca-j] testing architecture] >> ________________________________ >> >> >> Forwarding per Jacques' request. He is not a member of SCA-J and cannot >> post to this list. >> >> -Anish >> -- >> >> -------- Original Message -------- >> Subject: RE: [sca-j] testing architecture >> Date: Mon, 24 Nov 2008 18:13:18 -0800 >> From: Durand, Jacques R. <JDurand@us.fujitsu.com> >> To: Anish Karmarkar <Anish.Karmarkar@oracle.com>, OASIS Java >> <sca-j@lists.oasis-open.org> >> References: >> <OF4390713D.430E4D13-ON85257501.00463C7F-85257501.0046D95C@us.ibm.com> >> <49276ABD.5040906@oracle.com> >> <OFC0A22A74.BBA99BD8-ON8525750B.004A895F-8525750B.004C052C@us.ibm.com> >> <492B345E.3000003@oracle.com> >> >> Anish: >> >> Yes I'm listening :-) >> And indeed some legal issue still keeps me from full SCA membership, >> but I am willing to help on the testing. The XSLT-based analyzer >> technology we developed for WS-I can certainly be a good starting point, >> as far as I can tell, to derive something suitable for SCA. >> >> On the TA methodology side, worth noting: the Test Assertions Guidelines >> (TAG) OASIS TC that I am chairing is almost done with its guideline doc: >> first CD can be seen at: >> http://www.oasis-open.org/apps/org/workgroup/tag/document.php?document_i >> d=30070 >> Not surprisingly, the TA methodology and model of this guideline is >> already applied by WS-I successfully ;-). I drafted a 2-page digest of >> this guideline, which is visible on the Tech Advisory Board (TAB) wiki >> (proposal in progress - no OASIS / TAB commitment!) >> http://wiki.oasis-open.org/tab/TestabilityGuide >> >> I am saying this because the ambition of some of us is to provide an >> XSLT-based analyzer technology for this more general TA mark-up. >> >> So my advice is to indeed (1) to reuse the TA model mentioned above, and >> (2) adopt Xpath2 (preferably not XPath1) for scripting some parts of the >> TA (target, co-target if any, prerequisite if any, predicate). >> Of course, prior to scripting Tas you will have to define an xml markup >> for the material under test to be used as Analyzer input (combining >> metadata like bindings, and message capture). Ther Analyzer technology >> itself is independent from your specific mark-ups. >> >> I could explain more over an informal conf call after ThxGiving. >> >> Jacques >> >> >> >> -----Original Message----- >> From: Anish Karmarkar [mailto:Anish.Karmarkar@oracle.com] >> Sent: Monday, November 24, 2008 3:10 PM >> To: OASIS Java >> Cc: Durand, Jacques R. >> Subject: Re: [sca-j] testing architecture >> >> Thanks for the clarification. >> >> +1 to creating architecture to cover all the TCs. >> >> Thinking more on this, I'm not sure in which situations we would >> necessarily need two runtimes. Initially, I was thinking of testing >> references which have absolute URIs of services, but I think this can >> still be managed without two runtimes (as long as the URLs where the >> services are deployed can be determined apriori). I'm sure you have >> thought of cases where we would necessarily require two runtimes for >> testing. Would appreciate if you can elaborate on that. >> >> One more thing we might want to think about is binding testing. This >> would require capturing messages on the wire and analyzing them. This is >> very similar to what WS-I has done. They defined a log format, where >> messages get logged and wrote an analyzer to correlate and validate the >> messages with the metadata (in WS-I's case WSDL). We might want to do >> something similar. Not sure if Jacques Durand from Fujitsu (cc'ed) is a >> member of sca-j or is listening, but he was/is heavily involved in >> writing a XSLT based analyzer to validate messages on the wire in WS-I >> and may have some opinions on this. >> >> -Anish >> -- >> >> David Booz wrote: >> > I'll try to clarify the "two runtime" question. In my charts, the two >> > runtimes are meant to be from the same vendor. The last bullet on the >> > first slide was supposed to make this clear. >> > >> > From the whiteboard pic, the two runtimes are indeed from different >> > vendors but runtime 2 was added to the board as part of a different >> > discussion not addressed by the slides. That larger discussion was >> > about what constitutes a compliant runtime, which bindings needed to >> > be supported, etc. We didn't come to any conclusions on this >> > discussion, thus I did not include it in the slides. >> > >> > We certainly don't need two runtimes (even from the same vendor) for >> > all the tests, but we need it for some of the tests. The architecture >> > was intended to cover all the TCs. See the 3rd bullet on the 1st >> > slide. I really don't want 6 testing architectures. I want the TCs to >> > collaborate and share as much as possible. We need to position >> > ourselves to take advantage of economies of scale. >> > >> > I hope that clarifies. >> > >> > Dave Booz >> > STSM, BPM and SCA Architecture >> > Co-Chair OASIS SCA-Policy TC and SCA-J TC "Distributed objects first, >> > then world hunger" >> > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 e-mail:booz@us.ibm.com >> > >> > Inactive hide details for Anish Karmarkar ---11/21/2008 09:20:10 >> > PM---Dave, Thanks for sending this out.Anish Karmarkar ---11/21/2008 >> > 09:20:10 PM---Dave, Thanks for sending this out. >> > >> > >> > From: >> > Anish Karmarkar <Anish.Karmarkar@oracle.com> >> > >> > To: >> > sca-j@lists.oasis-open.org >> > >> > Date: >> > 11/21/2008 09:20 PM >> > >> > Subject: >> > Re: [sca-j] testing architecture >> > >> > ---------------------------------------------------------------------- >> > -- >> > >> > >> > >> > Dave, >> > >> > Thanks for sending this out. >> > I certainly agree with the 1st slide. >> > On the 2nd slide (as well as in the whiteboard picture), it shows two >> > runtimes (in the whiteboard picture, it shows two runtimes from two >> > different vendors). Since I wasn't present for the testing discussion, >> >> > I may not be interpreting this correctly, but why do we necessarily >> > need two runtimes for testing? >> > >> > To pass our exit criteria [1], we need to have at least two >> > independent implementations. But I don't see that we necessarily need >> > to have two runtimes for all of our tests. >> > >> > For example, a very simple test would be an echo/ping consisting of >> > two Java components (with PBV and deployed locally) A and B with B's >> > ref connected to A's service. The logic of B could consist of invoking >> >> > A's service and println-ing the return value into some stream that >> > could be directed to stdio/file. >> > >> > For binding test, it can get tricky because we have to capture >> > messages on the wire and ensure that it did the Right Thing, and every >> >> > implementation may not provide access to the messages, possibly >> > requiring two runtimes. But for non-binding tests we don't need that. >> > Furthermore, even for binding tests, we *may* be able to get away with >> >> > a single implementation by using utilities like tcpmon and setting the >> >> > URLs to facilitate redirecting/forwarding. >> > >> > I hope I haven't completely misinterpreted the slides. >> > >> > -Anish >> > -- >> > >> > [1] http://www.oasis-open.org/committees/sca-j/charter.php >> > >> > >> > David Booz wrote: >> > > Ashok asked me to recap my testing thoughts and the picture, and >> > put > them onto some slides. Some of these thoughts come from Mike E >> > also, so > I don't want mis-represent where they are coming from. I'm >> >> > happy to add > more material (there's only two slides here) as we >> > progress with the > discussion and clarify the ideas. The last slide >> > in this deck is meant > to be equivalent to the white board picture >> > that Mark C captured in a > photo and sent to the list. >> > > >> > > /(See attached file: testing-design.ppt)/ > > > Dave Booz > >> > STSM, BPM and SCA Architecture > Co-Chair OASIS SCA-Policy TC and >> > SCA-J TC > "Distributed objects first, then world hunger" >> > > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 > >> > e-mail:booz@us.ibm.com > > > >> > ---------------------------------------------------------------------- >> > -- >> > > >> > > >> > --------------------------------------------------------------------- >> > > To unsubscribe from this mail list, you must leave the OASIS TC >> > that > generates this mail. Follow this link to all your TCs in >> OASIS at: >> > > >> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php >> > >> > --------------------------------------------------------------------- >> > To unsubscribe from this mail list, you must leave the OASIS TC that >> > generates this mail. Follow this link to all your TCs in OASIS at: >> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php >> > >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe from this mail list, you must leave the OASIS TC that >> generates this mail. Follow this link to all your TCs in OASIS at: >> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php >> >> >> >> >> >> >> ________________________________ >> >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU >> >> >> >> > > > > -- > Stephen D. Green > > Document Engineering Services Ltd > > > > http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice > >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]