[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [sca-assembly-testing] Re: [sca-j] testing architecture
I mostly agree. But I think there will be cases where examining the messages will be required. For example, if in any of the bindings spec we specify things that must/must not be sent on the wire it would require us to look at the messages. How else could we tell whether the conformance point was tested? -Anish -- David Booz wrote: > +1 to Mike. > > I'm not in favor of doing binding checking via examination of messages > as we're not doing interop testing. It should be possible to do > _sufficient_ binding tests simply by observing the outcome of the > interaction, along the lines of quantum principles that Mike likes to > bring up from time to time. > > Dave Booz > STSM, BPM and SCA Architecture > Co-Chair OASIS SCA-Policy TC and SCA-J TC > "Distributed objects first, then world hunger" > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 > e-mail:booz@us.ibm.com > > Inactive hide details for Mike Edwards ---11/25/2008 06:38:35 > AM---Folks, This is an interesting set of comments.Mike Edwards > ---11/25/2008 06:38:35 AM---Folks, This is an interesting set of comments. > > > From: > Mike Edwards <mike_edwards@uk.ibm.com> > > To: > "OASIS Assembly Test " <sca-assembly-testing@lists.oasis-open.org> > > Cc: > Jacques Durand <JDurand@us.fujitsu.com>, OASIS Java > <sca-j@lists.oasis-open.org> > > Date: > 11/25/2008 06:38 AM > > Subject: > Re: [sca-j] testing architecture > > ------------------------------------------------------------------------ > > > > > Folks, > > This is an interesting set of comments. > > Regarding the two runtimes, we should ponder that further - whether some > test scenarios > should demand that two components run in 2 separate processes (I note > that we do make > distinction between local and remote in some cases). I believe that this > is why "2 runtimes" > was mentioned in the test design document. I anticipate that most tests > would use one > runtime - and I agree with Anish that even testing of remotable > interfaces with interoperable > bindings can still be tested with a single runtime. However, we need to > get specific with > some of the tests. > > Regarding Binding testing, this is an interesting question. It will be > largely for the Bindings > TC to decide what level of conformance is required. One approach is the > message level > checking that is outlined below. If there are resources to build or > adapt message capture > and processing programs, then this may be feasible, but I have a concern > about the cost > involved. > > There is an alternative, which is to build or adapt a non-SCA program to > act as Client or as > Server for services over a given Binding technology - and that program > can embody the > required binding behaviour. (eg An Apache Axis2 program for Web > services, a JMS > program for JMS, etc.) This too may be non-trivial, but may also be a > more straightforward > approach to the testing problem. > > Yours, Mike. > > Strategist - Emerging Technologies, SCA & SDO. > Co Chair OASIS SCA Assembly TC. > IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain. > Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431 > Email: mike_edwards@uk.ibm.com > > From: Anish Karmarkar <Anish.Karmarkar@oracle.com> > To: OASIS Java <sca-j@lists.oasis-open.org> > Cc: Jacques Durand <JDurand@us.fujitsu.com> > Date: 24/11/2008 23:10 > Subject: Re: [sca-j] testing architecture > > > ------------------------------------------------------------------------ > > > > Thanks for the clarification. > > +1 to creating architecture to cover all the TCs. > > Thinking more on this, I'm not sure in which situations we would > necessarily need two runtimes. Initially, I was thinking of testing > references which have absolute URIs of services, but I think this can > still be managed without two runtimes (as long as the URLs where the > services are deployed can be determined apriori). I'm sure you have > thought of cases where we would necessarily require two runtimes for > testing. Would appreciate if you can elaborate on that. > > One more thing we might want to think about is binding testing. This > would require capturing messages on the wire and analyzing them. This is > very similar to what WS-I has done. They defined a log format, where > messages get logged and wrote an analyzer to correlate and validate the > messages with the metadata (in WS-I's case WSDL). We might want to do > something similar. Not sure if Jacques Durand from Fujitsu (cc'ed) is a > member of sca-j or is listening, but he was/is heavily involved in > writing a XSLT based analyzer to validate messages on the wire in WS-I > and may have some opinions on this. > > -Anish > -- > > David Booz wrote: > > I'll try to clarify the "two runtime" question. In my charts, the two > > runtimes are meant to be from the same vendor. The last bullet on the > > first slide was supposed to make this clear. > > > > From the whiteboard pic, the two runtimes are indeed from different > > vendors but runtime 2 was added to the board as part of a different > > discussion not addressed by the slides. That larger discussion was about > > what constitutes a compliant runtime, which bindings needed to be > > supported, etc. We didn't come to any conclusions on this discussion, > > thus I did not include it in the slides. > > > > We certainly don't need two runtimes (even from the same vendor) for all > > the tests, but we need it for some of the tests. The architecture was > > intended to cover all the TCs. See the 3rd bullet on the 1st slide. I > > really don't want 6 testing architectures. I want the TCs to collaborate > > and share as much as possible. We need to position ourselves to take > > advantage of economies of scale. > > > > I hope that clarifies. > > > > Dave Booz > > STSM, BPM and SCA Architecture > > Co-Chair OASIS SCA-Policy TC and SCA-J TC > > "Distributed objects first, then world hunger" > > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 > > e-mail:booz@us.ibm.com > > > > Inactive hide details for Anish Karmarkar ---11/21/2008 09:20:10 > > PM---Dave, Thanks for sending this out.Anish Karmarkar ---11/21/2008 > > 09:20:10 PM---Dave, Thanks for sending this out. > > > > > > From: > > Anish Karmarkar <Anish.Karmarkar@oracle.com> > > > > To: > > sca-j@lists.oasis-open.org > > > > Date: > > 11/21/2008 09:20 PM > > > > Subject: > > Re: [sca-j] testing architecture > > > > ------------------------------------------------------------------------ > > > > > > > > Dave, > > > > Thanks for sending this out. > > I certainly agree with the 1st slide. > > On the 2nd slide (as well as in the whiteboard picture), it shows two > > runtimes (in the whiteboard picture, it shows two runtimes from two > > different vendors). Since I wasn't present for the testing discussion, I > > may not be interpreting this correctly, but why do we necessarily need > > two runtimes for testing? > > > > To pass our exit criteria [1], we need to have at least two independent > > implementations. But I don't see that we necessarily need to have two > > runtimes for all of our tests. > > > > For example, a very simple test would be an echo/ping consisting of two > > Java components (with PBV and deployed locally) A and B with B's ref > > connected to A's service. The logic of B could consist of invoking A's > > service and println-ing the return value into some stream that could be > > directed to stdio/file. > > > > For binding test, it can get tricky because we have to capture messages > > on the wire and ensure that it did the Right Thing, and every > > implementation may not provide access to the messages, possibly > > requiring two runtimes. But for non-binding tests we don't need that. > > Furthermore, even for binding tests, we *may* be able to get away with a > > single implementation by using utilities like tcpmon and setting the > > URLs to facilitate redirecting/forwarding. > > > > I hope I haven't completely misinterpreted the slides. > > > > -Anish > > -- > > > > [1] _http://www.oasis-open.org/committees/sca-j/charter.php_ > > > > > > David Booz wrote: > > > Ashok asked me to recap my testing thoughts and the picture, and put > > > them onto some slides. Some of these thoughts come from Mike E > also, so > > > I don't want mis-represent where they are coming from. I'm happy > to add > > > more material (there's only two slides here) as we progress with the > > > discussion and clarify the ideas. The last slide in this deck is meant > > > to be equivalent to the white board picture that Mark C captured in a > > > photo and sent to the list. > > > > > > /(See attached file: testing-design.ppt)/ > > > > > > > > > Dave Booz > > > STSM, BPM and SCA Architecture > > > Co-Chair OASIS SCA-Policy TC and SCA-J TC > > > "Distributed objects first, then world hunger" > > > Poughkeepsie, NY (845)-435-6093 or 8-295-6093 > > > e-mail:booz@us.ibm.com > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > --------------------------------------------------------------------- > > > To unsubscribe from this mail list, you must leave the OASIS TC that > > > generates this mail. Follow this link to all your TCs in OASIS at: > > > > _https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php_ > > > > --------------------------------------------------------------------- > > To unsubscribe from this mail list, you must leave the OASIS TC that > > generates this mail. Follow this link to all your TCs in OASIS at: > > _https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php_ > > > > > > > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail. Follow this link to all your TCs in OASIS at:_ > __https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php_ > > > > > > > ------------------------------------------------------------------------ > / > / > > /Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU/ > > > > >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]