RE: [sca-assembly] A New SCA Testing TC?

[Bryan Aupperle <aupperle@us.ibm.com>]

No.  And I am not sure it is appropriate
for the various C & I documents to be concerned with assembly test
cases.

What we may need to do is split the
test case document into a main test case document that defines the test
cases and identifies the implementation type-independent artifacts and
a set of implementation-type test case mapping documents.  These mapping
documents would catalog the implementation type-specific files and their
use by individual test cases.  This mapping document could identify
test cases that are not applicable to an implementation type (and why).
 Any new implementation type would need to provide the appropriate
mapping document.

Bryan Aupperle, Ph.D.
STSM, WebSphere Enterprise Platform Software Solution Architect

Research Triangle Park,  NC 
+1 919-254-7508 (T/L 444-7508)
Internet Address: aupperle@us.ibm.com

"Martin Chapman" <martin.chapman@oracle.com>

07/17/2009 11:00 AM

To	"'Bob Freund'" <bob.freund@hitachisoftware.com>
cc	<ashok.malhotra@oracle.com>, "'OASIS Assembly'" <sca-assembly@lists.oasis-open.org>
Subject	RE: [sca-assembly] A New SCA Testing TC?

Do our current c+is do this? We need to be purer than
white!

> -----Original Message-----
> From: Bob Freund [mailto:bob.freund@hitachisoftware.com]
> Sent: 17 July 2009 15:15
> To: Martin Chapman
> Cc: ashok.malhotra@oracle.com; 'OASIS Assembly'
> Subject: Re: [sca-assembly] A New SCA Testing TC?
> 
> AND, it should contain a specific list of those features (and tests)
> that could not map due to the nature of the implementation language
if
> any.
> -bob
> On Jul 17, 2009, at 10:09 AM, Martin Chapman wrote:
> 
> > Each language has to have a mapping to SCA so b) requires public
> > documentation of how this is done. I think having this will be
> > important to understanding the tests, and aid verification.
> >
> > Martin.
> >
> >> -----Original Message-----
> >> From: ashok malhotra [mailto:ashok.malhotra@oracle.com]
> >> Sent: 17 July 2009 14:58
> >> To: Martin Chapman
> >> Cc: 'OASIS Assembly'
> >> Subject: Re: [sca-assembly] A New SCA Testing TC?
> >>
> >> Martin:
> >> Why do we need b) ?
> >>
> >> If they show the test suite and say they pass all the language-
> >> dependent
> >> and language-independent tests, isn't that sufficient?
> >> All the best, Ashok
> >>
> >>
> >> Martin Chapman wrote:
> >>>
> >>> On the MS/Siemens issue, how about we change the conformance
> >>> requirements in the assembly spec to the effect of a)
and b) in your
> >>> email.
> >>>
> >>> Currently we say in Section 13.2
> >>>
> >>> .
> >>>
> >>> 3.The implementation MUST support and comply with at
least one of
> >>> the
> >>> OpenCSA Member Section adopted implementation types.
> >>>
> >>> How about replacing this with (something like):
> >>>
> >>> 3. The implementation MUST support and comply with a
publically
> >>> available implementation type. A publically available
implementation
> >>> type MUST:
> >>>
> >>> a) Have a publicly available version of the test suite
written in
> >>> the
> >>> implementation type, available for any party to download,
validate
> >>> and
> >>> execute
> >>>
> >>> b) Have a publicly available document(s) describing the
> >>> implementation
> >>> type, which, at a minimum, describes the mapping from
implementation
> >>> type artifacts to the componentType when the implementation
artifact
> >>> is used as the implementation of an SCA Component.
> >>>
> >>> Note that OpenSCA Member Section adopted implementation
types
> >>> fulfill
> >>> these requirements.
> >>>
> >>> *From:* Mike Edwards [mailto:mike_edwards@uk.ibm.com]
> >>> *Sent:* 17 July 2009 10:01
> >>> *To:* OASIS Assembly
> >>> *Subject:* [sca-assembly] A New SCA Testing TC?
> >>>
> >>>
> >>> Folks,
> >>>
> >>> Bob's points below really go into the area of the responses
that we
> >>> owe Microsoft and Siemens
> >>> relating to their public review comments on the Assembly
spec.
> >>>
> >>> It is my belief that the Assembly tests have been structured
in
> >>> such a
> >>> way that they are adaptable
> >>> to a any new future implementation type, although I agree
that the
> >>> TestCases document needs to
> >>> be more explicit and normative about which parts of the
testcases
> >>> can
> >>> be changed and what
> >>> changes are allowed.
> >>>
> >>> The problem remains however, about how we can ensure
that a vendor
> >>> implementing a new
> >>> implementation type and wanting to claim conformance
for their
> >>> runtime
> >>> using that implementation
> >>> type satisfies 2 key requirements:
> >>>
> >>> a) Have a publicly available version of the test suite
written in
> >>> the
> >>> new implementation type,
> >>> which can be used by all and which would be subject to
scrutiny for
> >>> the validity of the translation
> >>> of the testcases.
> >>>
> >>> b) Have a publicly available document describing the
new
> >>> implementation type which at a
> >>> minimum describes the mapping from implementation type
artifacts to
> >>> the componentType
> >>> when the implementation artifact is used as the implementation
of an
> >>> SCA Component.
> >>>
> >>>
> >>> So far, the only route we have of guaranteeing both of
these
> >>> requirements is to require that
> >>> the new implementation language is described by a specification
> >>> developed under the
> >>> aegis of an OASIS TC.
> >>>
> >>> I am interested to hear if there some alternatives available
which
> >>> will provide these requirements
> >>> while allowing some alternative approach to the development
and
> >>> publishing of the materials.
> >>>
> >>>
> >>>
> >>> I don't think that a "Testing TC" affects these
questions much at
> >>> all.
> >>> I tend to agree with Bob that
> >>> the value of a separate Testing TC is questionable and
there remains
> >>> an issue relating to the
> >>> implementation type specific tests that an impl language
TC may want
> >>> to require as part of the
> >>> conformance requirements for their spec. To me, these
tests seem to
> >>> naturally belong to the
> >>> language TC itself. Having a separate TC with different
membership
> >>> seems like a recipe for
> >>> conflict and/or missing testcases....
> >>>
> >>>
> >>> Yours, Mike.
> >>>
> >>> Strategist - Emerging Technologies, SCA & SDO.
> >>> Co Chair OASIS SCA Assembly TC.
> >>> IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN,
Great
> >>> Britain.
> >>> Phone & FAX: +44-1962-818014 Mobile: +44-7802-467431
> >>> Email: mike_edwards@uk.ibm.com
> >>>
> >>> From:
> >>>
> >>>
> >>>
> >>> Bob Freund <bob.freund@hitachisoftware.com>
> >>>
> >>> To:
> >>>
> >>>
> >>>
> >>> ashok.malhotra@oracle.com
> >>>
> >>> Cc:
> >>>
> >>>
> >>>
> >>> OASIS Assembly Test <sca-assembly-testing@lists.oasis-open.org>,
> >>> OASIS
> >>> Policy <sca-policy@lists.oasis-open.org>, OASIS
Bindings
> >>> <sca-bindings@lists.oasis-open.org>, OASIS Java
> >>> <sca-j@lists.oasis-open.org>, OASIS BPEL
> >>> <sca-bpel@lists.oasis-open.org>, OASIS CPP
> >>> <sca-c-cpp@lists.oasis-open.org>
> >>>
> >>> Date:
> >>>
> >>>
> >>>
> >>> 16/07/2009 22:03
> >>>
> >>> Subject:
> >>>
> >>>
> >>>
> >>> Re: [sca-policy] A New SCA Testing TC?
> >>>
> >>> ------------------------------------------------------------------------
> >>>
> >>>
> >>>
> >>>
> >>> Ashok,
> >>> I am somewhat sympathetic to the argument that it ought
to be
> >>> possible
> >>> for a new language to be brought in to the family, but
I am
> >>> concerned
> >>> about creating a centralized testing TC.
> >>> One main reason is that it is unclear how long such a
TC would be
> >>> staffed or its members remain interested. A new language
could come
> >>> along years from now.
> >>> IMO it would be better to describe the language interface
tests in a
> >>> language independent manner so that they could be implemented
in
> >>> whatever language as may arise in the future.
> >>> The problem is the determination of the correct implementation
of
> >>> those "meta" tests. Perhaps there is some technical
solution, but it
> >>> escapes my limited imagination.
> >>> an alternate approach might be that the implementors
might self-
> >>> certify and be required to publish the tests used as
well as the
> >>> results before being able to claim conformance. The customer's
might
> >>> them be able to judge for themselves the degree of rigor
used and
> >>> thus
> >>> the quality of the self-certification.
> >>> -bob
> >>>
> >>> On Jul 15, 2009, at 4:42 PM, ashok malhotra wrote:
> >>>
> >>>> If you have been following the Assembly Testing work,
you know that
> >>>> the Assembly Test Cases are written in Java.
> >>>> Mike is now preparing a BPEL version.
> >>>>
> >>>> Clearly, the Java test cases test the Java C&I
to some extent and
> >>>> Bryan has raised an issue that asks whether the Assembly
and Java
> >>>> tests should be more cleanly separated, but there
are
> >>>> complications. The Assembly tests need some C&I
to test and some
> >>>> duplication cannot be prevented. Also, if the Assembly
tests and
> >>>> the language tests were separated, then who adjudicate
differences
> >>>> of opinion and duplication? Further, if the Assembly
tests were
> >>>> shorn of any language C&I, someone could come
along, pass only the
> >>>> Assembly, Policy and Bindings tests and claim SCA
compliance.
> >>>>
> >>>> So, I'm suggesting we consider forming a new TC that
would be
> >>>> responsible for all SCA Testing.
> >>>> Before you go "Aaargh! Not another TC!",
please consider the
> >>>> advantages:
> >>>>
> >>>> 1. Bryan's issue would be closed as there would be
only a single TC
> >>>> and it would write and manage all the tests, some
> >>>> of which would, of course, cover Assembly and one
or more language
> >>>> C&Is. In fact, thinking about it, many of the
tests
> >>>> would cover some Assembly features, some Policy,
a binding and some
> >>>> C&I.
> >>>>
> >>>> 2.. If all the tests were in a single TC there would
be no need for
> >>>> someone else to settle disputes.
> >>>>
> >>>> 3. If an outside party were to come and claim SCA
compliance, the
> >>>> Testing TC would have the authority to vet their
> >>>> tests and say 'yay' or 'nay'. Or to enforce which
tests they should
> >>>> run.
> >>>>
> >>>> What do people think?
> >>>> --
> >>>> All the best, Ashok
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe from this mail list, you must leave
the OASIS TC
> >>>> that
> >>>> generates this mail. Follow this link to all your
TCs in OASIS at:
> >>>> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ------------------------------------------------------------------------
> >>>
> >>> /Unless stated otherwise above:
> >>> IBM United Kingdom Limited - Registered in England and
Wales with
> >>> number 741598.
> >>> Registered office: PO Box 41, North Harbour, Portsmouth,
Hampshire
> >>> PO6
> >>> 3AU/
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe from this mail list, you must leave the OASIS
TC that
> >> generates this mail.  Follow this link to all your TCs
in OASIS at:
> >> https://www.oasis-open.org/apps/org/workgroup/portal/
> >> my_workgroups.php
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this mail list, you must leave the OASIS
TC that
> > generates this mail.  Follow this link to all your TCs in
OASIS at:
> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
> >



---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php