RE: [sca-assembly-editors] [ASSEMBLY-132] Microsoft technical comment aboutthe SCA Assembly Model specification #1 - WAS RE: [sca-assembly] A New SCATesting TC?

[Mike Edwards <mike_edwards@uk.ibm.com>]

Folks,

This got sent to the editors list by
mistake - now copied to the main Assembly list....

Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

From:	"Patil, Sanjay" <sanjay.patil@sap.com>
To:	Mike Edwards/UK/IBM@IBMGB
Date:	21/07/2009 16:26
Subject:	RE: [sca-assembly-editors] [ASSEMBLY-132] Microsoft technical comment about the SCA Assembly Model specification #1 - WAS RE: [sca-assembly] A New SCA Testing TC?

---

Did you intend to limit the participation
in this discussion to the elite members of the SCA Assembly Editors?

---

From: Mike Edwards [mailto:mike_edwards@uk.ibm.com]
Sent: Tuesday, Jul 21, 2009 2:17 AM
To: sca-assembly-editors@lists.oasis-open.org
Subject: [sca-assembly-editors] [ASSEMBLY-132] Microsoft technical
comment about the SCA Assembly Model specification #1 - WAS RE: [sca-assembly]
A New SCA Testing TC?


Folks, 

I'm deliberately changing the title of this discussion thread since the
discussion is about Issue 132 now... 

I think that the problem with Martin's proposal is that it is not easy
to ensure that the version of the test suite and 
the document(s) describing the implementation type are "publicly available"
and "meet the requirements for those artifacts".

- and I'm saying that even though I am sympathetic to Martin's proposal.

One approach to this is to lay down detailed requirements for both the
test suite and for the document(s).  The problem with this
that I see is that while we can probably hammer down the technical requirements
quite well, I am not sure that we will be able to 
handle the legal aspects at all.  What does "publicly available"
mean?  What are "acceptable license/usage terms"?

One possible way around this is that we can get together a set of technical
requirements for the test suite and the document(s)
but that judging whether the test suite and document(s) meet the technical
requirements and that the terms of availability are 
acceptable is left to the OpenCSA member section committee (or failing
them, the OASIS board...). This gives us a level of
flexibility while preventing abuse of the system. 


I again note that if the test suite and document(s) for a new implementation
type are done under the aegis of OASIS, all these 
problems melt away. 

Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

From:	"Patil, Sanjay" <sanjay.patil@sap.com>
To:	"Martin Chapman" <martin.chapman@oracle.com>, Mike Edwards/UK/IBM@IBMGB, "OASIS Assembly" <sca-assembly@lists.oasis-open.org>
Date:	20/07/2009 20:07
Subject:	RE: [sca-assembly] A New SCA Testing TC?

---

The spirit of this proposal makes sense to me. However I am not sure I
follow what 'publicly available' means here. Where would the material for
claiming conformance for the new implementation type be published? Would
such material be available under same terms as applicable for similar material
published by the OpenCSA TCs? 
  
-- Sanjay

---

From: Martin Chapman [mailto:martin.chapman@oracle.com]
Sent: Friday, Jul 17, 2009 5:59 AM
To: 'Mike Edwards'; 'OASIS Assembly'
Subject: RE: [sca-assembly] A New SCA Testing TC?

On the MS/Siemens issue, how about we change the conformance requirements
in the assembly spec to the effect of a) and b) in your email.
  
Currently we say in Section 13.2 
… 
      3.The implementation MUST support and comply with
at least one of the OpenCSA Member Section adopted implementation types.
  
How about replacing this with (something like): 
  
      3. The implementation MUST support and comply with
a publically available implementation type. A publically available implementation
type MUST: 
      a) Have a publicly available version of the test suite
written in the implementation type, available for any party to download,
validate and execute 
      b) Have a publicly available document(s) describing
the implementation type, which, at a minimum, describes the mapping from
 implementation type artifacts to the componentType
when the implementation artifact is used
as the implementation of an SCA Component. 
      Note that OpenSCA Member Section adopted implementation
types fulfill these requirements. 
  
  
  
  
  
From: Mike Edwards [mailto:mike_edwards@uk.ibm.com]
Sent: 17 July 2009 10:01
To: OASIS Assembly
Subject: [sca-assembly] A New SCA Testing TC? 
  

Folks, 

Bob's points below really go into the area of the responses that we owe
Microsoft and Siemens 
relating to their public review comments on the Assembly spec.

It is my belief that the Assembly tests have been structured in such a
way that they are adaptable
to a any new future implementation type, although I agree that the TestCases
document needs to 
be more explicit and normative about which parts of the testcases can be
changed and what 
changes are allowed. 

The problem remains however, about how we can ensure that a vendor implementing
a new 
implementation type and wanting to claim conformance for their runtime
using that implementation 
type satisfies 2 key requirements:

a) Have a publicly available version of the test suite written in the new
implementation type, 
which can be used by all and which would be subject to scrutiny for the
validity of the translation 
of the testcases. 

b) Have a publicly available document describing the new implementation
type which at a 
minimum describes the mapping from implementation type artifacts to the
componentType 
when the implementation artifact is used as the implementation of an SCA
Component. 


So far, the only route we have of guaranteeing both of these requirements
is to require that 
the new implementation language is described by a specification developed
under the 
aegis of an OASIS TC. 

I am interested to hear if there some alternatives available which will
provide these requirements 
while allowing some alternative approach to the development and publishing
of the materials. 



I don't think that a "Testing TC" affects these questions much
at all.  I tend to agree with Bob that
the value of a separate Testing TC is questionable and there remains an
issue relating to the 
implementation type specific tests that an impl language TC may want to
require as part of the 
conformance requirements for their spec.  To me, these tests seem
to naturally belong to the 
language TC itself.  Having a separate TC with different membership
seems like a recipe for 
conflict and/or missing testcases....


Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

From:	Bob Freund <bob.freund@hitachisoftware.com>
To:	ashok.malhotra@oracle.com
Cc:	OASIS Assembly Test <sca-assembly-testing@lists.oasis-open.org>, OASIS Policy <sca-policy@lists.oasis-open.org>, OASIS Bindings <sca-bindings@lists.oasis-open.org>, OASIS Java <sca-j@lists.oasis-open.org>, OASIS BPEL <sca-bpel@lists.oasis-open.org>, OASIS CPP <sca-c-cpp@lists.oasis-open.org>
Date:	16/07/2009 22:03
Subject:	Re: [sca-policy] A New SCA Testing TC?

 

---

Ashok,
I am somewhat sympathetic to the argument that it ought to be possible
 
for a new language to be brought in to the family, but I am concerned  
about creating a centralized testing TC.
One main reason is that it is unclear how long such a TC would be  
staffed or its members remain interested.  A new language could come
 
along years from now.
IMO it would be better to describe the language interface tests in a  
language independent manner so that they could be implemented in  
whatever language as may arise in the future.
The problem is the determination of the correct implementation of  
those "meta" tests.  Perhaps there is some technical solution,
but it  
escapes my limited imagination.
an alternate approach might be that the implementors might self- 
certify and be required to publish the tests used as well as the  
results before being able to claim conformance.  The customer's might
 
them be able to judge for themselves the degree of rigor used and thus
 
the quality of the self-certification.
-bob

On Jul 15, 2009, at 4:42 PM, ashok malhotra wrote:

> If you have been following the Assembly Testing work, you know that
 
> the Assembly Test Cases are written in Java.
> Mike is now preparing a BPEL version.
>
> Clearly, the Java test cases test the Java C&I to some extent
and  
> Bryan has raised an issue that asks whether the Assembly and Java
 
> tests should be more cleanly separated, but there are  
> complications.  The Assembly tests need some C&I to test
and some  
> duplication cannot be prevented.  Also, if the Assembly tests
and  
> the language tests were separated, then who adjudicate differences
 
> of opinion and duplication?  Further, if the Assembly tests were
 
> shorn of any language C&I, someone could come along, pass only
the  
> Assembly, Policy and Bindings tests and claim SCA compliance.
>
> So, I'm suggesting we consider forming a new TC that would be  
> responsible for all SCA Testing.
> Before you go "Aaargh!  Not another TC!", please consider
the  
> advantages:
>
> 1. Bryan's issue would be closed as there would be only a single TC
 
> and it would write and manage all the tests, some
> of which would, of course, cover Assembly and one or more language
 
> C&Is.   In fact, thinking about it, many of the tests
> would cover some Assembly features, some Policy, a binding and some
 
> C&I.
>
> 2.. If all the tests were in a single TC there would be no need for
 
> someone else to settle disputes.
>
> 3. If an outside party were to come and claim SCA compliance, the
 
> Testing TC would have the authority to vet their
> tests and say 'yay' or 'nay'.  Or to enforce which tests they
should  
> run.
>
> What do people think?
> -- 
> All the best, Ashok
>
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS
at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU