Fw: TAB Comments on Test Assertions for SCA_J Common Annotations and APIsv1.1

[Mike Edwards <mike_edwards@uk.ibm.com>]

Folks,

I am copying Jacques' reply to my email
to the SCA-J mailing list as Jacques is unable to post there directly.

Yours, Mike

Dr Mike Edwards		Mail Point 146, Hursley Park
STSM		Winchester, Hants SO21 2JN
SCA & Services Standards		United Kingdom
Co-Chair OASIS SCA Assembly TC
IBM Software Group
Phone:	+44-1962 818014
Mobile:	+44-7802-467431 (274097)
e-mail:	mike_edwards@uk.ibm.com

----- Forwarded by Mike
Edwards/UK/IBM on 16/02/2011 09:10 -----

From:	Jacques Durand <JDurand@us.fujitsu.com>
To:	Mike Edwards/UK/IBM@IBMGB
Cc:	"sca-j-comment@lists.oasis-open.org" <sca-j-comment@lists.oasis-open.org>, OASIS Java <sca-j@lists.oasis-open.org>
Date:	15/02/2011 22:36
Subject:	RE: TAB Comments on Test Assertions for SCA_J Common Annotations and APIs v1.1

---

Mike:
 
Thanks for your response.
Fair enough.
Indeed there are assumptions
your TC made (e.g. about the default “target” of the tests) , that help
understand better some of your TAs, once known.
My comments were more from
an outsider perspective – i.e. to make sure your TA doc is well understood
by external folks with little background. Nothing critical.
I think it is great that
you found the TAG work useful to produce such a significant corpus of test
assertions !
 
(please forward as appropriate
– I may not have posting rights)
 
Cheers,
Jacques
 
From: Mike Edwards [mailto:mike_edwards@uk.ibm.com]
Sent: Monday, February 14, 2011 8:35 AM
To: Jacques Durand
Cc: sca-j-comment@lists.oasis-open.org; OASIS Java
Subject: Re: TAB Comments on Test Assertions for SCA_J Common Annotations
and APIs v1.1
 

Jacques, 

The SCA J technical committee thanks you and the TAB TC for the comments
on the SCA_J Common Annotations and APIs V1.1 Test Assertions.

The following are responses to your comments:

Issue JAVA-217: [Comment 1] : weak design: Target is often not well identified,
in several TAs. 

JCA-TA-2005, JCA-TA-2006,
JCA-TA-2007 and some others.

It is stated that the target of these statements is incorrect and that
the "real" target should be an SCA runtime. 

First, an observation in general that the test assertions for pretty well
all of the SCA specifications are all about verifying 
the behaviour of some SCA runtime, so that we take the view that attempting
to define the runtime as the target is pretty 
well useless in defining the test assertions.  Instead, the SCA TCs
took the view that what the assertions need to do is to 
concentrate on "observables" - things that could be measured
in one way or another. 

So taking JCA-TA-2005 as an example of a wider class of assertions, here
the "observable" is the behaviour of a Java class 
that is marked in a particular way - some specific behaviour is expected
when the SCA runtime deals with the class, and it 
is this behaviour that is observable - the creation and initialization
in this case.  We believe that it is correct and useful to 
describe the assertion in terms of this observable behaviour (and indeed
we successfully created a testcase that probes 
for this behaviour). 

As a result of these considerations, the TC is not minded to make changes
to the assertions along the lines that you suggest. 

Similarly, shifting some of the material into the prerequisite statement
is not particularly helpful as suggested does not in these 
cases appear to improve the assertions - reporting on artifacts that don't
meet the prerequisites is not the aim of the testcases 
at all - the testcases are deliberately set up to have artifacts that meet
certain conditions to probe that such artifacts are then 
handled correctly when presented to the SCA runtime. 


Issue JAVA 218: [Comment 2] : (minor)
Target is often too fine-grain, leading to convoluted references.  

It is not clear to us that the references identified in JCA-TA-3005 are
in any way convoluted - the target is clearly identified as the
specific attribute of an element and the predicate is clearly a statement
about the value that the attribute must have. 
It is not clear to the TC that the offered alternative formulation is any
clearer than the original. 


Issue JAVA-219: [Comment 3] : some spec requirements may need more than
one TA 

In the case of the SCA specifications, there is a general blanket requirement
that if there are artifacts in error, then the runtime must raise
an error.  We have taken the view that in general test assertions
are not created for each normative statement merely to state that an
error must be raised by the runtime if the statement is violated.  In
many cases, negative testcases are written that do depend on the
runtime raising an error, but this is ancillary to the "positive"
form of the test assertion that is used.

As a result, the TC is not minded to create the second negative form of
the assertion that is proposed.


Issue JAVA-220:  [Comment 4] : Target should not be an "event",
but an identifiable entity. 

As for comment 1, it is taken as read that it is an SCA runtime is involved,
but again, the issue is what is observable.
In this case the client invocation is what is observable (what happens
in the runtime itself is unobservable, other than
its impact on the invocation).

The invocation may sound like an event, but in reality it is a completed
process - and so a measurable "thing" that either
conforms or not. 

As for splitting the assertion into separate parts for the parameters and
the returned value, that is something that a testcase
might choose to do, but it is not a necessary change for the test assertion.
 In reality I think our testcase checks for both at
the same time. 


Issue JAVA-221: [Comment 5] : stronger
use of "tags" might help classify TAs based on targets:  

It is possible that the tags could be improved, but the suggested ones
such as "SCA_Runtime_component" and "SCA_Runtime_client"
are not appropriate - these do not reflect anything in the specification.

In general, where a specific element such as <interface.java/> is
the target, then the tags do typically contain the name of the element.
- such 
as JCA-TA-3002 



Yours, Mike

Dr Mike Edwards		Mail Point 146, Hursley Park
STSM		Winchester, Hants SO21 2JN
SCA & Services Standards		United Kingdom
Co-Chair OASIS SCA Assembly TC
IBM Software Group
Phone:	+44-1962 818014
Mobile:	+44-7802-467431 (274097)
e-mail:	mike_edwards@uk.ibm.com

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU