TAB COmments on Test Assertions for SCA_J Common Annotations andAPIs v1

comments from Jacques Durand (as a review assignment from the OASIS TAB) on:

Test Assertions for SCA_J Common Annotations and APIs v1.1

-------------------------------------------------------------------------------------------

[Comment 1] : weak design: Target is often not well identified, in several TAs.

In some TAs, the Target is not exactly what the Predicate is talking about.

For example: in several TAs about runtime behavior it is not clear that

the target (what is being tested and expected to either pass or fail) should be an actual SCA runtime.

Example: JCA-TA-2005

Target = a Java class ("Java implementation class marked with "composite scope" and with "eager

initialization")

Predicate = a statement on a runtime behavior ("The Java implementation class instance is created and initialized")

The real test Target here is the SCA runtime and its expected behavior, not the Java class.

A rule of thumb is: the target is what should fail or pass a test based on this test assertion.

Also the Predicate should refer to this Target as its main subject:

Target = SCA runtime with a Java implementation class marked with "composite scope" and with "eager

initialization".

Predicate = the SCA runtime (Target) creates and initialize the Java implementation class instance.

Or it is also OK to use a prerequisite and say:

Target = SCA runtime.

Prerequisite = the Java implementation class of the runtime is marked with "composite scope" and with "eager

initialization".

Predicate = the SCA runtime (Target) creates and initialize the Java implementation class instance.

(The difference is that when using a Prerequisite, allows a test tool to tell you about those targets that do not

meet the Prerequisite condition , in the test report).

Same remark for JCA-TA-2006, JCA-TA-2007 and some others.

A TA that is accurately stated is JCA-TA-2003 (target: Stateless scoped Java implementation instance" is clearly

refering to an instance as opposed to a Class, so does the Predicate)

-------------------------------------------------------------------------------------------

[Comment 2] : (minor) Target is often too fine-grain, leading to convoluted references.

Example: JCA-TA-3005

Target = @remotable attribute on <interface.java/> element

Prerequisite = Java interface class referenced by @interface attribute of interface.java

element contains a @Remotable annotation

Predicate = @remotable attribute has the value "true"

Here, it is preferable to target the <interface.java/> element rather than its attribute,

because then the Prerequisite can clearly refer to this target:

Target = <interface.java/> element with a @remotable attribute.

Prerequisite = Java interface class referenced by @interface attribute of interface.java element (the Target)

contains a @Remotable annotation

Predicate = @remotable attribute (of Target) has the value "true"

-------------------------------------------------------------------------------------------

[Comment 3] : some spec requirements may need more than one TA.

For some requiremetns such as JCA30005, two TAs could be written instead of one (say JCA-TA-3005a and JCA-TA-3005b) ,

as this spec requirement is also about class consistency as well as about SCA runtime behavior:

JCA-TA-3005a: renaming of current JCA-TA-3005.

JCA-TA-3005b:

Target = SCA runtime using a component with <interface.java/> element with a @remotable attribute of value "false".

Prerequisite = Java interface class referenced by @interface attribute of interface.java element

used by the Target contains a @Remotable annotation

Predicate = The SCA runtime (target) raises an error

[Comment 4] : Target should not be an "event", but an identifiable entity.

Maybe that is just a typo, but the Target in some TAs reads like an "event":

For example in JCA-TA-2009:

Target = "Client invocation of a method of a service implementation"

But what the specification is talking about in 2.3.4, is really an SCA runtime and its behavior:

(Target should always be an entity that is subject to conformance to the specification,

something that pass or fail a test - and not just an "event".)

Target = SCA runtime that is invoking a method with input parameters of the remotable service

Prerequisite =

(a) Service method implementation marked "allows pass by reference"

(b) Client reference proxy marked "allows pass by reference"

Predicate = the client SCA runtime is using pass-by-reference call semantics for the inputs.

Note above the narrowing on the *client* SCA runtime (not the invoked component),

so the Predicate only talks about what depends on client behavior: input parameters.

The statement in 2.3.4 apparently also applies to the remote component behavior,

so another TA could be written here (JCA-TA-2009b)

because thats not quite the same behavior that is tested

(you could pass JCA-TA-2009a and fail JCA-TA-2009b):

Target = SCA runtime that is being invoked by a client on a method with return value.

Prerequisite =

(a) Service method in the service implementation (or Target) marked "allows pass by reference"

(b) Client reference proxy marked "allows pass by reference"

Predicate = the service SCA runtime is using pass-by-reference call semantics for the returned value.

--------------------------------------------------------------------------

[Comment 5] : stronger use of "tags" might help classify TAs based on targets:

Tags seem to be used exclusively as "keywords" in TAs. A suggestion is to use them also

for categorizing the TA based on its target type, as this may help figure which TAs relate to which

conformance target, when creating the test suite.

So one could define general categories of "test targets", e.g.:

- SCA_runtime_component

- SCA_runtime_client

- interface.java_element

- etc.

Then one may use a named tags in each TA, for this information:

JCA-TA-2005:

...

tag:target_category = "SCA_runtime_component"

JCA-TA-3005:

...

tag:target_category = "interface.java_element"

sca-j-comment message