Compliance Tests

[Rex Brooks <rexb@starbourne.com>]

Hi Allen,

I was asked by the message and notification subcommittee to draft 
this message to you concerning the action item from the TC, as it was 
referred to in the msg-sc mtg today, to design or consider what might 
be required for a "compliance" test suite for CAP.

To make sure I understood what was meant by "compliance" I went to 
the TC page and looked under Action Items, and did not find a 
specific item of this name or effect. So I reviewed the minutes, and 
also did not find a reference. There was a discussion about the 
operational and/or "intent" tests that have already been done, but no 
specific mention of what is or was meant by designing or developing 
or discovering requirements for CAP "compliance" tests.

One of the reasons why I volunteered to take on the task of writing 
this message is that I have some familiarity with a conformance test 
suite that has been devised by the Web Services for Remote Portlets 
TC. So I, at least, know something, as little as it is, about this 
particular topic.

No one was able to really narrow down what was expected for this test 
suite. I mentioned that WSRP tested "MUST" assertions In the WSRP TC 
a test suite that has been developed by IBM based on the work of the 
Conformance Subcommittee, which is its own entity, rather than a part 
of another subcommittee.

That's about the extent of what I know, outside of some of the ins 
and outs of what is tested for. However, as far as I can tell all it 
tests for is if an application conforms to the spec. It does not 
"certify" that an application is compliant or conformant. So, insofar 
as testing at all is concerned, it is not intended to be cited for 
public purposes, at least as far as I know.

My sense of the consensus of the Messages and Notifications SC is 
that we do not have enough of an idea of what is needed or wanted in 
a compliance test suite. So, we suggest that asking Karl might be the 
wisest course of action for a number of reasons.

Not least of these reasons is that there may well be OASIS-wide 
policy that bears on the issue or liability concerns over the notion 
that a TC might offer formal or informal testing, and what purposes 
such testing would serve.

So that is what the Messages and Notifications Subcommittee requests: 
Formal guidance from the TC and OASIS on what is needed or requested 
in a suite of tests for compliance with CAP. What should be tested, 
and to what degree.

Thanks,
Rex Brooks
-- 
Rex Brooks
GeoAddress: 1361-A Addison, Berkeley, CA, 94702 USA, Earth
W3Address: http://www.starbourne.com
Email: rexb@starbourne.com
Tel: 510-849-2309
Fax: By Request