OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

sca-assembly message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [sca-assembly] Normative language for conformance testing requirement

At the risk of being overly naive, perhaps the issue is that the pseudo-rfc 2119 capitalized MUST [NOT] usage implies some enforcement, some rigor, some absolute requirement etc., to state the obvious.  "MUST" in standards or standards related documents carries a certain implied weight that's (purposefully) different than "it'd be really nice if..." wording. 

Would it not be sufficient to state [the obvious] that
  • A conforming implementation passes all the tests.  All developers of SCA-conformant implementations are strongly encouraged to verify their conformance using these tests.
That way, we state or imply 1) that the tests are there to use, 2) that they are optional (there is no enforcement), 3) an implication that purchasers of SCA software may be able to verify compliance, etc.  Yet we do not get into the gray areas of Rfc 2119 w.r.t. products, etc.

I'll go back into my hole now.

Thus quoth David Booz (~ 6/3/2009 12:23 PM ~)...
OF57236D6F.53A07E33-ON852575CA.00630341-852575CA.006A8900@us.ibm.com" type="cite">

I told myself yesterday that I wasn't going to get into this debate....oh well.

I don't see what's wrong with "....MUST pass all the tests." This does not say that one MUST run the tests, only that the runtime MUST pass them if someone runs them against your implementation. I'm playing with words. The tests need to have some teeth otherwise there's no point in doing them. Let's also keep in mind some very important points:
1) We want to enable compliance under a self certifying model. There is no enforcement mechanism, nor do I think we want one. A runtime is compliant because the implementer of the runtime says it is. I believe this is a majority view in the TC.
2) The spec is authoritative over the test suite (and even the TA and TC specs), similar to how the spec is authoritative over the XML schemas we use, so the tests don't cause any irreconcilable problems.
3) The tests are open precisely so that anyone can run them. Non-compliant liars will be found out quite publicly by the industry or users of the software. Also, bugs in the test suite can be fixed quickly, in the open so that the test suite can be brought into line with the spec.

As a result, runtime implementation providers will run the tests just to make sure they are compliant. This will happen due to human nature, not due to some RFC keyword in the spec document.

Dave Booz
STSM, BPM and SCA Architecture
Co-Chair OASIS SCA-Policy TC and SCA-J TC
"Distributed objects first, then world hunger"
Poughkeepsie, NY (845)-435-6093 or 8-295-6093

Inactive hide details for Simon Nash ---06/03/2009 09:05:34 AM---Following on from the discussion on yesterday's call, the two Simon Nash ---06/03/2009 09:05:34 AM---Following on from the discussion on yesterday's call, the two forms of language suggested for a norm


Simon Nash <oasis@cjnash.com>


OASIS Assembly <sca-assembly@lists.oasis-open.org>


06/03/2009 09:05 AM


[sca-assembly] Normative language for conformance testing requirement

Following on from the discussion on yesterday's call, the two forms
of language suggested for a normative requirement for conformance
testing were along the lines of
 1) A conforming implementation MUST be able to pass all the tests....
 2) A conforming implementation MUST NOT fail any of the tests....

On the call it was stated that these are logically equivalent.

My preference is for the second form.  An advantage of this form
is that the statement is clear cut and easy to verify.  To show that
an implementation is non-conforming, it is only necessary to produce
a single test that fails to run with that implementation.

The language of the first form with its use of "be able to" seems
less clear and harder to verify.  The "be able to" language means that
there is no requirement to actually run any of the tests.  Without
doing this, how could it be shown that an implementation conforms to
the testing requirement (or not)?


To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:

Fred Carter / AmberPoint, Inc.

tel:+1.510.433.6525 fax:+1.510.663.6301

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]