Conformance, Specification Quality and OASIS Recommendations

[robert_weir@us.ibm.com]

Here is my contribution for discussion under the conformance clauses 
follow-up topic on Monday.

OASIS Technical Committee Process says the following with respect to 
conformance:

"A specification that is approved by the TC at the Public Review Draft, 
Committee Specification or OASIS Standard level must include a separate 
section, listing a set of numbered conformance clauses, to which any 
implementation of the specification must adhere in order to claim 
conformance to the specification (or any optional portion thereof)."

There is additional guidance provided in the OASIS Guidelines Writing 
Conformance Clauses (
http://docs.oasis-open.org/templates/TCHandbook/ConformanceGuidelines.html
)

I've gone through these guidelines and compared them to what we currently 
have written in CD 01. 

I have the following observations on how we can tighten up the good start 
that we have:


1)  CD 01 Section 1.2 says, "Within this specification, the key words "
shall", "shall not", "should", "should not" and "may" are
to be interpreted as described in Annex H of [ISO/IEC Directives] if they 
appear in bold letters."  I think the " bold letters" restriction is a 
carry over from our earlier IETF usage.  This is not typical in ISO 
standards.  If we have this clause, as written, it will immediately lead 
the reviewer to ask, "What does 'shall' or 'should not' mean when not in 
bold letters'?"  I think we would be much better off if we removed this 
quirky convention.  All uses of shall/should/may etc. would then be 
expressing provisions, when used in normative clauses.  Of course, if 
there are any uses of these terms where we do not wish to state a 
provision then we should chose an alternative phrasing.

2) In the same section, 1.2 "Notation" we should also explain the textual 
and typographical convention used for denoting quoted element names, 
attribute names and attribute values.  I've seen one reviewer confused by 
this, so it is worth stating the convention (monospace) explicitly.

3) In section 1.5, "Document Processing and Conformance", the document 
processing piece is out of place.  Obviously we may have document 
processing conformance requirements, but the OASIS guidelines suggest that 
these be stated under the appropriate  conformance "target", i.e., 
Consumer or Producer, and not set into its own clause.  In other words, if 
we put Document Processing into its own section, then we can point to it 
from the Conformance clause. 

4). The heading of 1.4 are not consistent.  In some cases the top level 
headers, like 1.4.4 "Consumer Conformance" indicate a conformance clause. 
But in other cases, like 1.4.3 "Producer Conformance" no provisions are 
expressed.  1.4.3 is merely used to group 1.4.3.1 and 1.4.3.2.  Maybe this 
is OK?

5) I recommend that section 1.4 be called "Conformance". If we want to 
have a separate section on "Document Processing" then that is OK.  If we 
want to specify document processing provisions in section 1.4, then that 
is OK.  But we should have 1.4 be clearly and unambiguously be called 
"Conformance".

6) 1.4.2.1 is either circular or insufficiently defined  We have now "A 
conforming OpenDocument document shall adhere to the specification 
described in this document...".  But what does it mean to "adhere"?  If 
this is the same as to conform, then this is circular.  If it means 
something else, then it is undefined.  So the question is:  Did we mean to 
require something here?  If so, what?

7) 1.4.2 "If the document is an OpenDocument package..."  this is circular 
as well, since we don't define "OpenDocument package" except in Part 3. So 
we're really saying "if the document is A, then it shall conform with A". 
To me it feels like there are two conformance targets masquerading as one 
here: Packaged ODF Document and Single XML File ODF Document.  If we 
define these each as their own conformance target, then we can easily 
combine them and say "A conforming OpenDocument document is either a 
Packaged ODF Document or a Single XML File ODF Document."  In other words, 
build the definition up from the individual targets.  Otherwise we get 
these circular problems.

8) I'm not sure I like "conforming" to be part of the name of a 
conformance target or conformance class.  If the conformance target is 
called "conforming extended opendocument document" then what do you call 
it when such a document does not conform?  For example, it has an error in 
the core part of the schema and is not valid.  Is it then a "nonconforming 
conforming extended opendocument document"? So I would not have 
"conforming" be part of the name of any conformance target or class.

9) We mention "schema" and "strict schema" in the conformance clause, but 
we don't really connect to it. In many places of the text we refer to "the 
schema defined by this specification".  We need to be precise here.  I 
think upfront, before we even get to the conformance clause we need to 
state something like: "This Part consists of a specification and an 
associated schema definition file.  The schema definition file is named 
odf12-cd01.rng". Otherwise the reader is lead to believe that the schema 
is in the specification.  In fact we should probably be clearer

10)  1.4.3.1 "It shall not produce any non-conforming OpenDocument 
document of any kind" contradicts the next  line "It may produce 
conforming OpenDocument extended documents".  A conforming extended 
document is not at the same time a conforming non-extended document, so 
obviously if a producer may produce an extended document, it is producing 
something that is a non-conforming non-extended document.  We can probably 
clean this up if have documents declare, with an attribute on the root of 
context.xml or via some similar mechanism, which conformance class they 
belong to.  That way 1.4.3.1 can be a simple consistency test.

11) 1.4.3.1 says "It shall not produce any non-conforming OpenDocument 
document of any kind" while 1.4.3.2 says "It shall not intentionally 
create any non-conforming OpenDocument extended documents of any kind.". I 
don't think we can test "intent" so we should probably take out that 
qualification.

12) 1.4.4 "parse and interpret" -- I'd like to see if we can make a more 
testable requirement here.  It may not be possible, since a Consumer 
ranges from a full text editor down to possibly a Zip program or an XML 
parser.  Our practical problem is that we have not defined a minimal set 
of elements which a Consumer must understand.   In the general case, this 
is probably right.  But we might consider also defining conformance 
targets according to Appendix D.  So define an "ODF Text Consumer", "ODF 
Spreadsheet Consumer", etc. and require interpretation of corresponding 
ODF features.   I think this would be a tremendous gain for 
interoperability.


-Rob