Our Position on the Conformance Proposal

[Stephen Peront <stepper@microsoft.com>]

Hello Michael/Rob and TC Members/Observers...

Several TC members have asked us to clarify our vote against the most recent committee draft (CD).  Let me note at the outset that the sole reason for our "no" vote was the conformance proposal that had not yet been voted on within the TC.  I will therefore describe our general view of conformance, explain our disagreement with the current proposal, and recommend an alternative approach. 


Conformance Principles.  We believe the following principles should guide document format standards activities in general, and ODF activities in particular:

- Conformance should be defined in terms of the minimum requirements; it should not be used to establish an "upper limit" for implementations.

- Extensibility mechanisms are useful for all file formats.  Extensibility is central to XML, of course, and it's useful in the context of a file format for all of the same reasons that it's useful in other XML applications and vocabularies.

- The ability to add custom semantics to documents, and in particular custom semantics that are not broadly applicable (and are therefore unlikely to ever be formalized as an international standard), is critical to many document users.  This is especially true for organizations that are automating large-scale workflows and business processes.  Such capabilities should be as simple and straightforward as possible for implementers because they are typically used in rapidly changing competitive environments where needless complexity would slow the pace of innovation.  It should also be possible to validate document content against the standard without a need to understand the custom semantics that may be used in the document.  We believe these types of extension points are a "must-have" feature for modern XML-based document formats because they enforce a standardized approach to custom semantics.

- We have found many interoperability problems between existing ODF implementations in our testing, and for the most part these have not been caused by the use of foreign elements.  Rather, the problems we've found are caused by other issues such as variation in interpretation of the standard, unsupported features in some implementations, product bugs, and other undocumented details of various implementations.  If the goal is to improve real-world interoperability, we feel implementers can make a big difference by documenting these sorts of details, and the removal of foreign element support from ODF won't do anything to address the vast majority of interoperability problems we've encountered.


The Current Proposal.  Looking now at the specific conformance clause that appears in the latest CD, the aspect of it that we find most problematic is the distinction between "Conforming OpenDocument Documents" and "Conforming OpenDocument Extended" documents.  This distinction implies that it is not possible to allow for standards-based extensibility, which is not true.  ODF has historically allowed standards-based extensibility (through use of language similar to the current proposal's "extended" class), and so does IS29500 (through the semantics of the customXml element, as well as the Markup Compatibility and Extensibility mechanisms defined in Part 3 of IS29500).

The distinction between "conforming" and "extended" documents would also create unrealistic expectations because of the misleading implication that "conforming" documents are more interoperable than "extended" documents.  As I mentioned above, the vast majority of real-world ODF interoperability problems we've seen have nothing to do with this distinction.


Next Steps.  There are two possible solutions that would address our concerns:

1) Eliminate the "conforming" class altogether, and use the existing definition of the "extended" class (for producers and documents) as the one and only conformance class/level.  This would mean that implementers would continue to have the same extensibility options that they have  had in prior versions of ODF.

2) Add some other extensibility mechanism, such as those used in IS29500, which would allow for standards-based validation of document content that may contain custom semantics from non-standardized namespaces.  This would require some engineering work within the TC, and for that reason we are slightly inclined toward option #1 above, but we would be glad to contribute to this work if the TC would like to explore other extensibility mechanisms.


One question that has arisen in our discussions to date is "why not use RDF for custom semantics in documents?"  Although it is theoretically possible to use RDF for document semantics, there is a great deal more complexity in that approach than in the simple straightforward use of foreign elements (as in ODF 1.1) or customXml tagging (as in IS29500).  This complexity grows rapidly if there are nested structures, which are typical in these scenarios.  If we are going to require implementers to re-design their extension points, we feel the TC should work to provide a more straightforward mechanism.


A final point I'd like to reiterate is that our implementation of ODF 1.1 in Office 2007 SP2 does not use foreign elements at all.  We have not extended the spec in any way.  Others have extended the ODF spec, however, including many smaller implementers who have at most one vote in this process, and in some cases no votes.  We believe it is inappropriate to force all implementers of ODF to submit every potential extension to the TC for consideration in a future version of ODF.  Consider, for example, an implementer who had an idea for an extension in 2006 - they would still be waiting today, in 2009, for the next version of the spec which might contain that extension.  And worse yet, there is little chance that a small implementer could get any extension into a future version of ODF without support from  the larger implementers, as a simple analysis of last week's voting on the CD would show.


I hope this helps clarify our voting on the CD last week.  Please let me know if you have any questions, and I look forward to working with the TC on finishing up ODF 1.2.


Rgds,
-Stephen