Re: [oiic-formation-discuss] Caution and Disclaimer on Interoperability

[jose lorenzo <hozelda@yahoo.com>]

--- On Thu, 6/12/08, Dave Pawson <dave.pawson@gmail.com> wrote:

> > I make this last statement because I truly doubt that
> > ODF will ever be tied down enough to prevent one
> > application, designated as "interoperable"
> according
> > to the ODF standard, from arbitrarily inserting binary
> > blobs (or an equivalent mapping into printable
> > characters, CDATA, PCDATA, etc) into the document as a
> > way to store arbitrary proprietary content, arbitrary
> > proprietary application or platform state, arbitrary
> > proprietary semantics, etc, bypassing the preferred
> > ODF structures (if there even exist any in the
> > particular case).
> 
> We haven't addressed implementation extensions in
> any detail as yet. Your point is valid, though an example
> or two would help me understand it better.

I flipped through http://docs.oasis-open.org/office/v1.1/OS/OpenDocument-v1.1-html/OpenDocument-v1.1.html some, but it would take me some time to learn the details of relax-ng and to read every word of ODF carefully.

What I will do is to describe informally an example of a document that I think would be strictly conformant (again, I am taking an educational guess), yet would hold a ton of proprietary material from a single vendor. I'll also overview the process undertaken by the proprietary app that made the document as it newly opens and handles same document. I'll contrast this behavior with what competing apps would be able to do.

Let's start by loosely describing the contents of our hypothetical ODF document. Let's say we have all the requisite outer xml that is needed for a simple one page one paragraph text document (to make things simple, assume the office:document root element, so this means the entire document is contained right here in a single file).

Parsing the document, we come to the single paragraph element (text:p I believe is the element name/type), but first I have a question.. or rather, I'll state an assumption. I am not sure lexically what can be included inside a paragraph element, but I will assume (at least) that anything that is printable and basically does not look like this "<" "/" "t" "e" "x" "t" ":" "p" ">" is fair game. Let me know if I am wrong, so I can possibly adjust the example if you need me to.

What we find as the entire contents of this paragraph element is as follows (excluding the beginning and ending quotes):
"~##345435e7aa90430bf66a#90485209345034523845084396809348758309486093 285348609743584309689850938903670934590683409683789708870980930498523 4534544444444445829384098550809980982-3734047057086082347509709283948 957720374096903845073049582098430999898390853094859723487502830948592 837450230946893475743085794385938250909399485983094859439582034095834 850923458348528357435720384509834967309485903845098430957043758438687 582717513451345034537384509183645764359722848571279321723933232347878 383810923878387489783292929298999477983798178927308973877419273864762 374515263473746736433339`12387467834872364341364786128643264875182387 4678263754876237663763782838738764872514523456235478236476`8543876237 268743653864873627814687364997899738748374617364782374182639478374817 374823746297483744791q723847837489"

OK. The above literal text would likely be spit out verbatim in some way by most or all odf comformant reader-renderers (assuming we're displaying for a sighted person).

But MyClosedApp created this ODF document and does something VERY different than other apps in order to present this seemingly almost empty document to its user.

It chugs along for a few seconds and then...

..It displays/handles this document exactly as what is expected by the document's author who used a licensed copy of MyClosedApp to create it. "This" document has several spreadsheets, movies and graphics, and two scripts that get run automatically at startup.

Surprise!

One script schedules an alarm to ring in an hour. The other sends a signal to an online app server to carry on some processing to produce a report for the day's updated content to a particular website owned by the author. If possible, as per the behavior encoded in this second script, that online generated report will be downloaded behind the scenes if it gets finished while this user is still online with this document opened. He would at that time be prompted to have it be displayed and saved within the document.

Wow! Talk about having an extension, lock-in, proprietary secret sauces.. you name it!

Yet all from a strictly conformant app and document!

[Now for the partial explanation of the magic (aka, the super secret proprietary voodoo):]

See, those numbers (and occasional backtick, hyphen, or space) above was an encoding that MyClosedApp could unscramble after generating a key from a hash of the document plus various other specific tidbits.

The "~##345435e7aa90430bf66a#" within the paragraph element cued MyClosedApp to this and even gave it the information needed in order to know how to unscramble the passage and even where the passage would be and if further processing would be done (in this case the passage was encrypted using closed proprietary secret system X and was to be found immediately following the last # symbol through to the end of the paragraph element). 

[Notice that there is a bunch of proprietary semantics here related to the code "~##345435e7aa90430bf66a#". None of it is part of ODF, but who is supposed to catch this? Are the semantics of a paragraph element precisely, completely, and unambiguously defined? Assuming for a minute that this app would not be strictly conforming, who would ever be able to tell? What test suite would catch this? There was no subtlety in this example as I am sure would exist in practice. In practice it would be difficult to identify "funny" sections of a document. .. Let's continue..]

After all pre-processing, MyClosedApp got this: "mcatp://somedomain.com/somedocument.secret" .

That's it, but under the proprietary context MyClosedApp derived, it knows what it has to do with that url.

What happened was that a user, the "author", made the document (forget for a moment how and how much was autogenerated or anything like that.. those are more secret sauces), and had chosen, in this particular case, to "save to my online account 344234234". Thus the actual document was actually online and was accessible through that url once the online "mcatp" server received some authentication information from MyClosedApp as part of the secret closed protocol "mcatp" (the secret authentication bits for this mcatp session was derived from the encrypted content quoted above and some other stuff from the document). [Yes, mcatp is fictional.]

Voila! I have a strictly conforming document that can be rendered usefully only on a single vendor's product.

MyClosedApp produces only similarly strictly conforming documents and does everything else by the book, so I am guessing (not having read the ODF standard yet) that MyClosedApp would qualify as a strictly conforming application.

The document was quite small, in fact, but secretly packed a truckload.

MyClosedApp sees this wonderfully complex "document", but no other app even has a clue. It's too bad, because the actual document was nice but contained the usuals: not overly fancy spreadsheet tables, some graphics and a few movie clips, etc. In other words, there was no reason (bar the secret sauce potentially stegonagraphic scrambling of MyClosedApp) that such a document was not able to be presented/handled in all its glory through any other ODF strictly conformant app. [Pretend, for the sake of getting this example over and done with, that "movie clips" are actually a slide show or something similar that ODF can handle if ODF doesn't actually handle "movie clips".]

If I didn't overlook something critical, it should be clear that you can hide anything at all into a strictly conforming ODF document.

[I used the online storage scenario so as to provide a fancy example and also to hint at a potential future technique that might be useful for covering up documents while frustrating reverse engineering efforts. The whole actual document material needed by MyClosedApp could have been hidden within the local document body.]

There were no "ODF extensions" inside the hypothetical document described above. I repeat, there were no extensions being used (unless I made a mistake skimming the standard or in my assumption about what can go inside a paragraph element). I simply masqueraded what was effectively a secret proprietary extension, as ordinary, legal, paragraph content data [And I imagine there are a ton of places besides text:p where I could have inserted the actual secret data.]

MyClosedApp, a strictly conforming ODF consumer/producer, will NEVER interoperate with any other similarly strictly conforming ODF c/p except at the most superficial level. So we have a virtually totally useless document producer (but "sanctioned" by OASIS). It's useless unless you are happy to be able to play around with xlst transformations of a few lines of XML.

Note that MyClosedApp can read everyone else's docs (by assumption) and process it normally, but it's a one-way ride. Possibly, every existing conformant ODF document that is opened and saved (over itself) from MyClosedApp will cease to ever be a useful ODF document when handled by any other app on the market.

But cheer up. Though transformed (shortened locally while partially stored online), it will still be a strictly conformant ODF document. [smiles]

Example finished.

I think this is a pretty important message just illustrated. I doubt most laypeople, when they talk about ODF compliance, realize this scenario is possible and actually quite possibly likely to happen at some point in the future.

I think this is a pretty important message.