Re: [oiic-formation-discuss] Interoperability versus Conformity

[robert_weir@us.ibm.com]

"Dave Pawson" <dave.pawson@gmail.com>
wrote on 06/09/2008 03:29:16 AM:

> 2008/6/8  <robert_weir@us.ibm.com>:
> >
> > "Dave Pawson" <dave.pawson@gmail.com>
> 
> >> The relationship between conformity and interop has me confused.
> >> I don't understand that.
> >>
> >
> > Excellent point.  Let me state my interpretation.  And
anyone let me know if
> > they think this contradicts how they think of the terms.
> >
> > Conformity is the relationship between a technological artifact
and the
> > standard or standards that defines that technology.  So
in our present case,
> > conformity is the relationship between an ODF document, or an
application
> > that produces or consumers an ODF document, and the ODF standard.
 The
> > artifact is conformant with the standard when it implements all
required
> > provisions of the standard, and implements none of the prohibited
provisions
> > of the standard.
> 
> A nuance there I think we need to clarify.
> 
> an ODF document, or an application
> that produces or consumers an ODF document,
> 
> Which? IMHO ODF specifies what a document instance should be/do.
> It says nothing about producers or consumers?
> Which are we testing.
> 

Actually, ODF defines conformance classes for both
documents and applications, though it is certainly far more detailed on
document conformance.

See for example, ODF 1.0, section 1.5.  It isn't
deep, but it is testable, or at least a claim of ODF application conformance
is falsifiable.

There are also hundreds of items that are simply stated,
but not with the operative "shall" or "should" language.
 For example, page headers can display the current page number.  Very
basic functionality.  ODF 1.0 defines this in 6.2.3 as:

"Page
number fields display the current page number. These fields are particularly
useful in headers and footers. E.g., if a page number field is inserted
into a footer, the current page number is displayed on every page on which
the footer appears."

Now maybe we should have phrased it as, "Conformant
ODF consumers which implement the page number field defined in this section
shall render <text:page-number>
with the current page number", or something like that.  The way
we have it stated in ODF 1.0 is strictly speaking a definition, and not
an implementation requirement.   But even without the control language,
the meaning is clear, unambiguous, and it is reasonable for a test suite
(interoperability not conformity) to give a warning if an implementation
rendered <text:page-number>
as (page-number-3)/2 rather than as the current page number.


> Another refinement Rob. "A" Document? I guess we could produce
a document
> that minimally is valid to one part of the standard.
> 

Yes.  I think these "atomic" tests
that exercise features in isolation can help isolate problems.  Of
course, we still need features used on combination, since there are unique
defects that can be exhibited only under those circumstances.

> How do we say that (a suite of) documents from vendor X complies
> with all the aspects of the standard that it uses/should be compliant
with?
> See the point? Vendor X may only implement a word processor.
> Vendor Y implements all 3. We need to address this 'scope' issue?

There is some guidance in ODF 1.0, Appendix D "Core
Feature Sets" which defines which features are applicable to text,
spreadsheets, drawings, presentations charts and images.  But it is
a fair point.  And ODF is not limited to those convention uses either.
 

One thing ODF does that makes our life a little easier
is that it reuses the same markup across application types for the basic
building blocks.  So font styles, tables, etc., are represented in
XML the same in word processor versus spreadsheets, etc.  So I would
imagine that a test suite for text styles would be defined once in the
abstract and then "inserted" into shell ODT/ODS/ODP documents
via a script, to make versions appropriate to ODF word processors, spreadsheets
and presentations.

> 
> >
> > Conformity can be stated as black or white :  "Application
X conforms,
> > Document Y does not conform" or as partial conformance,
"Application Z
> > conforms to parts 1 2 and 3, except for Part 3, clause 26 and
29."
> 
> And, possible, Test groups 123-148 not run, hence untested.
> (E.g. where a vendor doesn't implement a presentation app)
> 
> 
> >
> > Interoperability, on the other hand, is the relationship between
two or more
> > technological artifacts that implement the same protocol or protocols.
 I
> > can't give you a crisp black and white definition here.  But
I can suggest
> > some analogies.
> 
> <grin/> Just what has Rob swallowed!
> "technological artifacts that implement the same protocol"
> Agreed we need the terms for a spec, I'd prefer plain English for
this group.
> 
>
The nuance I was trying to suggest is that interoperability
is between two or more implementations.  Maybe applications, services,
command line tools, etc.  This isn't an exclusive definition, but
the one that comes to mind.  You can also talk about interoperability
between two markup standards that stand in a relationship with each other,
but that is a more abstract concept and we don't need to go there.

> For our case, I'm interpreting this as 'Two word
processor instances created
> by different vendors'. Is that right?
> 

That would be the main case, yes.  But I believe
that two word processors created by the same vendor, say version N and
version N+1, could be tested with regards to interoperability without abusing
the definition.

> 
> >
> > First, consider the C/C++ programming languages.  Both define
formal
> > provisions, and a compiler implementation, or a program file,
could be
> > tested as to whether it conforms to the underlying programming
language
> > standards.  However, this does not guarantee that two conformant
C++
> > compilers will create programs that yield the same runtime results.
 This is
> > because the C/C++ standards have items that are undefined and
> > implementation-defined, like the size of an integer, or the sign
of of a
> > character, etc.  This is well-known to practitioners --
they know where the
> > bodies are buried -- and there are a variety of practices which
they know to
> > institute if they want to create interoperable C/C++ code (or
portable code
> > as it is more often termed in this space).
> 
> Scary but understood. Thanks.
> 
> 
> >
> > Further, a more mature expression of these interoperability constraints
(and
> > that is what they really are -- additional constraints beyond
the C/C++
> > standards)can be written up in detail and agreed to by a set
of vendors,
> > becoming a standard that defines conformance of "portable
C/C++" within a
> > particular domain.  For example, Embedded C++ took that
route, as a proper
> > subset of ISO C++.  PDF/A did that as well, a constrained
subset of PDF to
> > increase interoperability in a particular domain.
> 
> No, you've lost me. Is this the two C++ compilers *without* the bodies
> I.e. the subsets that produce identical results, or the 'list of bodies'
> 

I am saying this:  the C++ standard had permitted
implementation-specific behavior.  This was allowed variability in
implementations conformant to the same standard.  In order to increase
interoperability among a specialized set of implementations (embedded devices)a
number of vendors in this area defined a set of additional constraints
on C++ that reduced the variability allowed in implementations of Embedded
C++.  So at that point you have 2 standards and 2 sets of implementations,
and interoperability looks like this:

a) among C++ implementations, consistency of runtime
results of the same C++ code is not the guaranteed , due to the allowed
variability in runtime behaviors.
b) among Emebdded C++ implementations, consistency
of runtime results is more prevelant, since they agreed to reduce variability
by adopting additional constraints.
c) All conformant Embedded C++ programs are also conformant
C++ programs since Embedded C++ added only constraints
d) Not all conformant C++ programs are also conformant
Embedded C++ programs for that same reason.

I'm not saying that profiles like this solve all our
problems.  There is likely a bunch of good we can do without touching
a profile.  But it is an arrow in our quiver, and we should keep it
in mind places where it might be appropriate.
 
> >
> > So "interoperability" in the large is not something
we'd just wantto go out
> > and start testing.  But we could define, for example, a
proper subset of ODF
> > for browser-based implementations, which contained only rendering
primitives
> > that could be losslessly mapped to the HTML/CSS2 model.  I
could also
> > imagine a profile of ODF for desktop use.
> 
> Wow. This is miles different from my own assumptions.
> This sounds like
> 
> "The subset of the Word processor parts of the standard
> that should be common between any two implementations"
> 
> Is that right? Especially the 'should be', i.e. we go hunting
> the mandatory bits, then move out towards the 'hard to spec'
> bits, determining what is a reasonable stop point?
> E.g. Omit  pixel perfect visual output of an example line of
text?
> 

Whether it is "should" or "shall"
is a decision we would need to make, if we define a profile.  Remember,
none of this changes conformance at the level of any of the ODF standards.
 But a profile is a standard, and can define its own conformance.
 So we could have a profile called "A Profile for web-based rendering
of ODF documents" that defines conformance to that profile in terms
of additional constraints on ODF application behavior.  This might
include subsetting the elements and attributes in the document conformant
to that profile.  This then opens the door to having a conformity
assessment definition that tests these additional constraints.

It really depends on the interest level.  If
we had 2 or 3 vendors or projects who were dealing with web rendering of
ODF documents, and wanted to negotiated a higher level of interoperability
among their implementations, then I think it would be reasonable to accomplish
this in the ODF IIC TC.

> I do like the idea of profiles though.
> The fluffy definition might be "The parts of the standard that
> a  user would reasonably expect to produce identical output"
> in profile X for application Y. "
> 
> 

I'm not sure we're using the word "profile"
the same way.   XHTML Basic is what I'd called a "profile"
for example (http://www.w3.org/TR/xhtml-basic/).
 A document set of constraints on an existing standard intending to
increase interoperability in a particular domain.

You can also have profiles that add functionality.
 I don't want to deny that they exist, but they can cause as many
interop problems as they solve, so I am not talking about them.

> 
> >
> > But I don't think we need to go that far initially.  We
would make progress
> > even with a check list of ODF features, set out in a table, and
some simple
> > tests cases that allowed an implementation to verify whether
or not the
> > features are even implemented.  Even doing this much would
raise awareness
> > of missing functionality, and when that is addressed interoperability
> > increases.
> 
> A long slog, but doable. It would highlight weaknesses in the standard
too.
> The other key aspect would be an inter-dependence table?
> This feature depends on that feature|para|clause whatever.
> I see this as the pre-conditions for a test etc.
> Equally I can't test this feature unless I have passed this list
> of tests.
> 
> 

This all makes good sense.  

> 
> >
> > Note also something unintuitive -- a high degree of interoperability
is
> > possible even without conformity.  I know this may sound
like sacrilege, but
> > a look at the web itself, where only a small fraction of web
pagesare valid
> > HTML or XHTML.
> 
> 
> Yes, I can see that :-)
> 
> Thanks Rob. Enlightening.
> 
> 
> regards
> 
> 
> -- 
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: oiic-formation-discuss-
> unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: oiic-formation-discuss-
> help@lists.oasis-open.org
>