RE: [oic] Testing Approach Decision #1: What Classes of Products Are To Be Tested?

["Dennis E. Hamilton" <dennis.hamilton@acm.org>]

I don't doubt you, I just don't know where you are reading this.

Where is that text?  It mentions W3C.

Also, where is the ISO language, if you don't mind.  

I do want to be consistent, and in that case I seek a neutral term that
relates to what I am talking about, whether or not it becomes related to the
interoperability understanding of OIC TC.

 - Dennis

PS: Producers and consumers are implemented of course.  In terms of
interoperability, especially because of the low floor of ODF, it would seem
that we want to attend to the need of a particular category of producers and
consumers of ODF documents.  I still think we need to be clear about that in
assessing the state of interoperability, for example.

PPS: Also, because expression of a feature in terms of how a processor makes
the document human usable is going to require some sort of exemplars
(clearly not normative but some how suggestive of what it means to support a
feature), in the case of office applications and ODF processors, how can we
avoid distinguishing these categories and indicating the ones we don't have
our primary attention on?  Is that cycle of learning and improvement
entirely between implementation producers and their clientele, and we deal
exclusively with some abstraction that user interests will not relate to
very well?  [This is an acutely personal question for me, but I do think we
need to be clear about it among ourselves.]

-----Original Message-----
From: robert_weir@us.ibm.com [mailto:robert_weir@us.ibm.com] 
Sent: Thursday, January 22, 2009 16:13
To: oic@lists.oasis-open.org
Subject: RE: [oic] Testing Approach Decision #1: What Classes of Products
Are To Be Tested?

"Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 01/22/2009 
04:56:05 PM:

 
> I didn't understand "class of product" in the sense of "class of
> conformance" or "conformance target" at all, if that is what you are 
driving
> at.
> 
> I was going by what ODF has been offered for and what it claims that 
target
> to be and where one would expect interoperability. 
> 

I'm not making this up.  Look at "Variability in Specification":


"Classes of products
From this categorization of specifications, a Working Group can identify 
the class of products that are affected by the specification. Classes of 
products can be generalized as either producers or consumers, or as 
content itself.
For example, identifying which are producers and consumers is clear for a 
protocol-type specification: the two parties to the dialog are the 
targets. For a processor-type specification, the processor is the consumer 
of an XML vocabulary defined in the specification. For content-type 
specifications, there may be one or more consumers that take the content 
and "play" or "read" it in some way.
The following is a list of the most common classes of products for W3C 
specifications:
Content (of type, meaning, and format as defined in the specification)
Producer of content (may be divided into initiators and modifiers)
Player (read-only consumer, conveys content in non- XML way)
Consumer in a one-way pipeline
Responding agent (e.g., server) of API (consumer and producer)
Processor (consumer of its vocabulary/instructions)
Module that hosts the processor
Producer of instructions/commands to processor
Profile derived from the specification's Rules for Profiles
Specification (guidelines)
This list does not exhaust all possibilities. Specifications may have to 
define their own classes of product if none of these fits."

So I think we go in the wrong direction if we equate "product" with 
"implementation".  Formally, a product is the subject of the 
specification, the thing that conformance is being defined for.  ISO 
nomenclature is similar.  There standards specify the conformance of 
products, processes or services.

> I am not clear what you are objecting too.  I do not mean to suggest 
that
> there is anything different with regard to the conformance of the 
documents
> they process and the conformance of that processing.  I do think, in 
terms
> of the state of interoperability, the practical concerns are specific to
> document types and these classes of product.  I saw this as part of 
focusing
> on where interoperability concerns exist.
> 

I just want to make sure we're not making distinctions without 
differences.  For example, I do not see how ODF 1.1 would allow us to 
craft a conformance test for an ODF spreadsheet document that would be 
different in any way from a conformance test for a word processor template 
file.  The standard itself does not define these as different products, or 
at least does not give them different conformance requirements.

> We should certainly clarify what we mean by "class of product" and 
whether
> we will have more than one sense of that (and find a way to be clear 
which
> one we intend in what context).  I can't imagine not distinguishing 
classes
> of products in the world in terms of where the focus is (or is desired) 
for
> interoperability.  I thought that was how we wanted to guide our focus 
on
> conformance details.
> 

I'd recommend using the use in "Variability in Specification".

> Furthermore, I don't understand your objection to the recognized and
> required document types that ODF defines.  Those distinctions are wired 
into
> the ODF 1.0/1.1 specifications, as well as I can tell, and I know of 
nothing
> in the ODF 1.2 work that is changing that.  I don't know whether 
OpenFormula
> introduces any distinct conformance requirements, but in the case of ODF 
1.1
> I agree that the conformance requirements are the same for all and any 
of
> the recognized document types.  I'm not so sure about .oth and some 
others
> that either receive no mention at all in the normative body of the ODF 
1.1
> specification, or are mentioned and left largely unspecified.
> 

Again, I'm just trying to avoid making fine distinctions that ultimately 
will not be testable.  We can certainly keep the distinctions for now, if 
you prefer, but since there is no underlying difference in conformance, at 
least not in ODF 1.1, we would only be labeling identical product classes.

Of course, anyone could prove me wrong by giving a single counter example 
of a conformance test assertion that would distinguish, say a conformant 
ODF spreadsheet document from a conformant ODF word processor template.


>  - Dennis
> 
> PS: I continue to separate application from the software that may be an
> instrument of that application.  (Or, put another way, for me 
application
> software is not the application, it is software intended to support a 
class
> of application.)  I shall never give up on keeping the two separate.  If 
you
> require different terms for maintaining the distinction, let me know.  I
> think the distinction is crucial. 
> 

I think most members will equate application and implementation, since 
that is the common usage, but I do understand the difference you are 
trying to enforce. 

> PPS: I also can be careless in how I use "application."  I'm training 
myself
> to be more careful.
> 
>    -   -   -   -   -   -   -   -   -
> 
> A. WHY I THINK OF THOSE CLASSES OF PRODUCTS (BOTH FORMATS AND SOFTWARE)
> 
> 1. It is the Open Document Format for Office Applications
> 
> 2. In section 1.1 (ODF 1.1),
> 
> This document defines an XML schema for office applications and its
> semantics. The schema is suitable for office documents, including text
> documents, spreadsheets, charts and graphical documents like drawings or
> presentations, but is not restricted to these kinds of documents.
> 
> The schema provides for high-level information suitable for editing
> documents. It defines suitable XML structures for office documents and 
is
> friendly to transformations using XSLT or similar XML-based tools. 
> 
> 3. From the Charter of the ODF TC, as last amended,
> 
> 3.1 The purpose of this TC is to create an open, XML-based file format
> specification for office applications.
> 
> The resulting file format must meet the following requirements:
> 
>   1. it must be suitable for office documents containing text, 
spreadsheets,
> charts, and graphical documents, 
> 
>   [ ... ]
> 
> 3.2 Scope
> 
> Since the OpenOffice.org XML format specification was developed to meet
> these criteria and has proven its value in real life, this TC will use 
it as
> the basis for its work.
> 
> A standard for office document processing and interchange will be of 
great
> utility to many users and software companies developing applications, 
and
> should be made available as soon as possible.
> 
>    -   -   -   -   -   -   -   -
> 
> I understand that the characteristics of such products we are concerned
> about is their support for ODF documents on behalf of office 
applications
> (which for me are what office workers are concerned with and use those
> products for in their work).
> 
> I do confess to looking at interoperability as having to be situated, 
even
> as we speak of general office-document creation, collaboration, and
> interchange.
> 
> Maybe we should have Marston's feedback on what he had in mind for 
Classes
> of Product, because I can't tell anything specific from my notes or the
> slides to suggest he was talking about Conformance Targets.
> 
> And, either way, I do think we need to have some sort of clarification 
like
> this with regard to the state of interoperability.
> 
> 
> B. ABOUT THE DISTINCT ODF DOCUMENT TYPES
> 
> Finally, while I agree there is a single specification for several 
document
> types, there are indeed distinct document types, called out in the
> specification.  Section 2.3 Body Elements and Document Types is pretty 
clear
> about this. 
> 
> I am referring to those types, which can occur independently and be
> processed independently.  So there are indeed conformant ODF text 
documents
> and ODF spreadsheet documents as well-defined divisions of conformant 
ODF
> document, driven by the correspondence of MIME type and the required 
single
> element of the required <office:body> element within content.xml. 
> 
> Not only is it the practice to speak of these types separately, but it 
is
> also commonplace that they have separate components for their processing
> when integrated in an office suite.  It is also common practice in 
office
> applications that one must commit oneself to a type of document at the
> commencement of document creation (with some exceptions for templates 
and
> master text documents), and the current division of document types 
requires
> such a decision to be made too.
> 
> I am recognizing that characteristic of ODF 1.1 and how it is 
encountered in
> practice.  What's wrong with that?  Won't each of the majority of test
> documents necessarily be one of the main three types?
> 
>  - Dennis
> 
> 
> -----Original Message-----
> From: robert_weir@us.ibm.com [mailto:robert_weir@us.ibm.com] 
> http://lists.oasis-open.org/archives/oic/200901/msg00054.html
> Sent: Thursday, January 22, 2009 12:10
> To: oic@lists.oasis-open.org
> Subject: Re: [oic] Testing Approach Decision #1: What Classes of 
Products
> Are To Be Tested?
> 
> "Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 01/21/2009 
> 04:46:58 PM:
> http://lists.oasis-open.org/archives/oic/200901/msg00048.html
> > 
> > Here is a straw man for the first of the "First Wave of Decisions" 
item:
> > What Classes of Products are to be Tested?
> > David Marston's list is at 
> > <
> 
http://wiki.oasis-open.org/oic/TestingThoughts1/800_The_First_Wave_of_Decisi

> ons>.
> > 
> 
> I'm taking "products" in the specific sense discussed in "Variability in 

> Specification".  In that sense, ODF 1.1 (and ODF 1.0) defines three 
> specific products:
> 
> 1) Conforming documents
> 2) Conforming applications
> 
> Applications are further split into:
> 
> 2a) Those that read or write ODF
> 2b) Those that both read and write ODF
> 
> Although it may be natural to suggest other products, such as 
spreadsheets 
> versus word processors, these are not really "products" in the 
conformance 
> sense.  There is no definition of a "conformant ODF spreadsheet" that 
> differs from a "conformant ODF application" in general. 
> 
> That said there are certainly a range of implementations that we might 
> test against these product types, and some implementations may be 
> spreadsheets while others may be word processors.  But in the 
conformance 
> sense there is no such distinction in ODF 1.1.
> 
> But I may be entirely wrong in the above analysis.  To show that I am 
> wrong, you will need to show me where conformance is defined different 
for 
> each product where you suggest a different product exists.  For example, 
I 
> may be wrong if we can show that there is conformance for an "ODF 
Package" 
> that is distinct from an "ODF Document", or that an "ODF Document 
> Template" has distinct conformance requirements from an "ODF Document".
> 
> Also, the above is just with respect to conformance.  For 
interoperability 
> tests we do not need to be as strict with the products defined by the 
> standard.  We have the freedom to also consider useful distinctions that 

> exist in the real world, and this could easily include "products" such 
as 
> spreadsheet documents, etc.
> 
> -Rob
> 
> 
> [ ... ]
> 


---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php