RE: [oic] Testing Approach Decision #1: What Classes of Products Are To BeTested?

[robert_weir@us.ibm.com]

"Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 01/22/2009 
08:03:03 PM:

> 
> I don't doubt you, I just don't know where you are reading this.
> 
> Where is that text?  It mentions W3C.

"Variability in Specifications" is here: 
http://www.w3.org/TR/spec-variability/  Look at section 3 "Classes of 
Products...".

> 
> Also, where is the ISO language, if you don't mind. 
> 

The ISO usage is ubiquitous.  Harder to point to publicly available, since 
most of them are freely available only to ISO participants, but one 
example is ISO Directives, Part 2:  
http://www.iec.ch/tiss/iec/Directives-Part2-Ed5.pdf

> I do want to be consistent, and in that case I seek a neutral term that
> relates to what I am talking about, whether or not it becomes related to 
the
> interoperability understanding of OIC TC.
> 

One idea is to simply move forward with identifying the product classes 
that the standard actually defines conformance for, and then also note 
that there are sub-products implied in the standard for which no 
differentiating conformance requirements are defined, but for which the 
market appears to distinguish.

>  - Dennis
> 
> PS: Producers and consumers are implemented of course.  In terms of
> interoperability, especially because of the low floor of ODF, it would 
seem
> that we want to attend to the need of a particular category of producers 
and
> consumers of ODF documents.  I still think we need to be clear about 
that in
> assessing the state of interoperability, for example.
> 
> PPS: Also, because expression of a feature in terms of how a processor 
makes
> the document human usable is going to require some sort of exemplars
> (clearly not normative but some how suggestive of what it means to 
support a
> feature), in the case of office applications and ODF processors, how can 
we
> avoid distinguishing these categories and indicating the ones we don't 
have
> our primary attention on?  Is that cycle of learning and improvement
> entirely between implementation producers and their clientele, and we 
deal
> exclusively with some abstraction that user interests will not relate to
> very well?  [This is an acutely personal question for me, but I do think 
we
> need to be clear about it among ourselves.]
> 

And perhaps we do make that distinction in the interoperability test 
cases.  But I don't think we have a basis, in the standard, at least not 
in ODF 1.1, to make that distinction in conformance tests.

Maybe David's questions each have two answers, one for conformance tests 
and another for interoperability tests?

-Rob


> -----Original Message-----
> From: robert_weir@us.ibm.com [mailto:robert_weir@us.ibm.com] 
> Sent: Thursday, January 22, 2009 16:13
> To: oic@lists.oasis-open.org
> Subject: RE: [oic] Testing Approach Decision #1: What Classes of 
Products
> Are To Be Tested?
> 
> "Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 01/22/2009 
> 04:56:05 PM:
> 
> 
> > I didn't understand "class of product" in the sense of "class of
> > conformance" or "conformance target" at all, if that is what you are 
> driving
> > at.
> > 
> > I was going by what ODF has been offered for and what it claims that 
> target
> > to be and where one would expect interoperability. 
> > 
> 
> I'm not making this up.  Look at "Variability in Specification":
> 
> 
> "Classes of products
> From this categorization of specifications, a Working Group can identify 

> the class of products that are affected by the specification. Classes of 

> products can be generalized as either producers or consumers, or as 
> content itself.
> For example, identifying which are producers and consumers is clear for 
a 
> protocol-type specification: the two parties to the dialog are the 
> targets. For a processor-type specification, the processor is the 
consumer 
> of an XML vocabulary defined in the specification. For content-type 
> specifications, there may be one or more consumers that take the content 

> and "play" or "read" it in some way.
> The following is a list of the most common classes of products for W3C 
> specifications:
> Content (of type, meaning, and format as defined in the specification)
> Producer of content (may be divided into initiators and modifiers)
> Player (read-only consumer, conveys content in non- XML way)
> Consumer in a one-way pipeline
> Responding agent (e.g., server) of API (consumer and producer)
> Processor (consumer of its vocabulary/instructions)
> Module that hosts the processor
> Producer of instructions/commands to processor
> Profile derived from the specification's Rules for Profiles
> Specification (guidelines)
> This list does not exhaust all possibilities. Specifications may have to 

> define their own classes of product if none of these fits."
> 
> So I think we go in the wrong direction if we equate "product" with 
> "implementation".  Formally, a product is the subject of the 
> specification, the thing that conformance is being defined for.  ISO 
> nomenclature is similar.  There standards specify the conformance of 
> products, processes or services.
> 
> > I am not clear what you are objecting too.  I do not mean to suggest 
> that
> > there is anything different with regard to the conformance of the 
> documents
> > they process and the conformance of that processing.  I do think, in 
> terms
> > of the state of interoperability, the practical concerns are specific 
to
> > document types and these classes of product.  I saw this as part of 
> focusing
> > on where interoperability concerns exist.
> > 
> 
> I just want to make sure we're not making distinctions without 
> differences.  For example, I do not see how ODF 1.1 would allow us to 
> craft a conformance test for an ODF spreadsheet document that would be 
> different in any way from a conformance test for a word processor 
template 
> file.  The standard itself does not define these as different products, 
or 
> at least does not give them different conformance requirements.
> 
> > We should certainly clarify what we mean by "class of product" and 
> whether
> > we will have more than one sense of that (and find a way to be clear 
> which
> > one we intend in what context).  I can't imagine not distinguishing 
> classes
> > of products in the world in terms of where the focus is (or is 
desired) 
> for
> > interoperability.  I thought that was how we wanted to guide our focus 

> on
> > conformance details.
> > 
> 
> I'd recommend using the use in "Variability in Specification".
> 
> > Furthermore, I don't understand your objection to the recognized and
> > required document types that ODF defines.  Those distinctions are 
wired 
> into
> > the ODF 1.0/1.1 specifications, as well as I can tell, and I know of 
> nothing
> > in the ODF 1.2 work that is changing that.  I don't know whether 
> OpenFormula
> > introduces any distinct conformance requirements, but in the case of 
ODF 
> 1.1
> > I agree that the conformance requirements are the same for all and any 

> of
> > the recognized document types.  I'm not so sure about .oth and some 
> others
> > that either receive no mention at all in the normative body of the ODF 

> 1.1
> > specification, or are mentioned and left largely unspecified.
> > 
> 
> Again, I'm just trying to avoid making fine distinctions that ultimately 

> will not be testable.  We can certainly keep the distinctions for now, 
if 
> you prefer, but since there is no underlying difference in conformance, 
at 
> least not in ODF 1.1, we would only be labeling identical product 
classes.
> 
> Of course, anyone could prove me wrong by giving a single counter 
example 
> of a conformance test assertion that would distinguish, say a conformant 

> ODF spreadsheet document from a conformant ODF word processor template.
> 
> 
> >  - Dennis
> > 
> > PS: I continue to separate application from the software that may be 
an
> > instrument of that application.  (Or, put another way, for me 
> application
> > software is not the application, it is software intended to support a 
> class
> > of application.)  I shall never give up on keeping the two separate. 
If 
> you
> > require different terms for maintaining the distinction, let me know. 
I
> > think the distinction is crucial. 
> > 
> 
> I think most members will equate application and implementation, since 
> that is the common usage, but I do understand the difference you are 
> trying to enforce. 
> 
> > PPS: I also can be careless in how I use "application."  I'm training 
> myself
> > to be more careful.
> > 
> >    -   -   -   -   -   -   -   -   -
> > 
> > A. WHY I THINK OF THOSE CLASSES OF PRODUCTS (BOTH FORMATS AND 
SOFTWARE)
> > 
> > 1. It is the Open Document Format for Office Applications
> > 
> > 2. In section 1.1 (ODF 1.1),
> > 
> > This document defines an XML schema for office applications and its
> > semantics. The schema is suitable for office documents, including text
> > documents, spreadsheets, charts and graphical documents like drawings 
or
> > presentations, but is not restricted to these kinds of documents.
> > 
> > The schema provides for high-level information suitable for editing
> > documents. It defines suitable XML structures for office documents and 

> is
> > friendly to transformations using XSLT or similar XML-based tools. 
> > 
> > 3. From the Charter of the ODF TC, as last amended,
> > 
> > 3.1 The purpose of this TC is to create an open, XML-based file format
> > specification for office applications.
> > 
> > The resulting file format must meet the following requirements:
> > 
> >   1. it must be suitable for office documents containing text, 
> spreadsheets,
> > charts, and graphical documents, 
> > 
> >   [ ... ]
> > 
> > 3.2 Scope
> > 
> > Since the OpenOffice.org XML format specification was developed to 
meet
> > these criteria and has proven its value in real life, this TC will use 

> it as
> > the basis for its work.
> > 
> > A standard for office document processing and interchange will be of 
> great
> > utility to many users and software companies developing applications, 
> and
> > should be made available as soon as possible.
> > 
> >    -   -   -   -   -   -   -   -
> > 
> > I understand that the characteristics of such products we are 
concerned
> > about is their support for ODF documents on behalf of office 
> applications
> > (which for me are what office workers are concerned with and use those
> > products for in their work).
> > 
> > I do confess to looking at interoperability as having to be situated, 
> even
> > as we speak of general office-document creation, collaboration, and
> > interchange.
> > 
> > Maybe we should have Marston's feedback on what he had in mind for 
> Classes
> > of Product, because I can't tell anything specific from my notes or 
the
> > slides to suggest he was talking about Conformance Targets.
> > 
> > And, either way, I do think we need to have some sort of clarification 

> like
> > this with regard to the state of interoperability.
> > 
> > 
> > B. ABOUT THE DISTINCT ODF DOCUMENT TYPES
> > 
> > Finally, while I agree there is a single specification for several 
> document
> > types, there are indeed distinct document types, called out in the
> > specification.  Section 2.3 Body Elements and Document Types is pretty 

> clear
> > about this. 
> > 
> > I am referring to those types, which can occur independently and be
> > processed independently.  So there are indeed conformant ODF text 
> documents
> > and ODF spreadsheet documents as well-defined divisions of conformant 
> ODF
> > document, driven by the correspondence of MIME type and the required 
> single
> > element of the required <office:body> element within content.xml. 
> > 
> > Not only is it the practice to speak of these types separately, but it 

> is
> > also commonplace that they have separate components for their 
processing
> > when integrated in an office suite.  It is also common practice in 
> office
> > applications that one must commit oneself to a type of document at the
> > commencement of document creation (with some exceptions for templates 
> and
> > master text documents), and the current division of document types 
> requires
> > such a decision to be made too.
> > 
> > I am recognizing that characteristic of ODF 1.1 and how it is 
> encountered in
> > practice.  What's wrong with that?  Won't each of the majority of test
> > documents necessarily be one of the main three types?
> > 
> >  - Dennis
> > 
> > 
> > -----Original Message-----
> > From: robert_weir@us.ibm.com [mailto:robert_weir@us.ibm.com] 
> > http://lists.oasis-open.org/archives/oic/200901/msg00054.html
> > Sent: Thursday, January 22, 2009 12:10
> > To: oic@lists.oasis-open.org
> > Subject: Re: [oic] Testing Approach Decision #1: What Classes of 
> Products
> > Are To Be Tested?
> > 
> > "Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 01/21/2009 
> > 04:46:58 PM:
> > http://lists.oasis-open.org/archives/oic/200901/msg00048.html
> > > 
> > > Here is a straw man for the first of the "First Wave of Decisions" 
> item:
> > > What Classes of Products are to be Tested?
> > > David Marston's list is at 
> > > <
> > 
> 
http://wiki.oasis-open.org/oic/TestingThoughts1/800_The_First_Wave_of_Decisi

> 
> > ons>.
> > > 
> > 
> > I'm taking "products" in the specific sense discussed in "Variability 
in 
> 
> > Specification".  In that sense, ODF 1.1 (and ODF 1.0) defines three 
> > specific products:
> > 
> > 1) Conforming documents
> > 2) Conforming applications
> > 
> > Applications are further split into:
> > 
> > 2a) Those that read or write ODF
> > 2b) Those that both read and write ODF
> > 
> > Although it may be natural to suggest other products, such as 
> spreadsheets 
> > versus word processors, these are not really "products" in the 
> conformance 
> > sense.  There is no definition of a "conformant ODF spreadsheet" that 
> > differs from a "conformant ODF application" in general. 
> > 
> > That said there are certainly a range of implementations that we might 

> > test against these product types, and some implementations may be 
> > spreadsheets while others may be word processors.  But in the 
> conformance 
> > sense there is no such distinction in ODF 1.1.
> > 
> > But I may be entirely wrong in the above analysis.  To show that I am 
> > wrong, you will need to show me where conformance is defined different 

> for 
> > each product where you suggest a different product exists.  For 
example, 
> I 
> > may be wrong if we can show that there is conformance for an "ODF 
> Package" 
> > that is distinct from an "ODF Document", or that an "ODF Document 
> > Template" has distinct conformance requirements from an "ODF 
Document".
> > 
> > Also, the above is just with respect to conformance.  For 
> interoperability 
> > tests we do not need to be as strict with the products defined by the 
> > standard.  We have the freedom to also consider useful distinctions 
that 
> 
> > exist in the real world, and this could easily include "products" such 

> as 
> > spreadsheet documents, etc.
> > 
> > -Rob
> > 
> > 
> > [ ... ]
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 
>