OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

office message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: RE: [office] Re: Categories for Public Comment and Defect-Report Items

Thanks Rob,

With regard to the first part, the basic categories are with regard to what
the submission says the problem is, taking into account both the text and
any classification that is provided with the submission.

I agree that these are not orthogonal and there is often a gray area (for
example, between omission and incomplete, where the latter is more likely to
be a substantive matter relating to under-specification).

I think the determination of substantive in terms of whether or not an
implementation would be impacted by resolution is a separate determination.
Part of it has to do with whether the feature has even been implemented (as
we have discovered) and also whether the more-or-less obvious simple case
and/or "do whatever OO.o does" approaches have actually prevented there
being a problem.  I don't think the provisional assignment of categories can
or should deal with that, but rather be accounted for in the disposition and
the actions to determine what remedy, if any, is possible. 

There is now a "Source" column in the register, although most entries are
blank and the source is the public comment list.  SC34 defect-report items
are identified in the column already.

To sharpen some of these, I think I use "omission" only when the submission
says what the omission is or where there appears to be an omission.

Incompleteness is generally a submission that claims that something is
incompletely specified.

Unclear is generally when a portion of the specification is claimed to be
not understandable or just plain confusing.  

Inconsistency is when the submission claims that there is a contradiction or
an inconsistency with something said elsewhere.  Sometimes it may be changes
in nomenclature that might be resolved as an editorial correction (use of
the wrong or undefined term in place of the one introduced elsewhere) and
sometimes it might be something more substantial.

My general approach in this process is to take the submitter's word for it
until there is a closer examination.  The important thing is I don't think
these are dispositive.  That should be a separate item, which is what we say
it is, as opposed to what the main thrust of the submission is.  Also, we
might concur with the identification of a problem (and when someone says
something is unclear, we should believe them), but not the suggested remedy,
if any.  

Other matters, such as the standing of a submission and responsibility of
the TC to have a response of some sort is something I would like to keep
orthogonal to characterization of the substance of a submission (as opposed
to the import of that after review, disposition, and action).

 - Dennis

PS: As far as I am concerned, all comments from external sources, unless
clearly capricious, are meritorious at any time and under all conditions.  I
think there should be a transparent process for their disposition, with the
decision to go forward, take action, or simply let them sit made explicit.
(Although it does not happen often, it does happen that SC34 errata point at
something that already arose elsewhere, including within the TC or an
earlier public comment from other submitters.)  It is one of the things that
distinguishes open efforts such as OASIS TC, W3C comment handling, and of
course IETF technical committee conduct.

PPS: With regard to discussion on the public comment list, it is pretty
clear when that is happening, including when the discussion is between the
submitter and TC members.  I have not added those to the list beyond the
original submission which was directly to the list and not any subsequent
on-list discussion.  Sometimes the discussion may lead to retraction and
sometimes there is a proposed disposition (e.g., some adjustment in an
OpenFormula definition).  I would think closing out the original submission
involves finding out what has actually been done in the specification, just
as we need to regression-check 1.2 against previously-accepted comments and
any errata based on them.

-----Original Message-----
From: robert_weir@us.ibm.com [mailto:robert_weir@us.ibm.com] 
Sent: Sunday, January 18, 2009 09:12
To: dennis.hamilton@acm.org
Cc: Michael Brauer; ODF TC List; Patrick Durusau
Subject: [office] Re: Categories for Public Comment and Defect-Report Items

"Dennis E. Hamilton" <dennis.hamilton@acm.org> wrote on 01/17/2009 
08:32:34 PM:
> After working with the public comment register for a while, and 
> adding another 100 public comment items, I began to use an expanded 
> set of categories (beyond defect, no-defect, and editorial).
> My current working list is as follows, lining up pretty well with 
> the terms used in the SC34 defect report:
>  * editorial - about wording or whether a non-normative statement 
> should be a note or not, etc.
>  * omission - something specific is missing (may be a particular 
> kind of editorial problem)
>  * incomplete - there is insufficient information to understand what
> a feature is and/or what is normative about it
>  * inconsistent - statements that appear to be contradictory or not 
> consistent with what appears elsewhere
>  * unclear - a problem making sense of the statement in the 

This is tricky.  I'm not sure the comments necessarily lend themselves to 
any flat classification scheme.    For example, "unclear" could apply to 
editorial or technical comments, and be caused by incomplete or 
inconsistent text, etc.  Some things characterize the nature of the 
defect, while other characterize the cause of the defect, while others 
characterize the impact of the defect.  We also need to make it clear 
whether we are classifying the comment, or classifying the defect.

For classifying a comment, I think our process would lead to a 
multi-dimensional classification scheme:

A) Where did the comment come from:

1) Comment received on public comment list
2) Comment received in SC34 defect report
3) Comment received from ODF TC member

B) When was the comment received:

1) Comments received during an official public review period (these have 
mandated processing requirements under OASIS rules)
2) Comments received outside of official public review period

Then we can classify the comment as to the nature of the change requested.

C) What change is requested:

1) New feature proposed
2) Editorial correction requested
3) Technical correction requested
4) No change is suggested.

We could use the JTC1 definitions of editorial and technical here:

Editorial defect is "An error which can be assumed to have no consequences 
in the application of the IS, for example a minor printing error."

Technical defect is "A technical error or ambiguity in an IS inadvertently 
introduced either in drafting or in printing which could lead to incorrect 
or unsafe application of the IS."

(Note that the JTC1 definitions are not really binary, though they do not 
give a name to the set of defects which have consequences, so are not 
editorial, though the consequences do not lead to incorrect or unsafe 
application, so they are not technical defects.  We could introduce the 
terms "major technical" and "minor technical" to fill that semantic space 
more completely.)

We also need to consider the question of whether the change requested is 
Substantive, by OASIS definition, since that limits what can and cannot be 
changed in an Approved Errata document.

D) Is the requested change a Substantive Change?

1) Yes
2) No

OASIS TC Policy 1.aj says: "Substantive Change" is a change to a 
specification that would require a compliant application or implementation 
to be modified or rewritten in order to remain compliant. 

We also need to track our formal disposition of the comment.

E) How shall the comment be disposed:

1) No action.  Comment already addressed in XXX.
2) No action.  Comment duplicates previously received comment #XXX
3) No action.  Comment is in error.
4) Will correct in version XXX of ODF Standard
5) Will correct in version XXX of ODF version YYY Approved Errata
6) Will consider as a new feature in version XXX of ODF Standard

We've had other "actions" like referring to OpenFormula subcommittee, 
etc., but these are not really dispositions, but are more like steps on a 
workflow toward arriving at a disposition. 

> I've stopped using a no-defect category, although that can certainly
> be a disposition. 
> These seem to be sufficient for all of the N1078 items that we 
> should review for a disposition report back to SC34.  I have stopped
> using the simple category "defect."
> There are some additional categories that arise, but not among the 
> defect-report items:
>  * proposal - a request for a feature or some specific suggestions 
> about process or content
>  * comment - an expression of opinion with nothing actionable presented

Non-actionable comments are tricky.  Outside of a public review period we 
do not need to process them.  No need to even transcribe them into the 
spreadsheet.  But during formal review periods we must formally track all 
comments received, since OASIS process requires that the TC: "acknowledge 
the receipt of each comment, track the comments received, and publish to 
its primary e-mail list the disposition of each comment at the end of the 
review period. "  We could certainly get into the practice of recording 
all comments regardless of whether we are in a review.  But this might be 
difficult, especially when the comment list starts to degrade into a 
discussion forum.  What is a comment versus free-ranging discussion?

>  * contribution - feedback of some kind, with suggestions, not (yet)
> classified further
>  * objection - some exception taken to direction or existence of a 
> provision, often quite general (e.g., proposals to put spreadsheets 
> out of their misery) 
> These are probably too many, but they seem helpful in understanding 
> the general nature of a particular submission.  Of course, any 
> applications of categories that I have used are only straw-men 
> subject to the discussion and determination of the TC. 
>  - Dennis
> PS: There is one kind of comment that I have not found a 
> satisfactory initial category for.  An example is many of the public
> comments from David King that suggest clarifications and corrected 
> language around aspects of OpenFormula, as well as make questions. 
> I've been calling these contributions for lack of a better term that
> may arise out of closer screening. 

This probably doesn't impact the classification.  It is still editorial or 
technical defect, or feature proposal.  But how we dispose the comment 
will be difference depending on whether it pertains to an already approved 
OASIS Standards versus an in-progress  draft.

> Dennis E. Hamilton
> ------------------
> NuovoDoc: Design for Document System Interoperability 
> mailto:Dennis.Hamilton@acm.org | gsm:+1-206.779.9430 
> http://NuovoDoc.com http://ODMA.info/dev/ http://nfoWorks.org 

To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]