Request for feedback: Proposed procedure for handling public review comments

[Chet Ensign <chet.ensign@oasis-open.org>]

OASIS members, 

In December, on the chairs@ mailing list, I discussed what procedures to put in place to implement the OASIS requirements for handling comments received during public reviews. I also presented the proposed procedure with the OASIS Board's Process Committee for their information. 

Below is the draft proposal, amended with the feedback I received. I am sharing it with all of you now to gain broader member feedback before implementing a final set of rules. My goal is to publish the procedure in the last week of January with the goal of making it effective for all public reviews and charter proposals beginning after February 1st. (I do not intend to make it retroactive.) 

Please set aside some time over the next couple of weeks to review the proposal and share any thoughts or suggestions that you have. The final document will become the 'law of the land' so it is in all our best interests to ensure that the right balance is set between satisfying the requirements of the OASIS TC Process and minimizing the additional workload for TCs. 

To make sure that the discussion is public, please use the oasis-member-discuss@lists.oasis-open.org mailing list for your feedback (cc'ed on this message). You should not need to subscribe to the list; you should be able to use it automatically. If you have a problem posting to it, let me know. 

Also, attached is the comment resolution log for the feedback I received so far. 


Executive summary:

We need to get better consistency in how TCs track and report comments received during public reviews. Today, TCs have no clear direction or expectations from OASIS on how this is to be done. As a result, each TC arrives at their their own solution and candidly, some do not bother to track or report comments at all. 

With our strong liaison relationships with other Standards Development Organizatoins and, most recently, with the review of the OASIS Process as part of our application for ANSI accreditation, it is clear that we need a documented and consistent protocol that ensures: 

(a) a TC actively acknowledges receipt of comments, 
(b) at least the minimum facts needed are captured, tracked and reported, 
(c) the timeframe for a TC to deliver a final comment resolution log is clear, and 
(d) the reports of comments and their resolutions are persistent, immutable, publicly visible and readily found in the TC archives. 

The steps outlined below seek to meet these requirements with minimal impact on TCs. 

Definitions:

- For the purposes of this procedure, a "comment" is a statement received (a) in an email to a TC's -comment@ mailing list or (b) in the case of a TC member, in an email to the TC's primary mailing list during an announced public review period that refers to the work product under review.

In the case of a proposed charter call for comments, a "comment" is a statement received in an email to the oasis-charter-discuss@ mailing list that refers to the proposed charter.  

While a TC is free to handle any comments received at any time (and indeed is encouraged to do so), comments referring to earlier drafts or general comments unconnected with a public review underway do not fall under the requirements laid out below.

- For the purposes of this procedure, the "the end of the review period" mentioned in several places in the TC Process as the deadline for posting a link to a comment resolution log shall be deemed to be the point in time at which the TC next requests an action on a document (e.g. the next public review, a Special Majority Vote to approve a Committee Specification). 

- The TC Process refers to the record of comments and their resolutions in several different ways. For the purposes of this procedure, the document that records comments and their dispositions will be called a "comment resolution log." 

Draft Procedure: 

1) TC Obligations With Respect to Comments

The requirements for handling comments are laid out in sections 2.2, 2.8, 3.2 and 3.4 of the OASIS TC Process. Those requirements are:

- That comments be received only through the TC's <short name>-comment@ mailing list for non-members, the TC's primary mailing list for members, and the oasis-charter-discuss@ mailing list for comments on proposed charters.

- That comments be received by the Chair and optionally by one or more other members of the TC.

- That comments be acknowledged.

- That comments be tracked and their resolutions documented:

a) During a proposed TC call for comment period;
b) During a public review of a TC work product;
c) During a public review of a Candidate OASIS Standard.

- And that at the end of the comment / review period, the TC post to its email list a pointer to an account of each of the comments raised during the comment period along with its resolution.

2) Procedure for Meeting These Obligations

To meet the requirements, TCs must take the following steps:

a) The TC Chair is automatically subscribed to the TC's <tcname>-comment@lists.oasis-open.org mailing list and will always receive emails sent to that list. TCs may want to assign another member (e.g. the Secretary or editor)to subscribe to the list and track and report comments and issues back to the TC.

b) Someone from the TC must send an email back to the commenter acknowledging the comment. A simple "thank you for your comment; the TC will be consider it" message is sufficient.

c) The TC must track each comment received and how the group decides to resolve it. Per the TC Process, a TC is not required to take any action on the substance of the comment, but the decision not to must be recorded.

The TC can choose to track comments and resolutions however they wish: in a word processing document, in a spread sheet, in JIRA, in the TC wiki. The TC must at a minimum record:

- The date the comment was received;
- A link to the email in which a comment was received;
- The name of the person or entity providing the comment;
- A brief summary of the comment;
- A statement of the TC’s decision on how to handle the comment, in as much detail as the TC wishes to provide.

In the event that multiple emails raise the same issue, the TC can track and resolve them with one entry in the document, however a link to each of the emails and the names of each of the commenters must be included in the record.

Keeping a cumulative list of comments will likely to be the most convenient approach. The TC Process states that, before an OASIS membership ballot can be held on a Candidate OS, the TC must provide "a pointer to an account of each of the comments/issues raised during the public review period(s), along with its resolution." Having all the comments and their resolutions stored in one list will make satisfying this requirement easy.

d) By the end of the review period at the latest, the TC must post to its email list a document containing, at a minimum, the pieces of information listed above. The document can be either a word processing document, a spread sheet or a PDF document. A stand-alone document is required to ensure that the comment resolution log is persistent and immutable and publicly available on the OASIS system.

If the TC is using JIRA or a wiki to track comments, a document must be created by exporting the information from that source. 

If the document is a cumulative list of comments and resolutions, it should also contain, for each comment, identification of the draft to which it applied.

The posting email can be created by loading the document into the TC's document repository and using the automatically generated email to produce the pointer or by attaching the document directly to the email. The TC may also forward a pointer to the document to the TC's -comment@ mailing list if it wishes to do so.

With that email, the TC obligations will be considered fully met.

3) What TC Administration Will Do With the Comment Resolution Document

The key objective is to ensure that comments and their resolutions can readily be found by others who are interested in the TCs work. Here is what TC Admin will do with the documents.

a) When the TC is ready to take another action on its work product (e.g. request another public review, request a ballot to approve a Committee Specification), a pointer back to this document will have to be provided as part of the support request. TC Administration will not proceed with a requested action until the comment resolution document is provided.

b) A link to the comment resolution log will be included in the announcements of subsequent public reviews, document approvals, Special Majority ballots and other communications where it will provide additional, useful information.

c) A copy of the comment resolution log will be stored in the OASIS Library along with the public review draft to which it applies. This will help to ensure transparency and traceability between comments received during a public review and their resolution.

--- / ---

Attachment: TC-Admin-Comment-Handling-review-comments.xls
Description: Binary data

/chet 
----------------
Chet Ensign
Director of Standards Development and TC Administration 
OASIS: Advancing open standards for the information society
http://www.oasis-open.org

Primary: +1 973-996-2298
Mobile: +1 201-341-1393