Re: [chairs] Chairs: Requesting your feedback on comment resolution logs

[robert_weir@us.ibm.com]

Chet Ensign <chet.ensign@oasis-open.org> wrote
on 04/24/2012 06:21:40 PM:

> 
> TC Chairs,
> 
> Recently, I have had several discussions with people on comment
> resolution logs. As I'm sure you know from the TC Process, TCs are
> required to maintain logs of comments received and their resolution
> and provide them at several different points throughout the process.
> Specifically, in section 3.2 Public Review of a Committee Draft
> (http://www.oasis-open.org/policies-guidelines/tc-process#publicReview),
> the process reads:
> 
> "The TC must acknowledge the receipt of each comment, track the
> comments received, and post to its primary e-mail list its disposition
> of each comment at the end of the review period."
> 
> Our growing liaisons with other standards bodies is making getting
> this right even more important.
> 

The liaison aspect is interesting. We've had a few
rounds of defect reports between ISO/IEC JTC1/SC34 and the ODF TC.  On
the JTC1 side they expect documents.  They have a form for reporting
defects and expect a response in a particular form as well, ultimately
as draft corrigenda.  So there is a formal aspect to this, as well
as the responsiveness/professional courtesy aspect you noted before. 

Another new data point, since your last note. I was
recently at an ODF conference in Brussels (an ODF Plugfest).  I gave
an update on the work of of the ODF TC, and in the Q&A received some
feedback on the public comment process:

1) There is an expectations that those who submit
comments should get a timely personal response.  But not only for
comments submitted during a public review period.  They expect it
for all comments.  This is reasonable, though the OASIS process only
makes it a requirement for public review comments.  

2) Timely means days or weeks, not months.  

3) There was the expectation that the comment list
could be used to ask questions and seek clarifications about the interpretation
of the spec, and that TC members would reply with an official response.
 But as you know, this is not really possible.  We generally
turn "What does clause 1.2.3 mean?" into a defect report "Clause
1.2.3 is ambiguous".  But we don't issue interpretations.

4) The feedback I received might be particular to
a mature standard like ODF.  If you are working on your first edition
of a standard, then you may not get comments until your first public review.
 But once you have 3 revisions of your standard published, then comments
flow in at a regular pace, regardless of your current review status.  And
the comments are often from implementors, not spec reviewers.  If
they are actively implementing the spec, then their code may be waiting
for your response.   So a different situation, maybe different expectations
compared to CSD 01 of FooML 1.0.  

In a sense, responding to comments becomes a maintenance
activity, rather than a public review activity.

> I believe it is becoming more and more important
that we accomplish
> both the letter and the spirit of this requirement in order to make
it
> easier for commenters to learn of the disposition of their feedback,
> for reviewers to learn what has happened between one public review
and
> the next, and for voters to get a full picture of the history of a
> Committee Specification when it is advanced as a Candidate OASIS
> Standard.
> 
> I would appreciate hearing your thoughts on how best to accomplish
> this, how best to provide you with the flexibility you need while
> ensuring that the record of comments and resolutions is maintained
and
> made available over time. For example,
> 
> - Some TCs are using JIRA successfully to track, manage and report
on
> comments. Would training in using JIRA for this process be useful?
> 
> - Some TCs are using spread sheets to track and report on feedback.
> Would template spread sheets help you adopt this approach?
> 
> - Would a document on best practices help you and your TC put a
> mechanism in place to successfully track and report on comments.
> 

There might be some workflow things we can define
in JIRA to make comment processing easier.  For example, if responding
to the comment submittor is important, than maybe that should be be a stage
in the workflow?  Or a field that can be set and filtered on?

If we can get JIRA to manage this well, then it has
reasonable export to spreadsheet capabilities. 

More radical options, that would require some IT support:

1) Allow the public to register in JIRA and submit
issues directly.

2) Allow public to "watch" issues in JIRA
so they are notified when the status changes.

In other words, the things that are good about JIRA
for us are also good for those submitting comments to us.

Regards,

-Rob


> Please take a minute to share your thoughts with me on how we can
make
> this work conveniently and securely. I want to gather the fruits of
> your experience and thoughts, as well as the feedback from your TC,
> before making any proposals for next steps.
> 
> I look forward to hearing from you and meanwhile, thank you for all
> the work that you do here at OASIS.
> 
> Best regards,
> 
> /chet
> ----------------
> Chet Ensign
> Director of Standards Development and TC Administration
> OASIS: Advancing open standards for the information society
> http://www.oasis-open.org
> 
> Primary: +1 973-996-2298
> Mobile: +1 201-341-1393
> 
> Follow OASIS on:
> LinkedIn:    http://linkd.in/OASISopen
> Twitter:        http://twitter.com/OASISopen
> Facebook:  http://facebook.com/oasis.open
>