Re: [office-comment] Proposal for new document element, "Documenthistory"

[Michiel Leenaars <michiel.ml@nlnet.nl>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Dennis,

thanks for your detailed questions. I think the response from mr.
Rafterty is also very relevant re group voting is also very relevant.

Before I start answering your question there is one more suggestion I'd
like to make as an addition to my original submission yesterday: it
would add value to the track changes mechanism if people would be able
to crypt their revisions (i.e. make them unreadable unless people have
the right cryptographic key to decipher it). The main use case is the
following: governments these days have to deal with Freedom of
Information Act requests or similar legal obligations around the world
that force them to disclose electronic documents. In case of people with
a visual disability, these documents have to be delivered
electronically. Having crypted and electronically signed changes would
allow 'blanking out' those parts of a document which are not relevant
within the scope of such laws for more easy, while allowing for an audit
trail to go to the responsible person if there is unnecessary
censorship. So both sides would profit - faster and more automated
delivery of documents on the one side, and clear legal responsibility on
the other side.

Having said that, I will try to answer your answers inline.
Unfortunately this grew to be quite lenghty, apologies for that:

> 1. The existing ODF 1.0-1.2 Change Tracking mechanism captures
> descriptive information about each tracked change.  These are in
> <office:change-info> and other text that are part of the
> <text:changed-region> elements gathered up into a single
> <text:tracked-changes>.  There is provision for descriptive text,
> whether synthesized or provided by users in some
> implementation-specific manner. Have you examined that and also seen
> how the information is provided as part of change-tracking review in
> current implementations

I have focussed on presenting my thoughts on the document history
primitive and the demands that would bring, not on how to synthesize or
edit the descriptive information; currently, there is no such document
history inside the visible document, but there are many hand-crafted
versions around. I did sample the revision and undo stack in some
products, but since my thought was to leave this up to the application I
did not see a need to specify it in more detail.

The usefulness of a document history would obviously depend on the
ability of the track changes mechanism to adequately capture the user
intent (i.e. removing a column in a table is something distinctly
different in a document history than deleting a table and adding another
table with one column less/deleting a number of cells one by one;
similar would go for a move, or a search and replace) while being
flexible to produce a readable, concise output.

>  2. In the ODF 1.0-1.2 implementations, the change-tracking
> information tends to disappear when the changes are accepted or
> rejected.  I presume that is also the case for the generic proposal,
> since there is really no place to keep tombstones for changes that
> are no longer present and/or no longer tracked.  So the history would
> need to be kept persistently longer than the technical change
> tracking itself.  (I haven't checked to see how that works with
> regard to marginal change bars, which should in some cases survive
> the acceptance of changes.)
> >
> > Is it correct to assume that you want a history of accepted changes
> > and this needs to persist through successive revisions?  This would
> > appear to be a quite new feature if it is intended that the history
> > would reflect the pagination/section-numbering in the current
> > version, especially for the current place where material was
> > previously deleted.

I think there is a definite use case for a history of change
transactions in a document. This could be done in a number of ways, and
I would argue that enabling at least one complete revision-aware work
flow to make this possible would be useful for many types of official
documents.

ODF already has the version mechanism that allow to save multiple
versions of a document alongside in one document container; adding a new
version provides a crude form of document history. But the versions are
disjunct - it is not clear what the one has to do with the other.
Also, as it essentially duplicates a document with every version, in
large documents this is costly regarding size.

My goal is to create an alternative that would allow to keep history in
a more automated way. Currently, there is only one state for change
transanctions, and all records of a CT disappear as soon as you accept
it. I think that in many cases that is what the user wants, but that in
more demanding usage scenario's there are at least three states that
could be given to a change: "suggested", "proposed" and "accepted".

Note that the first is a more cautious/polite/culturally sensitive
version of a proposed change, as currently implemented in applications
(for instance AbiWord implemented this at the request of the OLPC
project and calls this 'Annotations'). It will show the unchanged
document even when you turn on show changes, but will visually or
otherwise indicate that a concrete suggestion has been made to make a
certain modification to a specific region. In other words, you don't
just rewrite someones text, you craft a suggestion or recommendation
(this is of course not limited to text, but could be any change to a
document, including changes to graphics). For official publications it
is essential that these 'editorial considerations' may be deleted easily
by some automated process so they don't end up in print or on a website,
unlike inline text with a different colored background - which is often
used in solutions that don't have a suggestion mechanism.

The remark by mr. Rafterty regarding group voting might also be taken
into account here. Multiple people could sign a 'suggested' change, and
if a threshold is met it can move to "proposed" or "accepted".

The currently implemented change tracking has one state, which equals
'proposed'. The third state - 'accepted' - would mean that the change is
approved *yet kept for the record*. This would give an interesting
document time slider option, where you can go back and forth between the
different versions of a document without explicitly saving them as such.
This is similar what zfs snapshots can do for the filesystem of the same
name. Of course it is also possible to keep the current behaviour, i.e.
that the change is merged into the document as the user accepts the
change.

Implementing this could be done by adding an element or attribute
delta:change-status within <delta:change-transaction> or elsewhere. In
case of a change that is 'accepted' but kept for the record it might be
necessary as well to store who 'accepted' the change (and possibly add
an electronically signature so one can be sure it is the person with the
right mandate).

If that is considered too costly from a runtime point of view, it might
also be an option to move the part of the actual history of accepted
changes into either a protected log within the (proposed)
<text:document-history> element, or in a separate history.xml file. A
(metadata) anchor could be added to the document to capture the
position.

I am very conscious of the fact that any mechanism retaining history
does have a cost. In some cases auditable history is an absolute
requirement, which makes the main question: should we not have the
choice for that cost to be automated instead of human.

>  3. I would also think that there is difficulty with changes that
> rescind/obsolete previous changes.  That means the obsoleted change
> history can't point to any place in the current document where they
> have effect.   I assume one might still want to have them in the
> history, optionally, although they are technically not in the history
> of the current document.

> Do you see limitations placed on what can be removed/suppressed from
> the history by someone editing the document?  (I assume it is not
> possible to prevent turning off the history.)

I don't see limitations as such to suppressing, as long as this can be
undone or verified by the user. Removing and turning off history would
defeat the purpose, but should not be forbidden at the application
level. There are too many ways to go around it, and good ways to detect.

I think additional certainty for the recipient of a document can be
created by signing (versions of) a document, and optionally signing
(sets of) Change Transactions. The user interface could make an effort,
to distinguish text describing history as created by the user and text
describing history as generated by tooling.

Any case of fraud would be easily detectable - as the 'real' changes are
captured by the CT, it is possible to turn on "show changes" and see the
actual low level changes. And by running XML diffing tools,
modifications made 'off the record could be retrieved as well.


> 4. I assume that the facts in a history entry would not be editable,
> although accompanying descriptive annotation might be.   (In the
> current ODF, such things can themselves be change-tracked, though I
> don't know that there is any implementation that provides any way to
> accomplish that.)  There is also some question how the authors
> involved are identified and known to the software in which a revision
> is being edited.
>
> Are there any limitations on what you see as alterable and
> non-alterable in a document history entry?

Like you say: in an audit trail, edited facts are suspicious. But if you
cannot control the editing environment (OFS of Norway offers such a
controlled solution) you may not be able to avoid tampering - unless
you resort to checksums and other mechanisms similar to what distributed
software development tools like git use.

Fortunately, document histories are the responsibility of the person
producing them - they are drafted by hand now, and thus very much the
creation of an author. Things get a lot better with automated document
history generation. If state A is known externally (and possibly signed
electronically by those responsible), and there are a number N changes
on that clean document (possibly again signed by those responsible) -
then verifying that there have been unaccounted modifications outside
of those changes can be done automatically. So the assurance mechanisms
need not be in ODF.

>  5. On reflection, this strikes me as a substantial feature,
> especially once one also deals with the styling and formatting of the
> document history, where it can be placed in the document, etc., as
> for other kinds of "indexes."  There are also granularity issues.
> For example, a mass change of a word or phrase throughout the
> document (for a change of name or organization or reference to
> something external) would appear to be many changes but it would
> probably best be handled in the history as one entry.
> >
> >  Is this an use case?

Yes, that is most definitely a use case. This is one type of change
transaction that impacts many places in a document, and should be (at
the discretion of the user) presented as either one thing or as separate
edits. I.e. in the first case the history would state something like
("Replaced 43 instances of Sun Microsystems with Oracle - page 1, 2, 3,
5") - with a each instance a hyperlink to the place where the change
occured, similar to the behavior in table of contents.

The second option might be a requirement of the chosen document history
format - in some cases these need to be in a sequential list where they
occured in the document, so first all the edits on page 1, then on page
2 etc. Typically this would be relevant in printed version of standards,
or in legal texts.

> 6. In current implementations, it is possible to turn change-tracking
> on and off independent on whether or not changes are shown or not.
> (This is for text.  The situation for spreadsheet change-tracking is
> rather more complicated). Is this in accord with your assumptions for
> the document-history element being produced? Is it also important to
> have excellent document comparison tools to deal with any questions
> about changes and ones not accounted for in the history (e.g., minor
> editorial matters not related to substance and either not tracked or
> not accounted for in the history)?

I think I've answered these questions above in a fragmented way.
Turning change tracking on and off is independent of whether you see
them. I don't think this would have to change when the changes are made
visible in the document.

I'm assuming the document comparison tools do not have to be that
advanced, as you can just undo all the changes and compare that state to
a known and trusted document version. These should be equal. If not, it
is clear that the document history is no good.

Best,
Michiel Leenaars
NLnet Foundation

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2c164ACgkQPPKB2FVlk19gbQCdGKq4CvdUrCLSwoweBxRLfaO6
BEUAn3jQYRXrM2PVUkhYqGJDLcTZ7Np0
=wTxq
-----END PGP SIGNATURE-----