RE: [clr-dev] Code list versioning

["G. Ken Holman" <gkholman@CraneSoftwrights.com>]

At 2009-03-05 14:35 +0200, Konstantin Hypponen wrote:
>Apparently, you have made a decision that codes can never change: 
>new codes can be added, and old ones removed (or marked as invalid), 
>but code value changes are not allowed. Are there more flexible 
>solutions available, which would allow code changes?

The CVA file format allows one to specify the union of more than one 
list for a given information item.

Consider this example of updating codes:

The CVA file in 2008:

    <ValueList xml:id="mylist2008" uri="mylist-v2008.gc"/>
    ...
    <Context item="myElement" values="mylist2008"/>

... where mylist-v2008.gc has the following list-level meta data, 
values and value-level meta data:

    <Identification>
      <Version>2008</Version>
    </Identification>
    <SimpleCodeList>
        ...AA...concept 1
        ...CC...concept 2

Everything is working fine with only a single list and the 
<myElement> item is correctly validated with:

    <myElement>AA</myElement>

and:

    <myElement>CC</myElement>

Now it is 2009, and there is a new genericode file, mylist-v2009.gc 
which has the following list-level meta data, values and value-level meta data:

    <Identification>
      <Version>2009</Version>
    </Identification>
    <SimpleCodeList>
        ...AA...concept 1
        ...BB...concept 3
        ...CC...concept 4

Note how we've added a new concept "BB", and the old "CC" concept 2 
has been replaced with a new semantic concept 4 using the same coded value.

The CVA file for 2009 could allow for the union of the two lists:

    <ValueList xml:id="mylist2008" uri="mylist-v2008.gc"/>
    <ValueList xml:id="mylist2009" uri="mylist-v2009.gc"/>
    ...
    <Context item="myElement" values="mylist2008 mylist2009"/>

Everything appears to be working fine, now, as now the following will validate:

    <myElement>AA</myElement>

and:

    <myElement>BB</myElement>

But what about the following?  It validates as acceptable, but it is, 
in fact, ambiguous to an application responsible for interpreting the 
represented semantic:

    <myElement>CC</myElement>

... because you don't know if it represents concept 2 or concept 4.

The role of instance-level meta data is to resolve such 
ambiguity.  The designer of the XML vocabulary has a responsibility 
to model in provision for instance-level meta data with the information item.

This would allow the user to specify concept 2 by using something like:

    <myElement v="2008">CC</myElement>

... and to specify concept 4 by using:

    <myElement v="2009">CC</myElement>

There is also a strategy that I won't go into detail now of omitting 
instance-level meta data in order to create an instance that will be 
valid for a *future* code list with a code you are expecting:  in the 
interim you union the public code list with a private code list that 
has the future extension to the public code list.  You omit the 
instance-level meta data so that it validates but doesn't use your 
private list-level meta data.  When the future list is published and 
put into production, your use of the new code still works because you 
didn't tie it to your private extension.

There are no standards for instance-level meta data, so each 
vocabulary will be making their own decisions of where to capture 
this important information.  Each implementation of CVA will need to 
be customized to recognize a particular vocabulary's conventions for 
instance-level meta data.

As we approach UBL version 2.1 in the UBL TC, I will be specifying 
for validation the union of each code list published in 2006 for UBL 
2.0 with the latest revision of that code list published in 2010 for 
UBL 2.1.  This will allow instances of UBL 2.0 created for the four 
years to continue to be valid with the release of UBL 2.1, while 
creators of new instances can simultaneously take advantage of new codes.

How far back do you go?  That becomes a policy decision.

If you are the custodian of a code list that is changed every 6 
months, you might decide to have a rolling union of, say, the last 
four issues of the code list.  Then you could claim validity testing 
for instances at least two years old, and for longer instances that 
have not tied their instance-level meta data to the list-level meta 
data of the old lists no longer in use.

Managing codes becomes a management task on its own, as the 
controlled vocabulary semantics and representations are in many ways 
just as important as the XML vocabulary of elements and 
attributes.  You'll have to publish policies and procedures regarding 
the maintenance of values and their representation.  And you'll have 
to inform the XML vocabulary people of your requirements for 
instance-level meta data properties to be available to users of the vocabulary.

I hope this helps.

. . . . . . . . . . . . Ken


--
XQuery/XSLT training in Prague, CZ 2009-03 http://www.xmlprague.cz
Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video
Video lesson:    http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18
Video overview:  http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18
G. Ken Holman                 mailto:gkholman@CraneSoftwrights.com
Crane Softwrights Ltd.          http://www.CraneSoftwrights.com/c/
Male Cancer Awareness Nov'07  http://www.CraneSoftwrights.com/c/bc
Legal business disclaimers:  http://www.CraneSoftwrights.com/legal