Re: [ubl-dev] Code List Value Validation

["Fraser Goffin" <goffinf@googlemail.com>]

Thanks David, I will take a look. As per Stephens comment below, one of the
nice things in the UBL proposal is the ability to use subsets (or the same)
code list in multiple contexts within the same document.

In terms of implementation, I had been thinking about how the code-lists can
be stored (genericode files are one proposal for this) and accessed
(schemaTron) and, because of the size and volatility of some of our lists,
and the need to provide access to many application platforms, and the desire
to have a centrally managed source of reference data, ... yadda , yadda. I
was thinking about RDBMS + XQuery as well. I guess there are (at least) 2
aspects to this. Firstly the creation of business rules for value-based
validation (this is what shemaTron is good at and is central to the UBL use
case), and secondly because I have other possible use cases in mind, the
implementation of a service for general access to the reference data to
providing full lists, subsets, decoding values, and so on ... Does you
implementation provide this sort of 'list manager' functionality ?

Fraser.


On 07/04/06, David RR Webber (XML) <david@drrw.info> wrote:
>
> Fraser,
>
> There are two levels to this - the specification and the implementation.
>
> I'd be interested in your thoughts on the lookup() functionality in the
> OASIS CAM system and the jCAM implementation that supports the use of
> codelists over XML transactions.
>
> The current implementation has baseline functionality for managing and
> using codelists:
>
>  http://www.jcam.org.uk
>
> Key from the implementers stance is being able to cover off the following:
>
> 1) Declarative rule values in XML
>  2) Lookups to SQL table code values
> 3) Simple inline code alignment (e.g. 1,2,3,4,5 | A,B,C,D,E)
> 4) Associative code lookups - if A then X,F,J ; if B then X,F etc
> 5) Callout to Java routines - permits extended calculated lists via
> multiple
> 6) Versioning and local extension values
> 7) Potential for referencing codelists stored in shared registry service
>
> Thanks, DW
> Chair - OASIS CAM TC.
>
>
> -------- Original Message --------
> Subject: [ubl-dev] Code List Value Validation
> From: "Fraser Goffin" <goffinf@googlemail.com>
> Date: Fri, April 07, 2006 8:09 am
> To: "G. Ken Holman" <gkholman@cranesoftwrights.com>
> Cc: abcoates@idmm.co.uk, xml-dev@lists.xml.org,
> ubl-dev@lists.oasis-open.org
>
> Ken,
>
> its been a while since I have had a chance to catch up, so hi.
>
> I have been re-reading your UBL Code List Value Validation Methodology (
> v0.4)
> again while I've had a few days off (I know I need to get out more :-),
> a few questions if I may be so bold :-
>
> 1. How has this work been received in UBL. Is it proceeding as planned. Do
> you think there will be any statement on adoption of this methodology any
> time soon ?
>
> 2. A genericode file contains ONE code list at ONE version right ??
>
> 3. In the doc you state that when an enumeration appears WITHIN a schema,
> TPs may ligitimately operate a subset since all values are valid in the
> full
> set, but they may not add new values. Am I correct in assuming that the
> same
> is true for a code list which is defined by a standards organisation (not
> UBL) but which is NOT embedded within schema ?
>
> As you know in my situation we operate code-lists than are defined by a
> standards body and in most cases want to use them in all situations where
> there are equivalent semantics (including internal app integration) rather
> than create alternative bespoke lists.
>
> Problem is, we do sometimes want/need to extend these lists often to
> provide
> higher fidelity mapping to our operational systems. Another example might
> be
> where a code identifies some high level semantic, but we want to be able
> to
> create a bunch of 'sub' codes to provide a more granular view - accepting
> that in 2-way translation there will be data loss. Lobbying the standards
> body and getting a timely change/addition can be problematic ?  -  anyway
> -
> I digress
>
> 4a. For code lists where there is no established [complete and/or
> definative] standard or where the semantics and values are TP relationship
> specific, the set of permissible values can be extended and/or restricted
> from an offered base set (if available - using your
> DocumentStatusCodeCodeType example) or the participating organisations can
> agree the set of values (and presumably the list ID to be used in XML
> instances). Is this correct ?
>
> 4b. If I have many TPs and each has a slightly different relationship,
> might
> this cause me to need a separate genericode file for EACH code list that
> differs, however slightly, from another ?. Is there a suggested low
> maintenance approach to this problem ?
>
> 4c. Similarly to (4b.), if a custom code list is shared across service
> contracts for multiple TP relationships, but a need arises to create a new
> version with [say] some values added or removed, and we need to be able to
> operate both versions concurrently for some period of time, does this
> require a complete re-statement of the new code list (in a new .gc file)
> with a new version number even if the difference is ONE codified value
> (added/deleted/changed) amongst a set of 10,000 values ?
>
> This also means that the implementation will repeat a lot of code. I guess
> I
> am wondering whether there is/should be a way of expressing a 'delta' of
> values ?
>
> 5. Continuing the theme of (4), we have some code lists which are both
> highly volatile (values added and deleted (rarely changed) every month)
> and
> are very large (e.g. > 50000 entries). An example is Vehicle Make/Model.
> Do
> you think this approach is suitable for this type of reference data
> (multiple 'active' versions, large number of values) ?
>
> 6. Can you explain the difference between the UBL 'CodeType' and
> 'IdentifierType' in terms of what circumstances you would use either ?
>
> If a schema identifies the ListID, but we want to use a different one (to
> employ as 'richer set' of values) , how would an industry standard schema
> accomodate this possibility such that the schema remains a standard and
> unchanged definition of the structural constraints (is this the
> CodeType/IdentifierType approach) ?
>
> 7. Do you think that a skeleton context association file could be
> auto-generated ?
>
> 8. Changing tack slightly. I am interested in using genericode files for a
> number of purposes including, value-based validation, UI generation (e,g,
> to
> populate UI controls such as list boxes), transcoding between application
> specific codes. It would appear from the genericode materials that this
> would be feasible, do you agree ? :-
>
> Std Code    Std Desc        Appl'n A Equiv    Appl'n B Equiv    UI Text
> (key)                                (key)                 (key)
>
> abc            Std Widget      def
> ghi                     Part No 3321-7 (small widget)
>
> 9. What is the suggested approach to deal with deprecated code values. Is
> this considered as a versioning issue both for standards based code-lists
> (embedded in schema or not) and custom code lists ? Should code lists
> include validity date/time values or other 'active/deprecated' indicators
> ?
>
> 10. Caller assertion of list version. If there is no matching version is
> it
> best to flag the validation failure (and possibly reject the message) -
> that
> is, 'trust' the caller assertion, or validate against the un-vesioned
> complete list (similar point to the one we discussed earlier about whether
> to to an xsi:schemaLocation attribute value) ?
>
> 11. Devils advocate: Whats the difference in having to distribute the
> latest
> .gc file versus having to use the latest XSD with updated embedded enums ?
> (Ok, I think I know the answer to this one, but it would be good to have a
> quote from 'championing' designer, for the benefit of my peer group and
> sceptical and untrusting bosses :-)
>
> Cheers
>
> Fraser.
>
> Anthony: So that you are aware, I am attempting to stimulate interest in
> the
> use of genericode within the organisation that I work with (a large UK
> financial services company) from a number of potential perspectives. One
> of
> these is value-based validation, hence discussions with Ken i.r.o his work
> with UBL, but also for a more broadly accessible resource for reference
> data
> used for a variety of purposes such as UI generation, transcoding, etc..
>
>