Re: [ubl] Review of Two Diffs (Michael/Sue's and Stephen's) - mycomments

[Tim McGrath <tmcgrath@portcomm.com.au>]

for what its worth....  here are my views on the modifications needed.

Stephen Green wrote:

>Here is my assessment of the differences found using the 'diff' technique,
>and my resolution
>proposals/suggestions. There are ten in all.
>
>
>1.   Address. Street Name. Name    vs    Address. Street Name. Text
>This appears in both sets of diffs.
>
>Proposed amendment   -   change published spreadsheet to Address. Street
>Name. Name
>Reason    this will match Address. Additional_ Street Name. Name
>  
>
i agree

>
>2.   Michael and Sue's spreadsheet picked up on
>Address. Additional_ Street Name. Name, etc   -   published spreadsheets
>have extra spaces after the underscore in the DEN's
>
>This is a known issue which has been pointed out many times before and
>appears to be due to a 'bug' in the formula generating the DENs in the
>published
>spreadsheets. In the past the DEN's in Schemas were not taken from the
>spreadsheet
>so this was just overlooked. Now the DEN's still are not taken from the
>spreadsheet
>so I overlooked it as before.  The effort to put this right might be out of
>proportion to the
>importance and the impact, if any, of leaving it. I propose to ignore it for
>now.
>
>  
>
i agree.  both the UBL name and the DEN in the spreadsheets are 
 non-normative.  we should try and keep those pesky trailing spaces out 
ot the names but it is only cosmetic.

>3.   Michael and Sue's work picked up on the extra dot at the end of ASBIE
>DEN's
>e.g. Address. Address Line. in the published spreadsheets but not the
>Schemas (labelled 'rt' in their work)
>I ignored it as too minor to warrant discussion since, again, the DEN's are
>generated
>and not normative. It might be worth fixing the formula to correct this but
>experience
>from 'beta' with the formulas makes me think it might be more trouble than
>the impact
>warrants.
>
>  
>
i agree.  unless we have a spreeadsheet formula guru who wants to take 
this on, it is too hard for us mere mortals.  perhaps we should clearly 
state that UBL Names and DEN in the spreadsheets are non-normative and 
for understanding the semantics only. the precise punctuation may not be 
accurate and that the schema and its annotations contain the correct names.

>4.   Both sets of diffs picked out the error in Allowance Charge. Reason.
>Code
>This has little significant impact. I suggest amending the spreadsheet
>because that
>avoids changing the Schemas but I accept that it could be argued that the
>Schemas
>could be changed because the change is to the documentation only. The 'tag'
>name
>is unaffected.
>
>  
>
there are two things wrong with Allowance Charge. Reason. Code.

firstly, the spreadsheet should have indicated this is a data type of 
 "Allowance Charge Reason_ Code. Type" to link it to the specialized 
data type of the same name. So the spreadsheet does need to be changed.

secondly,(and more significant) is that the DEN of this is given in the 
schemas as "Allowance Charge. Reason. Allowance Charge Reason_ Code" and 
not "Allowance Charge. Reason. Code".  in fact, i think this is a 
mistake in all schemas where we have used a specialized code (e.g. 
"Exchange Rate. Source Currency. Currency_ Code" instead of "Exchange 
Rate. Source Currency. Code").  It seems we have been using the data 
type qualifer in the DEN and I thought that we had agreed these data 
type qualifers are only for data types and not the representation term. 
 To quote the CCTS, it says...

> [B24] The Dictionary Entry Name of a Basic Business Information Entity 
> shall consist
> of the following components in the specified order:
>   the Object Class Term of the corresponding Basic Core Component, and
> possibly additional Qualifier Term(s),
>   the Property Term of the corresponding Basic Core Component, and 
> possibly
> additional Qualifier Term(s),
>   the Representation Term of the Data Type on which the corresponding 
> Basic
> Business Information Entity Property is based.

The last statement tells us the in the cases where we use a specialized 
data type (such as "Allowance Charge Reason_ Code. Type") we use only 
its Representation Term in the DEN (in this case the Representation Term 
is "Code").

If we accept this as true then many of the schemas that use specialized 
codes must modified by changing the 
/xsd:annotation/xsd:documentation/ccts:Component/ccts:DictionaryEntryNames.

i agree with stpehen this can be seen as typographical errors and will 
not affect any implementations.

>5.   Michael and Sue's worked picked up on the dropping of 'Currency_' from
>the
>DEN of Allowance Charge. Currency. Currency_ Code in the published
>spreadsheets
>
>I don't know which is correct but I think this is particularly worth
>changing if the error is in
>the Schemas since they are normative. However, it would be worth changing
>the spreadsheets
>if it was clear which version was correct (from the CCTS). If it is unclear
>and the Schemas
>don't need to be changed then I'd change the spreadsheets to keep them
>consistent with the Schemas
>  
>
i disagree.  i think this is another example of the issue i raised in 
(4.) above.  the spreadsheets are correct on this point and the schemas 
are not.  The DEN should be "Allowance Charge. Currency. Code" and not 
"Allowance Charge. Currency. Currency_ Code".  the schemas have put it 
in when it shouldn't be there.

>
>6.   My diff ommitted the differences in the DEN's for Allowance Charge.
>Amount since
>this difference only occurs in the DEN and not in the other column values.
>Michael and Sue's
>diff picked up on it. I would propose that since this requires a change to
>the formula which generates
>DEN's it is probably not worth fixing, except to add a note to the
>spreadsheets or package documentation
>to say that the spreadsheet DEN's should not be treated as normative or
>relied on. Perhaps the values from the Schemas
>should be those which are submitted to TBG17 if it is not too late. I do
>anticipate push-back on this but I'm
>bearing in mind with this and all such spreadsheet formula errors or
>inadequacies, that a spreadsheet formual has
>limited 'granularity' or precision when compared to a scripting or
>programming language for generating such strings.
>  
>
i agree

>
>7.   Both diffs picked up on a high-impact difference. Exchange Rate.
>Operator. Code
>The impact of this is high because it has resulted in the loss of the
>Operator codelist from the CAC Schema.
>This needs a correction to the CAC Schema. EF has it just as Datatype 'Code.
>Type' but it should be
>'Operator_ Code. Type' which would bind it to the operator codelist.
>Backwards compatibility might be barely affected
>if the NDR rule is followed which requires new namespaces anyway. This
>correction would require that that be
>done anyway (as we are discussing on ubl-dev with regard to restrictions of
>'bound' codelists.) The impact of
>the new namespaces on backwards compatibilty is obvious but acceptable, I
>would think, in the move from cd
>to specification (I take it that is what we are doing at this stage - if not
>then we have another problem, we need a second
>'cd' namespace, probably.)
>  
>
i agree with the diagnosis.  in the Common Aggregate Schema...
      <xsd:element name="OperatorCode" type="udt:CodeType" minOccurs="0" 
maxOccurs="1">
should be:
      <xsd:element name="OperatorCode" type="opr:OperatorCodeType" 
minOccurs="0" maxOccurs="1">
and this means adding statement for its namespace, such as...
    xmlns:opr="urn:oasis:names:tc:ubl:codelist:OperatorCode:1:0"
  <xsd:import namespace="urn:oasis:names:tc:ubl:codelist:Operator:1:0" 
schemaLocation="../codelist/UBL-CodeList-OperatorCode-1.0.xsd"/>

i defer to stephen on the impact of this - but it has to be done. 
 without this code set the exchange rate structure becomes uncontrollable.

>
>8.    There are two obvious errors in the spreadsheets published, both
>similar, but both can be corrected in the spreadsheets
>without affecting anything else:  DocumentDocumentStatusCode   and
>IssueIssueDateDate, both in OrderReference.
>The reason my diff overlooked these was because I had chosen to ignore the
>DEN column which is one place the error
>is evident and, more pertinently, I had ignored the DataTypeQualifier column
>due to the fact that EF treats this completely differently
>to the published spreadsheets, which therefore masked the difference. (Sue
>and Michael hadn't filtered out the DEN differences
>and so highlighted this error.) . [Also, this was already listed elsewhere
>in our bug-list.]
>The Datatype Qualifiers in the published spreadsheets should be corrected.
>  
>
i agree. the spreadsheets can be easily fixed by refreshing the formula 
calculations on these cells.  for some reason OpenOffice doesn't always 
do this.

>
>9.   Both diffs found the Schema error Payment Means. Payment Means. Code
>which is incorrectly named
>Payment Means. Payment_ Means Type. Payment Means_ Code in the Schemas which
>also have an incorrect
>'tag' name PaymentMeansTypeCode. This was changed to remove the name part
>'Type' during the drafting phase of
>1.0 but to change it now would produce a backwards incompatibility into 1.0.
>I threfore propose that the spreadsheet be changed
>to include the extra word 'Type'. A loose end will then be the name of the
>codelist. This is currently PaymentMeansCode but it
>cannot (as far as I can imagine) be changed so there will be a discrepancy
>which should probably be well documented. The
>documentation will need to be examined for any other mentioning of this
>element name to ensure it is kept consistent.
>(ubl-dev pointed out a discrepancy here.)  Chee-Kai pointed out that the
>list of values included the 'Type' word but
>this will now have stay like that, rather than being corrected as Chee-Kai
>suggested.
>
>Another discrepancy is in the distribution of terms between columns which
>should be corrected in the Schema documentation
>(with no impact on instances)
>  
>
i agree.

again, this points back to the incorrect use of qualifiers for data 
types in the DEN.  but in this case it also appears that schemas have 
changed the whole structure of the name. The correct name for this BBIE 
is "PaymentMeansCode" not "PaymentMeansTypeCode".   the schemas have not 
only added the word "type" to the Property Term but they have created a 
new qualifier!  so instead of this being Property Term = "Payment Means" 
(as stated in the spreadsheet) we have Property Term Qualifer = 
"Payment" and Property Term =  "Means Type"  i am not sure how this 
happened but it reflects a real problem with the schema generation process.
the only practical solution I can suggest is to modify the spreadsheets 
to reflect the tagname given in the schema.  we can do this by changing 
the spreadsheet to say...
Property Term Qualifier = "Payment"
Property Term Possessive Noun = "Means"
and
Property Term Primary Noun= "Type"
This creates a Property Term of "Means Type" qualified by "Payment" 
giving us "PaymentMeansTypeCode" as the UBL Name (and hence tag name). 
 This also  makes it consistent with other UBL BBIEs such as 
TransportMeansTypeCode.
Of course, the schema annotations must also be changed to reflect this 
new naming.

finally, I have no problem with the code list name being different from 
the BBIE - there is no fixed rule that says they should be the same. 
 The code list name is from the qualifer of the data type.

>
>10.   Both diffs found the error in the published Order Change document
>spreadsheet. The Schema is correct. The Amount
>for LineExtensionTotal should be of Datatype UBL_ Amount. Type rather than
>Amount. Type.  The published spreadsheets
>should be changed.
>
>  
>
i agree.  the spreadsheet needs updating.  we also should not forget 
that the DEN needs to be fixed in the schema.  instead of "Order Change. 
Line_ Extension Total. UBL_ Amount" it should be "Order Change. Line_ 
Extension Total. Amount". (As already mentioned, this should happen with 
all other specialized code usages)

>
>
>With regard to my filtering out of spreadsheet DEN formula discrepancies, I
>think it is most important to
>find any errors which might impact on instances. This is not to say that
>DEN's etc are not important to
>get right, of course, for interoperability. I just think we are well aware
>of the limitations and almost to say
>crudeness of the spreadsheet DEN formula and would not really wish it to be
>relied on when Schemas
>can rather use a properly and more extensively programmed formula. Then
>there is consideration of there
>perhaps being some room for interpretation in the production of the DEN
>anyway.
>
>  
>
i agree that trailing punctuation and embedded spaces are regrettable 
but not critical.  however the alignment of naming components between 
spreadsheet and schema is critical.  we need to be careful that the 
schema documentation for the naming of components matches those on the 
speadsheet models.

i also want to re-iterate that the DEN naming rules used in the schemas 
appear to be incorrectly picking up the data type qualifers.  The DEN 
should only be concerned with the Representation NOT the Datatype. 
 these are not synonymous    (good grief!    i am starting to sound like 
Gunther ;-) )

>Stephen Green
>
>
>
>To unsubscribe from this mailing list (and be removed from the roster of the OASIS TC), go to http://www.oasis-open.org/apps/org/workgroup/ubl/members/leave_workgroup.php.
>
>  
>

-- 
regards
tim mcgrath
phone: +618 93352228  
postal: po box 1289   fremantle    western australia 6160