Review of Two Diffs (Michael/Sue's and Stephen's)

["Stephen Green" <stephen_green@seventhproject.co.uk>]

Here is my assessment of the differences found using the 'diff' technique,
and my resolution
proposals/suggestions. There are ten in all.


1.   Address. Street Name. Name    vs    Address. Street Name. Text
This appears in both sets of diffs.

Proposed amendment   -   change published spreadsheet to Address. Street
Name. Name
Reason    this will match Address. Additional_ Street Name. Name


2.   Michael and Sue's spreadsheet picked up on
Address. Additional_ Street Name. Name, etc   -   published spreadsheets
have extra spaces after the underscore in the DEN's

This is a known issue which has been pointed out many times before and
appears to be due to a 'bug' in the formula generating the DENs in the
published
spreadsheets. In the past the DEN's in Schemas were not taken from the
spreadsheet
so this was just overlooked. Now the DEN's still are not taken from the
spreadsheet
so I overlooked it as before.  The effort to put this right might be out of
proportion to the
importance and the impact, if any, of leaving it. I propose to ignore it for
now.


3.   Michael and Sue's work picked up on the extra dot at the end of ASBIE
DEN's
e.g. Address. Address Line. in the published spreadsheets but not the
Schemas (labelled 'rt' in their work)
I ignored it as too minor to warrant discussion since, again, the DEN's are
generated
and not normative. It might be worth fixing the formula to correct this but
experience
from 'beta' with the formulas makes me think it might be more trouble than
the impact
warrants.


4.   Both sets of diffs picked out the error in Allowance Charge. Reason.
Code
This has little significant impact. I suggest amending the spreadsheet
because that
avoids changing the Schemas but I accept that it could be argued that the
Schemas
could be changed because the change is to the documentation only. The 'tag'
name
is unaffected.


5.   Michael and Sue's worked picked up on the dropping of 'Currency_' from
the
DEN of Allowance Charge. Currency. Currency_ Code in the published
spreadsheets

I don't know which is correct but I think this is particularly worth
changing if the error is in
the Schemas since they are normative. However, it would be worth changing
the spreadsheets
if it was clear which version was correct (from the CCTS). If it is unclear
and the Schemas
don't need to be changed then I'd change the spreadsheets to keep them
consistent with the Schemas


6.   My diff ommitted the differences in the DEN's for Allowance Charge.
Amount since
this difference only occurs in the DEN and not in the other column values.
Michael and Sue's
diff picked up on it. I would propose that since this requires a change to
the formula which generates
DEN's it is probably not worth fixing, except to add a note to the
spreadsheets or package documentation
to say that the spreadsheet DEN's should not be treated as normative or
relied on. Perhaps the values from the Schemas
should be those which are submitted to TBG17 if it is not too late. I do
anticipate push-back on this but I'm
bearing in mind with this and all such spreadsheet formula errors or
inadequacies, that a spreadsheet formual has
limited 'granularity' or precision when compared to a scripting or
programming language for generating such strings.


7.   Both diffs picked up on a high-impact difference. Exchange Rate.
Operator. Code
The impact of this is high because it has resulted in the loss of the
Operator codelist from the CAC Schema.
This needs a correction to the CAC Schema. EF has it just as Datatype 'Code.
Type' but it should be
'Operator_ Code. Type' which would bind it to the operator codelist.
Backwards compatibility might be barely affected
if the NDR rule is followed which requires new namespaces anyway. This
correction would require that that be
done anyway (as we are discussing on ubl-dev with regard to restrictions of
'bound' codelists.) The impact of
the new namespaces on backwards compatibilty is obvious but acceptable, I
would think, in the move from cd
to specification (I take it that is what we are doing at this stage - if not
then we have another problem, we need a second
'cd' namespace, probably.)


8.    There are two obvious errors in the spreadsheets published, both
similar, but both can be corrected in the spreadsheets
without affecting anything else:  DocumentDocumentStatusCode   and
IssueIssueDateDate, both in OrderReference.
The reason my diff overlooked these was because I had chosen to ignore the
DEN column which is one place the error
is evident and, more pertinently, I had ignored the DataTypeQualifier column
due to the fact that EF treats this completely differently
to the published spreadsheets, which therefore masked the difference. (Sue
and Michael hadn't filtered out the DEN differences
and so highlighted this error.) . [Also, this was already listed elsewhere
in our bug-list.]
The Datatype Qualifiers in the published spreadsheets should be corrected.


9.   Both diffs found the Schema error Payment Means. Payment Means. Code
which is incorrectly named
Payment Means. Payment_ Means Type. Payment Means_ Code in the Schemas which
also have an incorrect
'tag' name PaymentMeansTypeCode. This was changed to remove the name part
'Type' during the drafting phase of
1.0 but to change it now would produce a backwards incompatibility into 1.0.
I threfore propose that the spreadsheet be changed
to include the extra word 'Type'. A loose end will then be the name of the
codelist. This is currently PaymentMeansCode but it
cannot (as far as I can imagine) be changed so there will be a discrepancy
which should probably be well documented. The
documentation will need to be examined for any other mentioning of this
element name to ensure it is kept consistent.
(ubl-dev pointed out a discrepancy here.)  Chee-Kai pointed out that the
list of values included the 'Type' word but
this will now have stay like that, rather than being corrected as Chee-Kai
suggested.

Another discrepancy is in the distribution of terms between columns which
should be corrected in the Schema documentation
(with no impact on instances)


10.   Both diffs found the error in the published Order Change document
spreadsheet. The Schema is correct. The Amount
for LineExtensionTotal should be of Datatype UBL_ Amount. Type rather than
Amount. Type.  The published spreadsheets
should be changed.




With regard to my filtering out of spreadsheet DEN formula discrepancies, I
think it is most important to
find any errors which might impact on instances. This is not to say that
DEN's etc are not important to
get right, of course, for interoperability. I just think we are well aware
of the limitations and almost to say
crudeness of the spreadsheet DEN formula and would not really wish it to be
relied on when Schemas
can rather use a properly and more extensively programmed formula. Then
there is consideration of there
perhaps being some room for interpretation in the production of the DEN
anyway.


Stephen Green