Re: [ubl-dev] ubiquitous UBLExtensions: a proposal

[David Goodenough <david.goodenough@broadwellmanor.co.uk>]

From the replies so far it would appear that extensions fall into three groups.

 

1) Those can safely be ignored. An example of this is process information put in by something like a SAP system which is of use to another SAP system but has no general semantic meaning for the document and can therefore safely be ignored.

 

2) Those that we need to be able to display and possibly reply to (by including related extension objects in reply documents). They are optional, but we have to have software support for them at the receiving end, so if that is missing any document containing such extensions MUST be rejected.

 

3) Required extensions. Such extensions might be needed by reason of local legislation, but the need for them to be required may only be for some document types (say only for Invoices) and for some DigitalAgreements (it might only be required for within country transactions, but only be optional for international transactions). In this case if the extension is missing (see below) the transaction MUST be rejected.

 

In the first and second cases there is no need to say where these extensions may exist, as the software support can make that call. However for the third case there is a choice to be made. The software can be called on to validate its presence and can say that the requirement is met because the transaction in question is a DepatchAdvice and it is missing but unnecessary, but that it is not met if the transaction in question is an Invoice and it is missing if this is an Invoice only extension, or we can introduce a presence grammar. My preference would be that handling the "requiredness" is a matter for the software to implement from the specification not the UBL datastream to enforce.

 

So a field in the PermittedExtension object would be needed to say that this extension is Ignorable, Needed or Required (or some similar options).

 

I have to say that the Columbian example is an example of something that from the description given I believe has no place inside the UBL datastream. This data has no life within any UBL process, and is only used as side information when sending the invoice to the tax administrator. This process is akin to internal approval procedures, it just happens to go to the tax admin, and UBL (currently) has little or nothing to say about internal procedures. Surely a better approach would have been to wrap the Invoice in an outer structure which contained all the required information and way if you needed to make an audit trail the tax admin could counter sign the invoice, and that counter signature could be propagated to the receiver so that they could be sure that all the invoices they received had been properly authorized. But that is merely my observation based on the description given, so it can be ignored.

 

David

 

On Tuesday, 27 August 2019 11:52:46 BST David Goodenough wrote:

Following on from my comments on the UBL 2.3 draft I would like to offer a draft proposal which I think might solve the problem.

 

I appreciate that UBL is not a recognised international standard for legal documents, but it does replace what would otherwise be legal documents and for ease of end user comprehension it would be easier if the same basic rules applied to paper and electronic documents. In addition it seems a reasonably aspiration for UBL documents to be recognised as the same legal status as paper ones.

 

The problem arises because a generic client will not be able to read and therefore present to the user or to an API into other systems items that are encoded in the UBL extension for which it has no support. The extensions are therefore hidden from the user which is legally undesirable. Note that the Signature extension as described in the UBL documentation is not included in this discussion as it is well defined and publicly documented.

 

In the notes below upper cased words such as MUST or SHOULD are intended to be interpreted following the notes in IETF RFC 2119.

 

The proposed solution to this problem involves adding a single element to DigitalCapability and DigitalAgreement documents which describes the permitted extensions that can be used (and where they can be used) in document exchanges sanctioned by this DigitalAgreement. This element is tentatively called PermittedExtension, and is minOccurs="0" maxOccurs="unbounded". If there is no PermittedExtension object in the final DigitalAgreement then any document that is received which contains a UBLExtensions clause other than the Signature extension document in the UBL 2.3 documentation MUST be rejected by the receiver. This maintains the current behaviour on all but the document level tags. Whether it should apply immediately to the existing document level tags is an open question, my instinct is that it should as it will be difficult to increase the enforcement later.

 

I would suggest, but I have only given 18 hours thought to this, that these PermittedExtension objects could be included in the DigitalProcess or DigitalCollaboration objects as they are they are about a conversation and the same rules should apply to both ends of the conversation.

 

A PermittedExtension object MUST contain an ID which defines the name of this extension and the controlling agency. It MUST also include a URI for the schema of the extension and a URI for a narrative document describing the meaning of the extension. Both these URIs and the documents they point at MUST be publicly and freely accessible and usable - the rational for this is that if subsequent inspections of such documents by outside bodies (be they governmental, trade assurance schemes, or such as auditors ) must be possible without let or hindrance. A PermittedExtension MUST also include a minimum and maximum version number of the extension, so that the handling of the corresponding UBLExtension (I hope these are versioned if not they need to be) can be ensured. Finally the PermittedExtension object MUST contain a list of UBL objects within which the extension can be used. It might be useful for an extension to be used on for instance all Party objects where ever they are used, but it might be that they should only be used in a say DespatchAdvice object. Do we need a UBL version check in here as well?

 

If a receiver's software receives a DigitalCapability or draft DigitalAgreement (the one in response to a DigitalCapability) which requests a PermittedExtension that it is not equipped to handle the document MUST be rejected with an ApplicationResponse including the PermittedExtension that can not be accepted. The sender's system MAY then amend the DC or DA to remove the unacceptable PermittedExtension or it MAY terminate the conversation. The sender's system SHOULD NOT re-send a refused PermittedExtension unless the receiver's system has been updated to accept it. Coordinating this update notification is outside the scope of UBL communications.

 

If a receiver's software receive a UBL document which contains one or more UBLExtension objects that are not included in the list of PermittedExtension objects in the current DigitalAgreement the software MUST reject the document with an ApplicationResponse including the un-permitted UBLExtension.

 

One question that worries me is how we do schema validation in this environment. Doing schema validation is desirable as it makes the code easier to write if the datastream is known to obey the schema, but I am unsure (I simply have not had time to look) whether schema validating XML loaders have the ability to merge schemas in this fashion. This is an area that needs further research.

 

I hope that this document is a useful starting point (along with my original objection) to resolve this problem. My preference would be either to define and include a PermittedExtension like object in 2.3, or if it is to be deferred to 2.4 that the UBLExtension proliferation that occurs in the 2.3 draft be withdrawn until the PermittedExtension functionality can be included.

 

David