OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ubl-comment message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [ubl-comment] feedback on the OASIS UBL JSON representation


Thanks very much Steve. 

On Tue, May 30, 2017 at 4:40 PM, Steve Capell <steve.capell@gmail.com> wrote:
Ok no problem 

Then I'll respond in writing :)

Steven Capell
Mob: 0410 437854

On 31 May 2017, at 6:28 am, Chet Ensign <chet.ensign@oasis-open.org> wrote:

Hi Steve, 

I am afraid that a conversation with the Technical Committee is not possible under OASIS rules for the reasons set forth here. 

When members join OASIS and sign our Membership Agreement, they agree to be bound by the OASIS IPR Policy (https://www.oasis-open.org/policies-guidelines/ipr) and then again, when they join a Technical Committee they agree to be bound by the specific IPR mode chosen by the TC (Royalty Free on Limited Terms in the case of UBL). By these agreements, they grant OASIS and the TC certain rights with respect to their IP contributions and their participation that, downstream, provide implementers with assurances that they can safely use the resulting specification. 

People outside OASIS have made none of these commitments. That is why we require feedback to be provided through the comment facility, in the case of UBL, the ubl-comment@ mailing list. People are notified as part of the subscription process that they are agreeing that the committee can use their feedback under the same rules. (This is part of the reason that we can't accept links as submissions. There is no straightforward way to know in the future what exactly was contributed as feedback at a particular point in time.) 

The only ways to directly interact with the committee are either through the comment facility or to join the TC so as to make the necessary commitments. 

I appreciate this sounds bureaucratic, especially as you are working in an open source project, but it is how we have to do things in order to protect our TCs and the work they produce under our program of governance. 

Best regards, 

/chet


On Sun, May 28, 2017 at 5:01 PM, Steve Capell <steve.capell@gmail.com> wrote:
Hi guys 

Thanks for your review and feedback 

I think something like a teleconference might be handy as what is evident is that our fundamental goals for the specification are different, even though we both are defining a JSON syntax 

Steven Capell
Mob: 0410 437854

On 28 May 2017, at 11:56 pm, Michael Onder <mike.onder@gmail.com> wrote:

Ken,

Does the info in the below paragraph fit with the interopability we are tr8ying to achieve between freaight data exchange standards, like EDIFACT, GS1, ANSI X12 etc.?

Any semantic interoperability is subject to the semantic interpretation of applications acting on the different choice of syntax.  Accordingly, our choices in JSON _expression_ support semantic-free transliteration.  This would accommodate a process, such as one supported by an Access Provider on a network, to transliterate between XML and JSON syntaxes without knowledge of the underlying semantics.

Mike

Michael P. Onder
9751 Rehanek Ct.
Burke VA 22015
7036443867 HOME
7035984950 CELL
mike.onder@gmail.com

On May 28, 2017 8:44 AM, "G. Ken Holman" <gkholman@cranesoftwrights.com> wrote:
Thank you, again, Steve, for your project's considered formal feedback to the UBL Technical Committee regarding our specification for a JSON alternative representation for UBL.

The technical committee has prepared the agreed-upon response below to address the points that were raised.  At the end you will find our invitation for you to continue the dialogue on this topic.  Please let us know if you think anything was overlooked.

. . . . . . . Ken

Regarding "the primary goal of an OASIS UBL JSON specification should be to remain developer friendly so that it is usable just like any other JSON based APIs on the web", in fact, this was not the Technical Committee's primary goal by choice.  From section 1.1 Overview the specification states "intent is only to convey in syntax the information content reflecting the same abstract model of the UN/CEFACT Core Component Technical Specification 2.01 [CCTS] with which the document model was designed. Accordingly, and in parallel to an application's use of XML syntax, the JSON syntax used is generic in nature and is neither streamlined nor optimized for any particular application's objectives.  As one would undertake the unmarshalling of XML syntax into internal application data structures suitable for processing, one must also undertake the unmarshalling of JSON interchange syntax into whatever internal application data structures (or other JSON representations) of the content that are suitable for the task at hand."

And so, the intention of the JSON Alternative Representation is to specify a syntax suitable for interchanging data between different applications, while it is expected that APIs will be streamlined and optimized for the objectives of particular applications.  Thus it is not considered in the committee's purview to presume any particular aspect of UBL semantics would be better expressed using a less generic choice of JSON constructs suitable for interchange.  We note that the OASIS Business Document Naming and Design Rules (BDNDR) is not used just for UBL but can be used for any business vocabulary designed with CCTS constructs.  Thus, the syntactic choices in BDNDR are based on CCTS principles of granularity and cardinality, not on UBL nor any other semantic interpretation of the information expressed using CCTS.

Regarding "must be possible to transform XML <-> JSON so that semantic interoperability is still feasible even when one party supports JSON and the other XML.", machine transliteration between JSON and XML without semantic awareness is critical to blinkered syntax interoperability.  Choices made must allow information to be round-tripped losslessly without needing to know the underlying meaning or customized _expression_ of any particular construct.

Any semantic interoperability is subject to the semantic interpretation of applications acting on the different choice of syntax.  Accordingly, our choices in JSON _expression_ support semantic-free transliteration.  This would accommodate a process, such as one supported by an Access Provider on a network, to transliterate between XML and JSON syntaxes without knowledge of the underlying semantics.

A quick review of an example AusDigial approach to the JSON _expression_ of CCTS for UBL reveals a case in point of presuming semantic concepts in _expression_:

  http://ausdigital.org/specs/ausdigital-syn/2.0/#ccts-texttype
  "JSON representation SHALL NOT specify language for every string
   data element. Instead it will use the UBL document level language
   indicator "Language":"EN" using ISO 639-1 language codes"

Upon receipt of a UBL document following the BDNDR _expression_ of a JSON CCTS text field, an AusDigital-compliant process can choose what to do with an example as follows, which would not be possible to be round-tripped back to XML:

  http://docs.oasis-open.org/ubl/UBL-2.1-JSON/v1.0/cnd02/UBL-2.1-JSON-v1.0-cnd02.html#S-BBIE-_expression_
   "Note":[{"TextContent":"Test","LanguageIdentifier":"en"},
           {"TextContent":"Tester","LanguageIdentifier":"fr"}]

Regarding "Processing code is a mess", we cite our above response that the format for interchange is expected to be interpreted and translated into a format suitable for processing, in particular, the kind of expected processing for the semantics of the information.  Such assumptions can be tailored and tuned to the recipient's expectations, without the sender having to know the assumptions a priori.

Regarding "Potential data loss", future backward compatibility is not explicitly covered in BDNDR because BDNDR does not impose such constraints.  However, the UBL use of BDNDR does impose such constraints, and these are enumerated in detail to enforce future backward compatibility as described in our UBL Governance document:

  http://docs.oasis-open.org/ubl/UBL-Governance/v1.0/cn01/UBL-Governance-v1.0-cn01.html#S-PRESERVING-BACKWARD-COMPATIBILITY
  "Therefore any proposed changes to existing elements should be as follows:
   - from optional singular to optional multiple
   - from mandatory singular to mandatory multiple
   - from mandatory singular to optional singular
   - from mandatory singular to optional multiple
   - from mandatory multiple to optional multiple"

Each of the examples cited in your comment appear to avoided by the constraints UBL imposes in its use of BDNDR.

Regarding "Namespace URI properties", it is acknowledged such have not a lot of use in JSON beyond identification of the dictionary of semantics behind the labels (which your users could presumably assume to be UBL), but are critical to the blinkered syntax interoperability of BDNDR between XML and JSON.  Without the preservation of this information from XML, an Access Point wouldn't be able to recreate the XML without knowing the JSON to be UBL.  As BDNDR would easily be used for non-UBL information such as medical health records, border control records, or, really, any interchange document, each JSON _expression_ would need to carry with it the suite of namespaces applicable to the XML _expression_ of CCTS as described by BDNDR.

Moreover, the suggested cited JSON approach of a single "$id" URI would not accommodate the need for multiple URIs supporting round-trip blinkered transliteration.

The UBL TC welcomes your feedback regarding these observations to your comments.

We hope these comments reflect your continuing ability to express UBL documents internally in your applications using JSON following your own AusDigital conventions, provided that whatever interface is used to the outside world when exchanging JSON files would support through interpretation and translation the existing BDNDR conventions tailored for the blinkered interchange of CCTS-based documents.


At 2017-05-11 06:35 +1000, Steve Capell wrote:
Hi Chet

I acknowledge that the attached email is the substance of our contribution

Although there are a further 6 links at the bottom of the principles section.  What about those?

Steven Capell
Mob: 0410 437854

On 11 May 2017, at 12:29 am, Chet Ensign <<mailto:chet.ensign@oasis-open.org>chet.ensign@oasis-open.org> wrote:

Hi Steven,

I'm Chet Ensign, the person in charge of helping the TC's do their work within the OASIS rules. My thanks too for the detailed review and comments.

One of the details of the rules is that contributions cannot be made via links. Feedback must be in or attached to the message itself. No need to spend time on why unless you'd like to know. I'm just writing to ask if you can confirm that the following text is the content of the contribution:

--- Begin content ---



OASIS UBL JSON Specification Review





OASIS has <https://www.oasis-open.org/news/announcements/ubl-2-1-json-alternate-representation-v1-0-and-business-document-naming-and-desig>released a specification for a JSON version of the UBL document library. The purpose of this page is to privide formal feedback to the technical committtee.

We have tested the specification and have encountered some difficulties that we would like to share - together with proposed improvements.



Principles.





JSON has seen significant uptake accross the web over the last decade - to the extent that the vast majority of new services are provided as REST/JSON APIs, usually with some form of OAuth2 based authentication. The reason is that it's simple and developer freindly and yet powerful enough for most needs.

We would imagine that the rationale for the OASIS technical committee to develop a JSON representation of UBL is so that UBL can ride the JSON adoption wave and thereby extend it's reach and adoption. In that case, the primary goal of an OASIS UBL JSON specification should be to remain developer freindly so that it is usuable just like any other JSON based APIs on the web. We would suggest that there are two strategic goals for a JSON representation:
   * That it must be developer freindly so that UBL get uptake in the REST/JSON world.
   * That it must be possible to transform XML <-> JSON so that semantic interoperability is still feasible even when one party supports JSON and the other XML.

We started with those goals and found that there is an inevitable choice that must be made:
   * You can design a simple transformation (XML <-> JSON) but you'll end up with complex and un-freindly JSON. Or
   * You can design a more complex transformation in order to get a simpler and more developer freindly JSON.

Since the transformation need only be designed once (and can be made freely avaialble) but dealing with complex JSON will impact every developer that attempts to use it, we chose the latter option. This public repository hosts our version of the UBL JSON syntax specification.
   * Our <http://ausdigital.org/specs/ausdigital-syn/2.0/>JSON Syntax Specification
   * Some <https://github.com/ausdigital/ausdigital-bill/tree/master/resources/ausdigital-syn/2.0/samples/Invoice>UBL JSON sample invoice documents
   * The <http://ausdigital.org/docson.html#https://raw.githubusercontent.com/ausdigital/ausdigital-bill/master/resources/ausdigital-syn/2.0/spec/Invoice.json>browsable JSON invoice Schema
   * The <https://github.com/ausdigital/ausdigital-bill/tree/master/resources/ausdigital-syn/1.0/samples/Invoice>XML versions of the same documents
   * The <http://testpoint.io/syn>lossless transformation API
   * A <http://ausdigital.org/specs/ausdigital-bill/1.0/>semantic specification that works with both XML and JSON versions



Some Specific Feedback from our Developers





About the OASIS UBL JSON representation.



Processing code is a mess:





To get IdentifierContent you need to go through 5 loops:

ROOT.Invoice[i].InvoiceLine[j].Item[k].StandardItemIdentification[l].ID[m].IdentificationSchemeIdentifier

If to define properties as objects when maxOccurs="1", you will need only 1 loop:

ROOT.Invoice.InvoiceLine[i].Item.StandardItemIdentification.ID.IdentificationSchemeIdentifier

Of course, you can always access items but their index without loop if you know that maxItems = 1:

ROOT.Invoice[0].InvoiceLine[i].Item[0].StandardItemIdentification[0].ID[0].IdentificationSchemeIdentifier

But to know this you need to go to json schema and check.

This is very likely that a lot of developers will generate POJOs from json schema and they will get a bunch of classes, where each property is a List.



Potential data loss





The main idea of the specification:
Future versions of UBL may raise the cardinality of any given object and so this approach is future-proof to revisions. This approach is also consistent and is used without thought of cardinality and so should be easier for the programmer (if a little verbose). Although a change in cardinality does not apply to the Document ABIE, the array notation is used for consistency with ASBIE expressions using arrays.

It works for cardinality changes in different UBL versions, but only partially.

For example, in UBL-Invoice-2.1.xsd:

<xsd:element ref="cac:InvoicePeriod" minOccurs="0" maxOccurs="unbounded"/>

And let's assume in UBL-Invoice-2.2.xsd maxOccurs changes from "unbound" to "1":

<xsd:element ref="cac:InvoicePeriod" minOccurs="0" maxOccurs="1"/>

In this case in JSON schema UBL 2.1:


"InvoicePeriod": {
        "title": "Invoice. Invoice_ Period. Period",
        "description": "A period to which the Invoice applies.",
        "items": {
                "$ref": "../common/UBL-CommonAggregateComponents-2.1.json#/definitions/InvoicePeriod"
        },
        "minItems": 1,
        "additionalProperties": false,
        "type": "array"
},


And in 2.2:


"InvoicePeriod": {
        "title": "Invoice. Invoice_ Period. Period",
        "description": "A period to which the Invoice applies.",
        "items": {
                "$ref": "../common/UBL-CommonAggregateComponents-2.1.json#/definitions/InvoicePeriod"
        },
        "maxItems": 1,
        "minItems": 1,
        "additionalProperties": false,
        "type": "array"
},


The good point is that property type hasn't been changed - it remains array. That means that no need to re-write code which is parsing JSON documents.

But if cardinality change is of the other type - increase maxOccurs.

For example:

<xsd:element ref="cac:OrderReference" minOccurs="0" maxOccurs="1"/>

and



"OrderReference": {
        "title": "Invoice. Order Reference",
        "description": "A reference to the Order with which this Invoice is associated.",
        "items": {
                "$ref": "../common/UBL-CommonAggregateComponents-2.1.json#/definitions/OrderReference"
        },
        "maxItems": 1,
        "minItems": 1,
        "additionalProperties": false,
        "type": "array"
}



Let's assume maxOccurs is changed from "1" to "unbound":

<xsd:element ref="cac:OrderReference" minOccurs="0" maxOccurs="unbound"/>
In this case, we get for an array with unlimited number of or items:



"OrderReference": {
        "title": "Invoice. Order Reference",
        "description": "A reference to the Order with which this Invoice is associated.",
        "items": {
                "$ref": "../common/UBL-CommonAggregateComponents-2.1.json#/definitions/OrderReference"
        },
        "minItems": 1,
        "additionalProperties": false,
        "type": "array"
}



Again, the type remains the same, so code, which is processing JSON will still be working. But here is the potential issue - all the items except first will be ignored, if a developer will not update processing logic, which is very likely as the code is working.

If increasing maxOccurs is not a valid case for newer versions, it is highly recommended to defined properties with maxOccurs="1" as objects and not as arrays.

Namespace URI properties





Although they are optional still they do not make much sense and likely will not be processed as they do not add anything to validation against json schemas. Lets to schema references the standard JSON way rather than a UBL specific way:

It would be good to add recommendation for using json schemas according to <http://json-schema.org/latest/json-schema-core.html#rfc.section.9>http://json-schema.org/latest/json-schema-core.html#rfc.section.9

--- end content ---

Thanks very much for confirming.

Best regards,

/chet ensign



On Tue, May 9, 2017 at 10:25 PM, steve capell <<mailto:steve.capell@gmail.com>steve.capell@gmail.com> wrote:
Dear working group,
As the REST/JSON technical working group that is part of the Australian e-invoicing initiative led by the digital business council in conjunction with the australian tax office, we would like to offer this feedback on the OASIS UBL JSON representation.

<https://github.com/ausdigital/ausdigital-syn/blob/master/OASIS-UBL-JSON-Specification-Review.md>https://github.com/ausdigital/ausdigital-syn/blob/master/OASIS-UBL-JSON-Specification-Review.md

<http://ausdigital.org>ausdigital.org specifications are licensed under <https://www.gnu.org/licenses/gpl-3.0.en.html>GPL3 and governed in accordance with <http://www.digistan.org/spec:1/COSS>COSS
Kind regards,
Steven Capell
Chair, REST/JSON technical working group
Digital Business Council




--

/chet
----------------
Chet Ensign
Director of Standards Development and TC Administration
OASIS: Advancing open standards for the information society
<http://www.oasis-open.org>http://www.oasis-open.org

Primary: +1 973-996-2298
Mobile: +1 201-341-1393


--
Contact info, blog, articles, etc. http://www.CraneSoftwrights.com/o/ |
Check our site for free XML, XSLT, XSL-FO and UBL developer resources |
Streaming hands-on XSLT/XPath 2 training class @ US$45 (5 hours free) |


--
This publicly archived list offers a means to provide input to the
OASIS Universal Business Language (UBL) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: ubl-comment-subscribe@lists.oasis-open.org
Unsubscribe: ubl-comment-unsubscribe@lists.oasis-open.org
List help: ubl-comment-help@lists.oasis-open.org
List archive: http://lists.oasis-open.org/archives/ubl-comment/
Feedback License: http://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
Committee: http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=ubl
Join OASIS: http://www.oasis-open.org/join/




--

/chet 
----------------
Chet Ensign
Director of Standards Development and TC Administration 
OASIS: Advancing open standards for the information society
http://www.oasis-open.org

Primary: +1 973-996-2298
Mobile: +1 201-341-1393 



--

/chet 
----------------
Chet Ensign
Director of Standards Development and TC Administration 
OASIS: Advancing open standards for the information society
http://www.oasis-open.org

Primary: +1 973-996-2298
Mobile: +1 201-341-1393 


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]