Thanks Hubert for the reply.
Would you please share the current stage of the discussion on compact JSON ? I was only able to find this
document which is relatively empty.
There is a fundamental difference between MessagePack and ProtoBuf. In ProtoBuf message format is defined upfront and needs to be known and carefully versioned for compatibility. MessagePack, has a fixed but very flexible and simple model
type system , and yes, arrayâs are length prefixed similar to all variable length structures like maps, strings, â). In that sense this is similar to JSON and XML where typically there is a second step in Memory that transforms message into Programming
In summary, my impression is that MessagePack is so much closer to JSON that I would expect that it should be easier for any OData implementation to add such a format.
As mentioned above, I do not have any insights into the discussion on compact JSON but it might also be worth investigating compression on top of a binary format.
Happy to help with any investigations.
From: Hubert Heijkers <Hubert.Heijkers@nl.ibm.com>
Sent: Sunday, August 2, 2020 09:19
To: Christof Sprenger <firstname.lastname@example.org>
Subject: [EXTERNAL] Re: action item 3913 - Concept for binary data format
I still have a note here to reach back out to Mike as well because he mentioned ProtoBuf a couple of months ago by now as well in this same realm.
For me this has always been an action item that was closely related to the Compact JSON work, hoping to iron out how we'd be able to specify very concisely what's to be expected in the payload in Compact JSON already
and then come up with a representation in which we could use a more binary (read: less human readable) format to optimize the communication further. Back then I'd looked at a bunch of binary formats that would potentially make sense, MessagePack and ProtoBuf
(the later one even more so since we use build on GRPC), but the time couldn't find any that I liked because either 1) they didn't have a separation between metadata data and a data file, 2) didn't support collections without needing to know the size up front
easily (a definite no go for us) or 3) didn't feel we could easily be embedded in our serialization abstraction logic.
So to answer your question, other then early investigations, I've not gotten to this again, wasn't planning to until we'd have taken the next step on Compact JSON (which is effectively taking my proposal - which
we have in production - and merge in all the ideas/thoughts we had since and come up with a normative proposal for it).
Hubert Heijkers | STSM, Chief Architect TM1 Server and OData Evangelist | IBM Business Analytics | +31-621-394123
"The crisis of today is the joke of tomorrow" - H.G. Wells
Christof Sprenger ---31-07-2020 08:29:54 PM---Hubert, We recently stumbled
upon MessagePack<INVALID URI REMOVED
From: Christof Sprenger <email@example.com>
To: Hubert Heijkers <firstname.lastname@example.org>
Cc: "email@example.com" <firstname.lastname@example.org>
Date: 31-07-2020 08:29 PM
Subject: [EXTERNAL] action item 3913 - Concept for binary data format
We recently stumbled upon
MessagePack and I remembered the action item
3913 which seems to be assigned to you.
May I ask in which stage this action item is? I would love to explore if/how Message Pack fits into this since it appears to be at first sight so similar to JSON that this should give a nice balance between efficiency and compatibility.
Tenzij hierboven anders aangegeven: / Unless stated otherwise above:
IBM Nederland B.V.
Gevestigd te Amsterdam
Inschrijving Handelsregister Amsterdam Nr. 33054214