[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [xdi] EUREKA!!! (Or, in current lingo, OMG!!!) New XDI JSONserialization format
Nice work Drummond, a very interesting idea. I see it as an evolutionary path to go down from one of our early JSON formats, flattening our {subj: {pred: [..],..},..} model to one level of keys through the use of XRIs such as =abc/() and the literal delimiter. From an implementation perspective it is definitely easier. Also given any key you should be able to get the XRI identifier for a specific element, say the first, by appending !1, yes? This means you could flatten it further to not have the arrays and simply be a key value store where we have the semantic value of XRI's within the keys. That would then fit nicely with the work I've been talking about in storing an XRI graph as KV pairs, where the key holds the XDI address, in a distributed hash table (DHT) such as TeleHash.
Kind regards,
Bill Barnhill
________________________________________
From: drummond.reed@gmail.com [drummond.reed@gmail.com] On Behalf Of Drummond Reed [drummond.reed@xdi.org]
Sent: Friday, March 25, 2011 6:25 PM
To: OASIS - XDI TC
Subject: [xdi] EUREKA!!! (Or, in current lingo, OMG!!!) New XDI JSON serialization format
Although I am in a car traveling east of the Cascade Mountains in Washington, my wife is driving so I could compose this message before the weekend because I am so excited to share this breakthrough with the TC.
At the very end of yesterday's telecon, we noted that writing up the collection of XDI statements that correspond to each of the example graphs in the XDI Graph Patterns document, both Bill and I were struck by how compact and readable a representation of the graph they represented. And the fact that there is exactly one XDI statement for every arc in the graph is very clean and clear.
Our conclusion on yesterday's call was that this meant there may be a very simple serialization of the graph based directly on the XDI statements.
Last night, to prepare for this morning's OpenXDI telecon, I wrote up the collection of XDI statements for the second example graph ("Simple Properties"), and I started thinking about how this collection would be serialized in JSON. It didn't seem very hard, but I was too tired to work it through.
This morning when I woke up, as soon as my mind returned to the problem, suddenly the solution was obvious: there was a super-simple way to serialize the graph using pure graph statements AND KEEP ALL THE ADVANTAGES OF OUR JSON SERIALIZATION FORMAT.
In other words, there is a much simpler JSON serialization that still supports using all the same native JSON datatypes.
Here are the rules:
1. The entire graph is serialized as one top-level JSON object containing one-or-more JSON objects.
2. Every contained object consists of a key and a value.
3. The key MUST be the first two segments of the XDI statement representing the XDI subject and XDI predicate.
4. The value MUST be an array.
5. If the key ends in the character !, then the value in the array is an XDI literal.
6. If the key ends in any other character, the value(s) in the array are XRIs.
The advantages of this format:
1. It is very simple and flat -- no JSON nesting is needed because all the nesting is reflected in the structure of the XRIs used to create the keys.
2. It is completely indexed, i.e., code can navigate directly to any value in the entire serialized graph just by looking up the key.
3. The JSON datatype returned for all objects is the same (an array), making it easy for code to process. (Mike likes this.)
4. All native JSON datatypes -- of arbitrary complexity -- are fully supported as literal values inside this array.
5. This format is so simple that it will allow us to do away with needing any other format (i.e., X2, X3) for human-readable/writeable serialization of the graph.
In retrospect, there are two secrets behind this dramatically simpler serialization format, neither of which was apparent at Whistler last August:
1. Although we had the seed of the idea at Whistler that we could use an XRI delimiter to signal that an XDI object was a literal, we didn't have the right semantics for what that delimiter was. Now that we have a trailing delimiter (!) for indicating a literal, it is easy.
2. This format uses both the first and second segments of each XDI statement as the key. All of our earlier formats had focused on just using the first segment as a key. Although using the first and second segments creates a small amount of redundant XRI usage in the serialization format, this is strongly offset by the simplicity and speed of processing. (The irony is that using a 2-segment key/value structure for serializing the graph was originally worked out and implemented my Markus about a year ago.)
Below are examples of both the collection of XDI statements and this new JSON serialization of the "Primary and Secondary Addressing" and "Simple Property" example graphs in the XDI Graph Patterns document. (Note that they don't yet reflect a few of the changes discussed on yesterday's and today's calls.)
Please do reply to this message with your thoughts about whether this should become our new official JSON serialization format. If so, I will post a format definition and start updating the XDI Graph Model document.
Have a great weekend,
=Drummond
Primary and Secondary Addresses
XDI Statements
()/$$/(=!0999.a7b2.25fd.c609)
()/$$/(http://example.com/ox/=!0999.a7b2.25fd.c609)
()/()/(=!0999.a7b2.25fd.c609)
()/()/(http://example.com/ox/=!0999.a7b2.25fd.c609)
()/()/=abc
()/()/=!0999.a7b2.25fd.c609
=abc/$/=!0999.a7b2.25fd.c609
=!0999.a7b2.25fd.c609/$$/=abc
=!0999.a7b2.25fd.c609/()/+pea-patch
=!0999.a7b2.25fd.c609/()/+garden
=!0999.a7b2.25fd.c609/()/!1
=!0999.a7b2.25fd.c609+pea-patch/$/=!0999.a7b2.25fd.c609!1
=!0999.a7b2.25fd.c609+garden/$/=!0999.a7b2.25fd.c609!1
Proposed New JSON Serialization
{
"()/$$": [
"(=!0999.a7b2.25fd.c609)",
" (http://example.com/ox/=!0999.a7b2.25fd.c609)"
],
"()/()": [
"(=!0999.a7b2.25fd.c609)",
"(http://example.com/ox/=!0999.a7b2.25fd.c609)",
"=abc",
"=!0999.a7b2.25fd.c609"
],
"=abc/$": [
"=!0999.a7b2.25fd.c609"
],
"=!0999.a7b2.25fd.c609/$$": [
"=abc"
],
"=!0999.a7b2.25fd.c609/()": [
"+pea-patch",
"+garden"
],
"=!0999.a7b2.25fd.c609/()": [
"!1"
],
"=!0999.a7b2.25fd.c609+pea-patch/$": [
"=!0999.a7b2.25fd.c609!1"
],
"=!0999.a7b2.25fd.c609+garden/$": [
"=!0999.a7b2.25fd.c609!1"
],
}
Simple Properties
XDI Statements
()/()/=abc
=abc/+age!/(data:33)
=abc/()/+age
=abc+age/$d!/(data:2010-10-10T11:12:13Z)
=abc+age/()/$v
=abc+age$v/!1!/(data:32)
=abc+age$v/!2!/(=abc/+age!)
=abc+age$v/()/!1
=abc+age$v/()/!2
=abc+age$v!1/$d!/(data:2010-09-09T10:11:12Z)
=abc+age$v!2/$d!/(data:2010-10-10T11:12:13Z)
Proposed New JSON Serialization
{
"()/()": [
"=abc"
],
"=abc/+age!": [
33
],
"=abc/()": [
"+age"
],
"=abc+age/$d!": [
"2010-10-10T11:12:13Z"
],
"=abc+age/()": [
"$v"
],
"=abc+age$v/!1!": [
32
],
"=abc+age$v/!2!": [
"(=abc/+age!)"
],
"=abc+age$v/()": [
"!1",
"!2"
],
"=abc+age$v!1/$d!": [
"2010-09-09T10:11:12Z"
],
"=abc+age$v!2/$d!": [
"2010-10-10T11:12:13Z"
]
}
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]