Two important tweaks to simplified JSON serialization format

[Drummond Reed <drummond.reed@xdi.org>]

Per another action item I had from two telecons back, I have documented the new simplified JSON serialization format at http://wiki.oasis-open.org/xdi/JsonFormat.

Since it's so short, here is the entire ruleset:

1. An XDI JSON graph serialization MUST be valid JSON according to RFC 4627.

2. The graph MUST be serialized as a single top-level JSON object ("root context object").
3. Every XDI statement in the graph MUST be serialized as a JSON object ("context object") contained within the top-level JSON object.
4. For every context object, the string representing the JSON object key MUST consist of the first two segments of the XDI statement, i.e., the XRIs representing the XDI subject and XDI predicate. The key MUST include the forward slash separating the XDI subject and XDI predicate, and MUST NOT include a trailing slash after the XDI predicate.
5. For every context object, the value MUST be a JSON array.
6. If the XDI predicate is more than one subsegment and the final subsegment is “!”, then the predicate MUST be a literal arc and the value of the JSON array MUST be interpreted as an XDI literal.
7. If the XDI predicate is “()”, then the predicate MUST be a contextual arc and the value of the JSON array MUST be an array of strings representing XRIs.
8. Any other XDI predicate MUST be a relational arc and:
   1. If a value in the array is a string, it MUST be interpreted as an XRI.
   2. If a value in the array is a JSON object, it MUST be interpreted as a nested XDI graph in which all the XRIs MUST be expressed as cross-references.

In doing the messaging examples, I realized that we needed the two tweaks represented in bold above:

1. First, in rule 6, ! is only an indicator of a literal if is the final subsegment of a multi-subsegment predicate. In other words, the XDI predicates ! and $! when used alone do not indicate literals. They are used for dictionary definitions.
2. Second, in rule 8.2, the value of an XDI object described by a relational arc may be an entire subgraph. This in fact is the case with any $add and $mod message. In the XRI statement serialization, that just means each of the objects is represented as an XRI cross-reference. Here's an example from the new XDI Messaging Patterns page:

@!9999!8888$msg!1234/$()/(=!1111!2222) 
@!9999!8888$msg!1234/$d!/(data:,2011-04-10T22:22:22Z) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3/()/+tel) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3+tel/!1!/(data:,+1.206.555.1111)) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3+tel/!2!/(data:,+1.206.555.2222)) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3+tel/+home/!1!) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3+tel/+work/!2!) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3+tel/+work+fax/!2!) 
@!9999!8888$msg!1234$do/$add/(=!1111!2222!3/+birthdate/(data:,1957-03-12))

Note that all the objects of the $add predicate are cross-references. That's because the graph being added is an entire nested subgraph.

It's very straightforward in the XDI statement serialization. But our original simplified JSON serialization specified that everything must be serialized as a flat single-level set of JSON objects. That would force all the literals in a nested subgraph to be XRI-encoded. For example, here would be the JSON serialization of the above graph:

{
    "@!9999!8888$msg!1234/$()": [
        "(=!1111!2222)" 
    ],
    "@!9999!8888$msg!1234/$d!": [
        "2011-04-10T22:22:22Z" 
    ],
    "@!9999!8888$msg!1234$do/$add": [
        "(=!1111!2222!3/()/+tel)",
        "(=!1111!2222!3+tel/!1!/(data:,+1.206.555.1111))",
        "(=!1111!2222!3+tel/!2!/(data:,+1.206.555.2222))",
        "(=!1111!2222!3+tel/+home/!1!)",
        "(=!1111!2222!3+tel/+work/!2!)",
        "(=!1111!2222!3+tel/+work+fax/!2!)",
        "(=!1111!2222!3/+birthdate/(data:,1957-03-12))" 
    ]
}

Notice all the bold text. That's really ugly, because the whole point of the JSON serialization format is so data can be serialization in native JSON data formats and not forced to be XRI encoded (which means being URI encoded first, so it's really two layers of unnecessary data encoding).

What the above serialization should really look like is this:

{
    "@!9999!8888$msg!1234/$()": [
        "/(=!1111!2222)" 
    ],
    "@!9999!8888$msg!1234/$d!": [
        "(data:,2011-04-10T22:22:22Z)" 
    ],
    "@!9999!8888$msg!1234$do/$add": [
        {
            "(=!1111!2222!3/()": [
                "+tel" 
            ],
            "=!1111!2222!3+tel/!1!": [
                "+1.206.555.1111" 
            ],
            "=!1111!2222!3+tel/!2!": [
                "+1.206.555.2222" 
            ],
            "=!1111!2222!3+tel/+home": [
                "!1!" 
            ],
            "=!1111!2222!3+tel/+work": [
                "!2!" 
            ],
            "=!1111!2222!3+tel/+work+fax": [
                "!2!" 
            ],
            "=!1111!2222!3/+birthdate": [
                "1957-03-12" 
            ] 
        } 
    ]
}

That's much cleaner, and forces zero XRI encoding of literal data.

Thankfully, once I noticed this problem, the solution was very simple: adding the rule that in any JSON array that is the object of an XDI relational predicate, if a value in the array is a JSON string, then it is an XRI, and if it is a JSON object, then it is a nested subgraph. If so, every XDI statement in that subgraph is automatically "unwrapped" from being a cross-reference when it is serialized into JSON and "rewrapped" into an XRI cross-reference when it is converted back to XDI statement format.

Pretty clean solution, no?

=Drummond