Re: [xdi] one advantage of "collection not typed, members typed"

[Drummond Reed <drummond.reed@xdi.org>]

On Sat, Mar 23, 2013 at 12:49 PM, Markus Sabadello <markus.sabadello@xdi.org> wrote:

In open issue #5, we're leaning toward option 2.

However I found one use case where option 1 works better.

That use case is mapping any arbitrary JSON structure to XDI.

This could be useful when implementing XDI connectors for external APIs that are based on JSON.

Example:

Currently, the following example JSON structure:

{

    "firstName": "John",

    "lastName": "Smith",

    "age": 25,

    "address": [

        "91 Western Road, Brighton, East Sussex, England, BN1 2NW",

        {

            "streetAddress": "21 2nd Street",

            "city": "New York",

            "state": "NY",

            "postalCode": "10021"

        }

    ]

}

Could be mapped to XDI as follows:

$!(+(age))/$is+/+$json$number

$!(+(age))/!/(data:,25)

$!(+(firstName))/!/(data:,John)

$!(+(lastName))/!/(data:,Smith)

$(+(address))$!(!cQlI_cKyMIfD3rCbHkWFRQUJtVLyfV1zJ3vKUPSSWvA)/!/(data:,91%20Western%20Road,%20Brighton,%20East%20Sussex,%20England,%20BN1%202NW)

$(+(address))$(!kClyz4ARycEJmZX5A7NIoVjStRf-jSGejXPtiRPv_Q4)$!(+(city))/!/(data:,New%20York)

$(+(address))$(!kClyz4ARycEJmZX5A7NIoVjStRf-jSGejXPtiRPv_Q4)$!(+(postalCode))/!/(data:,10021)

$(+(address))$(!kClyz4ARycEJmZX5A7NIoVjStRf-jSGejXPtiRPv_Q4)$!(+(state))/!/(data:,NY)

$(+(address))$(!kClyz4ARycEJmZX5A7NIoVjStRf-jSGejXPtiRPv_Q4)$!(+(streetAddress))/!/(data:,21%202nd%20Street)

This is possible, because the $(+(address)) collection can have one attribute member and one entity member at the same time.

With the proposed change to option 2, this would no longer be possible (or be more difficult).

I think then the way to do it would be to have TWO collections, i.e. one attribute collection and one entity collection, but I am not sure if that is very clean.

Therefore, although initially I also preferred option 2, I think for me this is reason enough to now switch to option 1.

 

Markus, what this comes down to is that JSON arrays allow any ordered set of values -- the values do not need to be of the same type.

It seems to me that with XDI collections, we either need to follow that rule -- that members can be of any type -- or follow the rule that members are of the same type.

If members of a collection can be of any type, I'm not sure we need the concept of collections at all. That just seems like an entity.

If members of a collection must be of the type defined by the collection, e.g., all members of a collection of email addresses must be an email address, then we only need to type the collection, which is why yesterday we were choosing option #2 (typed collections and untyped members).

Seen in this light, I actually don't think we have any choice. In the XDI graph model, the meaning of a collection is that it is a list (either unordered OR ordered) of members of the type defined by the collection.

Given our decision yesterday that the XDI graph model should incorporate the six literal data types of JSON (counting "true" and "false" as one boolean data type), which I think was a good and important decision, I thought a lot on my run this morning about the differences between the XDI graph model structure and primary JSON data structures -- objects, arrays, pairs, values. My conclusion was that, as much as I'd like to align them (JSON objects are a lot like XDI entities; JSON arrays are a lot like XDI collections; JSON pairs are a lot like XDI attributes; JSON values are now essentially synonymous with XDI literals), they don't actually align precisely.

The reason is that we're solving different problems. The goal of the XDI graph model is to produce a globally normalized, addressable semantic graph model for data. The goal of JSON is to be a lightweight, programming-language and transport-protocol neutral data interchange format.

The goals are very synergistic, which is why there is such a great fit between XDI and JSON. But they are still different goals, which explains why XDI has stricter data normalization rules than JSON.

To bring this back to your use case, the JSON "address":  array in your example has addresses in two formats -- one a string, and one a JSON object. That's allowable in JSON, but IMHO should not be allowable in an XDI collection.

So that means that to do automatic conversion of a JSON array to an XDI collection, we'd have to specify the following rules:

1. If all of the elements of the JSON array are of the same JSON data type:
   
1. Represent the JSON array as an XDI collection.
   
2. Represent each element of the JSON array as an ORDERED member of the XDI collection (since JSON arrays are ordered).
2. If any of the elements of the JSON array are of a different JSON data type:
   
1. Represent the JSON array as an XDI entity.
2. To this entity, add a collection called {$array} (we need to decide on $words for JSON data types).
3. Add ORDERED members corresponding to the JSON element order ( the plain [digit] ordering should work well for this).
4. For ordered member:
1. For all simple JSON data types, add an attribute representing the JSON data type, e.g., <$string>, and then the JSON array element value as a literal.
2. For complex JSON data types (objects and arrays), continue with your JSON-to-XDI conversion rules. 

This seems clean and capable of full round-trip conversion where needed.

=Drummond