Re: [xdi] Minutes: XDI TC Telecon Friday 2015-03-27

["=Drummond Reed" <drummond@respectnetwork.com>]

Joseph, indeed, it was my recollection that you were the first one to propose tilde as the relativity symbol. This wouldn't be the first time (in fact, I believe it's either the third or fourth time) that it's taken the rest of the TC time to catch up with you ;-)

Thanks,

=Drummond  

On Sat, Aug 22, 2015 at 3:01 PM, Joseph Boyle <planetwork@josephboyle.net> wrote:

This is when I proposed the tilde for local identifiers earlier:

On Mar 29, 2015, at 5:53 PM, =Drummond Reed <drummond@respectnetwork.com> wrote:

XDI TC Minutes

Following are the minutes of the unofficial telecon of the XDI TC at:

Date:  Friday, 27 March 2015 USA
Time:  09:00AM - 10:30AM Pacific Time (16:00-17:30 UTC)

ATTENDING

Drummond Reed
Markus Sabadello
Peter Davis
Joseph Boyle
Phil WIndley

PRESENTATIONS/DISCUSSIONS

Review of the XDI Core Spec Sections

On Tuesday Drummond uploaded another draft of the next sections of XDI Core—now finished through the Roots section. You can see where this section fits in the full outline of XDI Core 1.0:

https://wiki.oasis-open.org/xdi/CoreSpecOutline

Markus already sent feedback to this latest draft, other TC members are encouraged to do the same.

Drummond review the key new sections with the group, and noted that he will incorporate all of Markus’ suggested revisions into the next draft.

Parentheses and the Local Identifier Syntax Question

Joseph has voiced the concern that the current proposed ABNF syntax “overloads” parentheses syntax with several meanings. After the StarShift, which removed nested XDI addresses, the following table summarizes the current uses of parentheses syntax:

Usage	Syntax	Example
peer root	(_)	(=example)
URI peer root	(scheme:_)	(http://foo.example.com/)
inner root	(_/_)	(=example/#friend)
URI ID	x(scheme:_)	=(http://foo.example.com/foo/)
local ID	x(_)	=(foo)

Joseph has suggested that at least one of these could be replaced by another syntax to reduce the overloading of parentheses. Drummond feels strongly that root syntax should stay consistent, and further that the URI ID syntax is consistent with that. That leaves only local ID as a candidate for using a different syntax (the red text in the table above).

One option Joseph has sugested would be to introduce a “local symbol” with usage similar to the immutability symbol “!”, i.e., it would appended after the context symbol (and the optional immutability symbol) to indicate that the identifier scope was local and not global.

Here are examples of options for a local symbol character with examples so you can see how they “read” (the current parentheses syntax is included on the last row for comparison):

Local Symbol	= Example	+ Example	# Example	* Example
~	=~foo	+~foo	#~foo	*~foo
_	=_foo	+_foo	#_foo	*_foo
-	=-foo	+-foo	#-foo	*-foo
.	=.foo	+.foo	#.foo	*.foo
x( )	=(foo)	+(foo)	#(foo)	*(foo)

Immutable + Local Symbol	= Example	+ Example	# Example	* Example
!~	=!~foo	+!~foo	#!~foo	*!~foo
!_	=!_foo	+!_foo	#!_foo	*!_foo
!-	=!-foo	+!-foo	#!-foo	*!-foo
!.	=!.foo	+!.foo	#!.foo	*!.foo
x!( )	=!(foo)	+!(foo)	#!(foo)	*!(foo)

Joseph’s Email to the List on this Topic

Instead of the symbols Drummond proposed for local identifiers, I think we should just partition the existing identifier space which would also fix some atypical features of XDI identifiers as currently defined.

Currently XDI identifiers can contain the characters A-Z a-z 0-9 (and the Letter and Number character classes of Unicode - we need to discuss and confirm that too) and four special characters:

- . _ ~

These have been carried forward without change from either:   (I’ve only ever seen tilde at the beginning to indicate a following Unix account name)

https://www.ietf.org/rfc/rfc3986.txt (which does not allow non-ASCII at all):

unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"

or from:

https://www.ietf.org/rfc/rfc3987.txt (which uses more generic criteria for Unicode non-ASCII characters):

 iunreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~" / ucschar

  ucschar        = %xA0-D7FF / %xF900-FDCF / %xFDF0-FFEF
                 / %x10000-1FFFD / %x20000-2FFFD / %x30000-3FFFD
                 / %x40000-4FFFD / %x50000-5FFFD / %x60000-6FFFD
                 / %x70000-7FFFD / %x80000-8FFFD / %x90000-9FFFD
                 / %xA0000-AFFFD / %xB0000-BFFFD / %xC0000-CFFFD
                 / %xD0000-DFFFD / %xE1000-EFFFD

  iprivate       = %xE000-F8FF / %xF0000-FFFFD / %x100000-10FFFD

RFC 3987 excludes only:

1. 32 unassigned code points just after ASCII
2. http://en.wikipedia.org/wiki/Private_Use_Area_(Unicode_block) (including emoji unless they’ve been moved)
3. 32 unassigned code points in http://en.wikipedia.org/wiki/Arabic_Presentation_Forms-A
4. http://en.wikipedia.org/wiki/Specials_(Unicode_block)
5. two unassigned code points at the end of each Supplementary Plane whose lower 16 bits look like Byte Order Mark

So XDI currently can have arcs like:

+-identifier  +.identifier  +_identifier  +~identifier    =-1   +.1   =_1   =~1

While underscore is universally categorized with letters and used as a spacer in identifiers, the other three are not; in fact tilde is almost always an operator.

Also note that usually decimal point, minus, and plus are considered parts of a number if they are followed by digits.

Proposal 1:  Make tilde the local context symbol; don’t allow tilde in identifiers.

identifier-char   =   ALPHA / DIGIT / "-" / "." / "_"

global-identifier =       1*identifier-char

local-identifier  =   “~” 1*identifier-char

Proposal 2:  Make identifiers starting with special characters local, and identifiers starting with alphanumeric global.

identifier-char   =   ALPHA / DIGIT / “-“ / “.” / “_” / “~"   

global-identifier =   (ALPHA / DIGIT)         1*identifier-char

local-identifier  =   (“-“ / “.” / “-“ / “~”) 1*identifier-char

Proposal 3+:  ????

identifier-char   =   ALPHA / DIGIT / “-“ / “.” / “_” / “~"   

global-identifier =   (????)         1*identifier-char

local-identifier  =   (????) 1*identifier-char

********* END OF JOSEPH’S EMAIL ***********

********* START OF TC DISCUSSION OF THE TOPIC ***********

We considered implications of this choice on XDI parser implementations. If parentheses can contain a URI or a local identifier, a parser has to parse the string up to at least the URI scheme to determine the difference.

Peter asked if the character classes being discussed would limit content inside JSON string values. Joseph said this is a different question, with payload literal data only constrained by JSON’s own rules - when the parser encounters such in a literal statement, it switches to (calls) a JSON parser (such as GSON in XDI2) which applies JSON parsing rules for the remainder of the datum. Joseph noted the similarity to the case of IRIs in XDI. (Does XDI2 incorporate some standard library for IRI parsing? Probably not.)

Peter asked, if the JSON serialization is going to be the only one used over the wire in production, doesn’t that make all these XDI expressions into JSON strings? Joseph answered: yes, they are transported in JSON strings while in the serialized form, but are not the same as literal value JSON values, and the set of characters allowed for XDI names (arc labels) is not related to the set of characters allowed in JSON strings in general, which is much broader, allowing any Unicode character except “ or \ or control characters.

We concluded that serialization rules are a different layer and independent of general XDI syntax rules.

As for proposing a new non-parenthetical syntax for local names, Joseph preferred his proposal 1, where ~ can only appear at the beginning of a name to indicate it is local, and is not allowed to appear farther along in the name. However, he said he was open to any non-parenthetical choice of local-prefix character if one enabled consensus allowing moving forward.

Joseph noted that ~ is a longstanding Unix convention for one’s home directory, and that this carried over to earlier internet providers where http://provider.com/~boyle would be the URL for user boyle’s private space for web pages and files. Peter noted this was still in use in some places.

Joseph noted this change would be made without breaking current implementations, unless they are using XDI names containing ~ and assuming those names are global, which is unlikely. Implementations can start recognizing the =~localname format and if necessary, continue to recognize the =(localname) format as local during a transition period.

Markus said his preference was to not make any changes. He felt introducing yet another character makes the syntax unnecessarily more complicated. Even though parentheses may be currently overloaded, in all cases their use is still similar insofar as they always indicate an “external” identifier being “imported” into XDI syntax.

Peter also supported not make any change. There have been many changes to the ABNF recently, so in order to not break existing implementations and data, any more changes should only be made for very good reasons.

Phil said he always disliked the overloading of parentheses, and he found their use for local identifiers confusing, however he also felt that the proposed alternatives were not attractive.

Drummond said the only choice he thought could work aesthetically was the . period, but he was also not convinced a change was necessary.

Valid characters for XDI identifiers

We further discussed which characters should be valid in XDI identifiers (the identifier-char ABNF rule).

Markus mentioned that in the currently deployed XRI infrastructure, there is a difference between what identifiers are allowed by the spec, and what identifiers can be registered by the XDI.org registry. For example, ~ is a valid character, but cannot be used in an identifier for registration. The registry explicitly “enables” certain scripts such as Hebrew.

Phil suggested we “reserve” the ~ tilde character for future use, i.e. not allow it the identifier-char rule.

Joseph thought we reached consensus that ~ does not need to be a name character and should be dropped from the list of four special characters ~ . ` _ allowed anywhere in XDI names, regardless if whether we are going to reuse it immediately or reserve it for future, and asked if any objections. None heard.

Markus noted underscore _ is not allowed in the registry. Joseph said this was anomalous as underscore is actually the most common non-alphanumeric character allowed in names in various languages. On the other hand, http://en.wikipedia.org/wiki/Domain_name#Technical_requirements_and_process says that hyphens are the only non-alphanum character allowed in a label, and is not allowed to begin or end a label. Period of course appears between labels in domain names.

Peter said that common control characters (e.g. bell) should not be allowed. Joseph re-emphasized that Unicode includes a lot of non-alphanumeric such as whitespace. Instead of creating a very detailed list of allowed characters in the ABNF (implying that implementations should check all characters against this list whenever parsing XDI, even on what should be a low-overhead pass-through), maybe one approach would be to allow all of (or most of) Unicode (which is what IRI is doing with the iunreserved rule), and to issue a “recommendation” [such as a SHOULD in the spec] that adopters draw only on Unicode’s Letter and Number character classes when creating and using XDI names.

We did not have time to discuss another issue mentioned in Joseph’s email: whether the fact that some XDI names are numbers has any implications we should think about.

XDI Messaging Spec

We did not discuss this topic.

$push Instead of $copy?

We did not discuss this topic.

Websocket Binding, Publish/Subscribe, $push

We did not discuss this topic.

NEXT CALL

The next call is next week at the regular time.