Re: [emergency-cap] CAP Issue 23 - Specify JSON serialization

[Art Botterell <acb@incident.com>]

Eliot -

That's another straw man, I'm afraid.  No CAP implementation would be required to implement JSON, either.

Nor do I agree with your interpretation of the spec as limiting the use of ASN.1 to "internal" applications.  As I understood it the TC's position has always been not to specify implementation internals at all.  While I agree the language may be a bit clumsy, my own understanding was always that the intent was to allow the use of ASN.1 intra-system... because otherwise there's be no logical reason to single out ASN.1 at all.  

Anyway, at this late date I have little patience for legalistic arguments.  The spec can say whatever we're wise enough to make it say.  To the extent the current spec can be misinterpreted as binding CAP to XML in any permanent fashion, I believe the spec needs to be corrected.  Otherwise we're on the road to arguing in defense of sperm-oil lighting and the SAME protocol.  

Technology advances and the marketplace notoriously routes around reactionary policies.  Do we seriously imagine that warning systems are going to rely on XML forever?  They won't, you know.  The only real question is whether CAP will be part of their future.

We didn't come this far by standing on precedent or fearing the occasional imperfection that are the price of progress.  I like to think that the world is better off for our having dared to advocate for improvement.

- Art

> On Oct 30, 2014, at 20:20, Eliot Christian <eliot.j.christian@gmail.com> wrote:
> 
> Art,
> 
> The specification for CAP states: "CAP is an XML-based format". Because the XML Schema for CAP (given in section 3.4 of the specification) is normative, only a message that is valid against that schema is properly a CAP message. 
> 
> Regarding ASN.1, Section 3.5 of the CAP version 1.2 specification states "The ASN.1 (see ITU-T Rec X.680) schema in 3.5.3 provides an alternative formulation of the XML 505 schema defined in 3.4." It further states: "Implementations can produce and process the CAP alert XML messages using either ASN.1-based or XSD-based tools (or other ad hoc software)." 
> 
> So, the use of ASN.1 (or any format other than XML) is allowed only for system-internal processing of CAP alerts. The given ASN.1 Schema is normative only in the case that ASN.1 is used to process messages that will be ingested into a system as CAP alerts or emitted from a system as CAP alerts. That ASN.1 Schema is provided to assure that conversions from ASN.1 to XML or XML to ASN.1 are loss-less.
> 
> I thought everyone well understood that the use of ASN.1 is entirely optional; no CAP implementation is required to implement ASN.1.
> 
> Eliot
>  
> 
> 
> On Thu, Oct 30, 2014 at 8:55 PM, Art Botterell <acb@incident.com> wrote:
> Por favor, Señor Eliot, leave those windmills alone!  Nobody said anything about advocating JSON "over" XML.  This proposal is precisely congruent with our earlier adoption of ASN.1... which you supported, if memory serves.  So I'm still curious as to how you stand on ASN.1 today.
> 
> Meanwhile, though, I'll take issue with a couple of your broad assertions.
> 
> First off, XML is not and never has been a fundamental aspect of CAP.  We've always discussed CAP in terms of a data model that could be serialized in a number of ways.  XML was what was happening at the time, but we were well aware of its limitations and peculiarities even in those younger days.
> 
> And I fear you're missed the point, or at least seized the wrong end of the stylus, when you talk about doing transformations between serializations.  That way lies spaghetti-code.  As you'll recall from the example of the Java caplib, a decade and more ago, a more scalable approach is to implement a common object model with separate methods to marshal and demarshal (serialize and parse) that common object to and from various serial formats.  Again, the abstraction of the object model is the essence that facilitates everything else.
> 
> While I feel your frustration with Tomcat, I'm sure you realize that it proves nothing as regards the current discussion.  What you or I personally can manage technically at this late point in our careers is of less significance every day.  What matters is what works for the implementers of the future.  My sense, having spent some quality time with them down at CMU-SV in Mountain View, is that they're voting with their feet, or at least their fingertips.  My goal is to get out of their way.
> 
> And it would be a waste of time to offer an extended a JSON tutorial in the OASIS record; there's plenty of information online for those of inquisitive spirit, and the folks who specialize in software are generally well aware of it.  My personal sense is that JSON provides all the flexibility of XML with less complexity and thus less risk.  Of course your individual mileage may vary.
> 
> - Art
> 
> > On Oct 29, 2014, at 12:54, Eliot Christian <eliot.j.christian@gmail.com> wrote:
> >
> > Art,
> >
> > My point is not really any kind of complaint about JSON. My point is that changing between data formats is always a very dangerous step.
> >
> > I recall when XML was being defined--lots of really smart people thought they had all the bases covered. To minimize complications, they even adopted wholly what was supposedly an already mature technology (language-independent data types) . Yet, it took XML more than ten years to mature into a format that in actual practice could be finally regarded as stable enough to trust with life-critical applications like CAP.
> >
> > In actual practice, programmers use all kinds of mechanisms to pass structures, and I'll bet various of these raise many subtle problems you never would have thought could be an issue. (Maybe this example would be of interest.) Just a couple weeks ago I found that Tomcat by default does not respect UTF-8 encoding whenever parameters are passed via HTTP--just amazing to me this could be the case in 2014!!
> >
> > Before the CAP SC could seriously consider the benefits versus risk associated with advocating JSON over XML, someone will have to compile a thorough document on the topic of how to get "CAP in XML" transformed from and to "CAP in JSON" (at both the XML schema and message instance levels). I suppose there are lots of tools to perform such transformations auto-magically, but I'm willing to bet that few if any of those have been rigorously tested and proven as absolutely loss-less in both directions. But, I am ready to be convinced if someone can show that such work has in fact been done.
> >
> > So, no matter what you or I think the eventual outcome will be, the next step in the CAP SC is that someone has to make the case for why we should "fix" what is perhaps the most fundamental aspect of CAP, especially as no one says that aspect is "broken" but just that it is  "un-trendy".
> >
> > Eliot
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Oct 29, 2014 at 11:17 AM, Art Botterell <acb@incident.com> wrote:
> > If anything the character-encoding situation is a bit simpler in JSON... most things are.  However, there are validated solutions for XML internationalization as well... the problem being that many (probably most) implementers aren't fully abreast of them.  "I18N" is still viewed in many English-speaking software shops as an arcane specialty.  Programmers in other parts of the world tend, in my experience, to be a bit more cognizant.
> >
> > You do raise an interesting additional question, however, which is whether it might be beneficial to specify the character encoding, optionally like the language, in the CAP message body itself.  How does ASN.1 address that, I wonder?
> >
> > - Art
> >
> > > On Oct 29, 2014, at 07:20, Eliot Christian <eliot.j.christian@gmail.com> wrote:
> > >
> > > There continue to be nasty issues cropping up in the details, especially wrt character encoding. Although not much of an issue for those whose applications just use ASCII, but it's a major pain for the rest of the world's languages. I would be surprised if applications using JSON do not have character encoding problems (indeed, such problems bedevil XML as well). Such problems may not be specific to the serialization itself, but how serialization and structure passing interacts with a programming language, a Web container, an operating system, HTML, and so forth.
> > >
> >
> >
> 
>