FYI: XML and ASN.1 in practice...

[Eliot Christian <echristi@usgs.gov>]

Here is an example of exactly the kind of "best of both"
solutions we should all have in mind to pursue, IMHO.

>-----Original Message-----
>From: Dalton, Barnaby
>Sent: Friday, November 14, 2003 8:44 AM
>To: 'asn1xml@oss.com'
>Subject: [asn1xml] Could we use ASN.1?
>
>Hi All,
>
>I'm involved with the development of a specification for the broadcast
>of an Electronic Programme Guide (EPG) over DAB Digital Radio
>(http://www.worlddab.org/). The current EPG specification defines the
>contents of the EPG using an XML Schema. However, the method of
>transport is currently in flux. As the XML files can be quite large
>and bandwidth usage must be minimised the current test setup uses gzip
>to compress the files which are then sent via DAB's MOT (Multimedia
>Object Transport) protocol.
>
>The EPG needs to be decoded by low end receivers (some with only 10's
>of KBytes of space avaiable for EPG decoding and storage). Gzip/XML
>decoding doesn't map terribly well to these resource constrained
>receivers so alternative ways of compressing the content are being
>sought.
>
>I was wondering if ASN.1 encoding could be used to reduce the
>bandwidth requirements compared to XML while also enabling small
>footprint decoding on low end receivers. It seems the proposed X.694
>standard might be just what we're looking for as it defines a mapping
>between XML schemas and ASN.1.
>
>I have found some information on the web but am still a bit in the
>dark and was wondering if anyone could answer the following questions:
>
>How do I get hold of X.694?
>How near completion is it? What are the timelines?
>Is the conversion of Schema to ASN.1 automatic or are manual steps
>required? Are there tools availabe to perform the conversion from
>Schema to ASN.1? If the Schema is changed how is this handled in
>ASN.1? Would previous decoders still work? What would be required to
>take an existing XML file conforming to the EPG Schema and encode it
>using PER or other encoding? Is there a company that could do this for
>me?
>
>Thanks,
>Barney

>From: "Bob Wyman" <bob@wyman.us>
>To: <asn1xml@oss.com>
>Subject: RE: [asn1xml] Could we use ASN.1?
>Date: Fri, 14 Nov 2003 11:21:39 -0500
>
>Barnaby Dalton wrote:
> > As the XML files can be quite large and bandwidth usage
> > must be minimised the current test setup uses gzip to
> > compress the files
>         This sounds to me like an excellent opportunity to use *both*
>XML and ASN.1... What I would propose is that you define your abstract
>data model using ASN.1 and include in that definition the encoding
>control notation needed to generate good XML. Given such a definition,
>encoders and decoders for both XML and ASN.1 binary forms can be
>automatically generated. Thus, you would be able to support XML for
>those who require it and are willing to deal with the bulk of XML
>encodings while also offering an encoding as compressed as PER for
>those who require compact encodings.
>         My company is currently building such a "dual encoding" system
>and we're finding it to be surprisingly pain-free. Our code only deals
>with the abstract data model while we rely on automatically generated
>encoders/decoders to transform what passes over the wire to and from
>the internal form that we process against. The XML and ASN.1
>interfaces are logically and semantically identical, the only
>difference between them is the encoding format. Basically, if you
>connect to us using XML, we respond using XML, if you make a request
>using ASN.1, we respond in ASN.1. The result is that encoding format
>becomes a question of "link tuning" and configuration -- it simply
>isn't a subject of interest when dealing with design issues. And, we
>don't have *any* coders who are worrying about getting all the
>angle-brackets in the right place or figuring out how to do binary
>encodings...
>         One nice aspect of the system is that since the XML and ASN.1
>interfaces are semantically identical, when debuggging, we can simply
>switch to XML mode and easily detect any logic errors in our code
>since the XML is so easy for humans to read. Then, when we want to get
>speed and compactness, we just set a bit in our code and we're back to
>binary encodings.
>         I strongly recommend that you consider implementing your
>system using "dual encodings." It really gives you the best of both
>worlds without causing problems of interoperability.
>
> >How do I get hold of X.694?
>         What you're really interested in is X.693, which defines the
>XER or XML Encoding Rules. The X.694 standard defines the mapping of
>XML Schema (WXS) to ASN.1. However, X.693 is the really important
>standard for an effort like yours unless you are defining your schemas
>using XML Schema.
>         Unfortunately, while both X.693 and X.694 have been finalized,
>they aren't published yet. So, to get copies during the next few weeks
>you would have to connect to an ITU member. However, for study
>purposes, you might want to look at the old X.693 that shows the
>simple XML encoding rules. You can find it, and other ASN.1 related
>specifications, at:
>http://www.itu.int/ITU-T/studygroups/com17/languages/
>         Note: Some of the updates that will become "official" soon
>have already been pre-published. Thus, you'll find that the updates to
>X.680, X.681, X.690, and X.691 are already available. The existing
>X.693 should be a good introduction. Then, the updated X.693 and X.694
>should be available there soon.
>         You might also want to take a look at John Larmouth's and
>Olivier Dubussion's books on ASN.1. (Unfortunately, neither covers
>X.693 and/or X.694 -- they are a bit dated.) Check out:
>http://www.oss.com/asn1/dubuisson.html.
>http://www.oss.com/asn1/larmouth.html
>         And, don't forget the ASN.1 project site and the great France
>Telecom site (which includes translators from XML Schema to ASN.1...)
>See:
>http://asn1.elibel.tm.fr/en/index.htm
>http://www.asn1.org/
>And the home page for the ASN.1 project at the ITU (the standards
>body)
>http://www.itu.int/ITU-T/asn1/index.html
>
> > Is the conversion of Schema to ASN.1 automatic
> > or are manual steps required?
>         Conversion is completely automatic and deterministic.
>Encoders/Decoders are generated automatically from the ASN.1 schema
>definition. You don't need to write *any* encode or decode code.
>
> > Are there tools availabe to perform the
> > conversion from Schema to ASN.1?
>         Such tools have been delivered for use for almost 20 years
>now. For a list of tools and vendors see: (some free, some commercial
>-- some good, some not so good...)
>http://asn1.elibel.tm.fr/en/links/#tools
>
> > If the Schema is changed how is this handled in ASN.1?
> > Would previous decoders still work?
>         Schemas can be written in a manner that allows them to be
>extensible (within reason!). Extensibility is not normally a problem
>unless you do a *really* bad job of defining your problem up front.
>
> > What would be required to take an existing XML file conforming
> > to the EPG Schema and encode it using PER or other encoding?
>         If you have an XML Schema that describes EPG Schema, you just
>convert it to ASN.1 using the rules in X.694 (or the tool on France
>Telecom's site), and then you compile the ASN.1 specification. Once
>you've done that, you have a working system. I do this regularly. It
>takes maybe five minutes to go from having an XML Schema to having
>working dual-encoding code that can read ASN.1 and write XML or read
>XML and write ASN.1... It's all done automatically.
>
>Can you provide a pointer to your EPG Schema? I couldn't find it on
>the web site you pointed to. However, given some stuff we're doing, we
>would be very interested in anything that makes it easier to get
>program guides -- especially if we can encode it using ASN.1...
>
>                 bob wyman